article thumbnail

Migrating Critical Traffic At Scale with No Downtime?—?Part 1

The Netflix TechBlog

Migrating Critical Traffic At Scale with No Downtime — Part 1 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. This approach has a handful of benefits.

Traffic 339
article thumbnail

Datadog Creates Scalable Data Ingestion Architecture

InfoQ

Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs. By Rafal Gancarz

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Data Reprocessing Pipeline in Asset Management Platform @Netflix

The Netflix TechBlog

Existing data got updated to be backward compatible without impacting the existing running production traffic. After reading the asset ids using one of the ways, an event is created per asset id to be processed synchronously or asynchronously based on the use case. Generally, this flow is used for small datasets.

Media 237
article thumbnail

Stream logs to Dynatrace with Amazon Data Firehose to boost your cloud-native journey

Dynatrace

Take the example of Amazon Virtual Private Cloud (VPC) flow logs, which provide insights into the IP traffic of your network interfaces. With this out-of-the-box support for scalable data ingest, log data is immediately available to your teams for troubleshooting and observability, investigating security issues, or auditing.

Cloud 252
article thumbnail

Stuff The Internet Says On Scalability For June 22nd, 2018

High Scalability

. $40 million : Netflix monthly spend on cloud services; 5% : retention increase can increase profits 25%; 50+% : Facebook's IPv6 traffic from the U.S, using them to respond to storage events on s3 or database events or auth events is super easy and powerful. There are more quotes, more everything.

Internet 113
article thumbnail

How Data Inspires Building a Scalable, Resilient and Secure Cloud Infrastructure At Netflix

The Netflix TechBlog

Challenges & Opportunities in the Infra Data Space Security Events Platform for Anomaly Detection How can we develop a complex event processing system to ingest semi-structured data predicated on schema contracts from hundreds of sources and transform it into event streams of structured data for downstream analysis?

article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

Under the hood, Titus is powered by Kubernetes , but it provides a thick layer of enhancements over off-the-shelf Kubernetes, to make it more observable , secure , scalable , and cost-efficient. Explainer flow is event-triggered by an upstream flow, such Model A, B, C flows in the illustration.

Systems 226