Remove Database Remove Latency Remove Traffic Remove Video
article thumbnail

Data Reprocessing Pipeline in Asset Management Platform @Netflix

The Netflix TechBlog

Production Use Cases Real-Time APIs (backed by the Cassandra database) for asset metadata access don’t fit analytics use cases by data science or machine learning teams. Existing data got updated to be backward compatible without impacting the existing running production traffic.

Media 237
article thumbnail

Migrating Critical Traffic At Scale with No Downtime?—?Part 2

The Netflix TechBlog

Migrating Critical Traffic At Scale with No Downtime — Part 2 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Picture yourself enthralled by the latest episode of your beloved Netflix series, delighting in an uninterrupted, high-definition streaming experience. This is where large-scale system migrations come into play.

Traffic 279
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Rebuilding Netflix Video Processing Pipeline with Microservices

The Netflix TechBlog

The Netflix video processing pipeline went live with the launch of our streaming service in 2007. This architecture shift greatly reduced the processing latency and increased system resiliency. For example, in Reloaded the video quality calculation was implemented inside the video encoder module.

article thumbnail

Building Netflix’s Distributed Tracing Infrastructure

The Netflix TechBlog

Investigating a video streaming failure consists of inspecting all aspects of a member account. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

article thumbnail

Percentiles don’t work: Analyzing the distribution of response times for web services

Adrian Cockcroft

There is no way to model how much more traffic you can send to that system before it exceeds it’s SLA. I presented this analysis of response time distributions talk in 2016 — at Microxchg in Berlin ( video ). Mu is the mean of each component, the latency. I’ve been thinking about this for a long time.

Lambda 98
article thumbnail

Netflix Video Quality at Scale with Cosmos Microservices

The Netflix TechBlog

Moorthy and Zhi Li Introduction Measuring video quality at scale is an essential component of the Netflix streaming pipeline. Perceptual quality measurements are used to drive video encoding optimizations , perform video codec comparisons , carry out A/B testing and optimize streaming QoE decisions to mention a few.

Media 171
article thumbnail

Achieving observability in async workflows

The Netflix TechBlog

Once you finally find useful identifiers, you may begin writing SQL queries against your production database to find out what went wrong. Prodicle Distribution Our service is required to be elastic and handle bursty traffic. Things got hairy. We wanted a scalable service that was near real-time, 2.

Traffic 160