article thumbnail

Orchestrating Data/ML Workflows at Scale With Netflix Maestro

The Netflix TechBlog

We started seeing signs of scale issues, like: Slowness during peak traffic moments like 12 AM UTC, leading to increased operational burden. At Netflix, the peak traffic load can be a few orders of magnitude higher than the average load. Hence, the system has to withstand bursts in traffic while still maintaining the SLO requirements.

Java 202
article thumbnail

Experimentation is a major focus of Data Science across Netflix

The Netflix TechBlog

We don’t have unlimited traffic or time, so sometimes we have to make hard choices. The Open Connect engineering team faces numerous measurement challenges. Congestion can cause interactions between treatment and control groups; in other cases we are unable to randomize due to the nature of our traffic steering algorithms.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Achieving observability in async workflows

The Netflix TechBlog

Users of Prodicle: Production Office Coordinator on their job As the adoption of Prodicle grew over time, Productions asked for more features, which led to the system quickly evolving in multiple programming languages under different teams. Prodicle Distribution Our service is required to be elastic and handle bursty traffic.

Traffic 160
article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 100
article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 100
article thumbnail

Why Web Developers Choose PHP For Great Website Development?

Official Blog - World Web Technology

Choosing the correct programming language for web development is critical for producing outstanding websites. PHP is a popular server-side programming language due to its simplicity of use, vast functionality, and robust community support. In this post, we’ll look at why web professionals use PHP for unique website creation.

Website 59
article thumbnail

Sustainability at AWS re:Invent 2022 All the talks and videos I could find…

Adrian Cockcroft

Margaret leads the worldwide solution architect program for sustainability, and gives an excellent talk on how customers should think about optimizing their workloads. DOP315 Sustainability in the cloud with Rust and AWS Graviton  — Emil Lerch AWS Principal DevOps Specialist and Esteban Kuber AWS Principal Software Engineer.

AWS 64