Data Engineering Design Systems Traffic 
The Netflix TechBlog
MARCH 7, 2024
The Machine Learning Platform (MLP) team at Netflix provides an entire ecosystem of tools around Metaflow , an open source machine learning infrastructure framework we started, to empower data scientists and machine learning practitioners to build and manage a variety of ML systems. ETL workflows), as well as downstream (e.g.
The Netflix TechBlog
OCTOBER 18, 2022
Due to its popularity, the number of workflows managed by the system has grown exponentially. We started seeing signs of scale issues, like: Slowness during peak traffic moments like 12 AM UTC, leading to increased operational burden. The scheduler on-call has to closely monitor the system during non-business hours.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
The Netflix TechBlog
JANUARY 11, 2022
To directly support great decision-making throughout the company, there are a number of data science teams at Netflix that partner directly with Product Managers, engineering teams, and other business units to design, execute, and learn from experiments. Curious to learn about what it’s like to be a Data Engineer at Netflix?
The Netflix TechBlog
NOVEMBER 22, 2019
Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December
The Netflix TechBlog
NOVEMBER 22, 2019
Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December
The Netflix TechBlog
MAY 26, 2020
VPC Flow Logs VPC Flow Logs is an AWS feature that captures information about the IP traffic going to and from network interfaces in a VPC. At Netflix we publish the Flow Log data to Amazon S3. As with any sustainable engineering design, focusing on simplicity is very important. 43416 5001 52.213.180.42
The Netflix TechBlog
NOVEMBER 22, 2019
Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December
Expert insights. Personalized for you.
226 
Let's personalize your content