The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Dotcom-Montior

OCTOBER 6, 2021

A site reliability engineer, or SRE, is a role that that encompasses aspects of both software engineering and operations/infrastructure. The term site reliability engineering first came into existence at Google in 2003 when a site reliability team was created. At that time, the team was made up of software engineers.

Engineering 67

Engineering DevOps Monitoring Google 67

DZone

DECEMBER 8, 2021

Site Reliability Engineering (SRE) has grown immensely popular with many of the world’s largest tech companies, like Netflix, LinkedIn and Airbnb employing SRE teams to keep their systems reliable and scalable.

Engineering 245

Engineering Scalability Systems Traffic 245

VoltDB

DECEMBER 8, 2017

Virtualization of appliances and systems is seen as a necessary step to add the agility to meet these increasing and evolving service demands. Q2: How can you architect microservices and avoid increased latency due to multiple services replacing a monolithic service? Seeta Somagani] That’s a good question.

Latency 40

Latency Virtualization Cloud Software Engineering 40

VoltDB

DECEMBER 8, 2017

Virtualization of appliances and systems is seen as a necessary step to add the agility to meet these increasing and evolving service demands. Q2: How can you architect microservices and avoid increased latency due to multiple services replacing a monolithic service? Seeta Somagani] That’s a good question.

Latency 40

Latency Virtualization Cloud Software Engineering 40

The Morning Paper

JULY 7, 2019

Software engineering for machine learning: a case study Amershi et al., More specifically, we’ll be looking at the results of an internal study with over 500 participants designed to figure out how product development and software engineering is changing at Microsoft with the rise of AI and ML. ICSE’19.

Software Engineering 92

Software Engineering Engineering Software Software 92

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN.

AWS 37

AWS Entertainment Open Source Benchmarking 37

The Netflix TechBlog

NOVEMBER 3, 2022

As the number of Titus users increased over the years, the load and pressure on the system increased substantially. cell): Titus Job Coordinator is a leader elected process managing the active state of the system. For example, a batch workflow orchestration system may create multiple jobs which are part of a single workflow execution.

Cache 224

Cache Latency Traffic Systems 224

The Netflix TechBlog

FEBRUARY 9, 2021

The whole system was quite complex, and starting to become brittle. The API server orchestrates backend systems to authenticate the user. Upstream systems had to reopen the tokens to identify the user logging in and potentially manage multiple parallel identity data structures, which could easily get out of sync.

Architecture 221

Architecture Latency Servers Website 221

Dynatrace

JANUARY 18, 2023

Site reliability engineering (SRE) is a software operations methodology that enables organizations to create highly reliable and scalable applications. SRE applies software engineering principles to operations and infrastructure processes. Site reliability engineers, or SREs, lead these efforts.

DevOps 191

DevOps Best Practices Innovation Strategy 191

The Netflix TechBlog

MAY 14, 2021

However, they are scattered across multiple systems, and there isn’t an easy way to tie related messages together. You’re joining tables, resolving status types, cross-referencing data manually with other systems, and by the end of it all you ask yourself why? Things got hairy.

Traffic 160

Traffic Latency Java Google 160

Abhishek Tiwari

NOVEMBER 3, 2018

Recently I was asked about content management systems (CMS) of the future - more specifically how they are evolving in the era of microservices, APIs, and serverless computing. Using raw content data (such as Markdown, YAML, JSON files) and templates an SSG engine (such as Jekyll, Hugo, Gatsby, etc.)

Systems 63

Systems Cache Website Network 63

The Morning Paper

NOVEMBER 10, 2019

It’s been clear for a while that software designed explicitly for the data center environment will increasingly want/need to make different design trade-offs to e.g. general-purpose systems software that you might install on your own machines. The desire for CPU efficiency and lower latencies is easy to understand.

Network 92

Network Transportation Latency Entertainment 92

The Netflix TechBlog

JULY 12, 2019

Gatekeeper is the system at Netflix responsible for evaluating the “liveness” of videos and assets on the site. Gatekeeper accomplishes its prescribed task by aggregating data from multiple upstream systems, applying some business logic, then producing an output detailing the status of each video in each country.

Cache 183

Cache Architecture Latency Engineering 183

The Netflix TechBlog

FEBRUARY 9, 2021

The Growth Engineering team is responsible for executing growth initiatives that help us anticipate and adapt to this change. In particular, it’s our job to design and build the systems and protocols that enable customers from all over the world to sign up for Netflix with the plan features and incentives that best suit their needs.

Engineering 142

Engineering Scalability Architecture Innovation 142

The Netflix TechBlog

APRIL 26, 2022

Low-latency Queries To avoid downloading all of the fact data from s3 in a spark executor and then dropping it, we analyzed our query patterns and figured out that there is a way to only access the data that we are interested in. and we had no additional information to optimize our queries.

Storage 187

Storage Design Scalability Latency 187

The Morning Paper

JULY 4, 2019

Are you ready to take your system assurance programme to the next level? In all cases we need to be able to carefully monitor the impact on the system, and back out if things start going badly wrong. Netflix’s system is deployed on the public cloud as complex set of interacting microservices.

Latency 77

Latency Engineering Metrics Traffic 77

The Morning Paper

MARCH 3, 2020

In the same spirit as Paxos Made Live , this paper describes the details, choices and tradeoffs that are required to put a consensus system into production. This work is latency critical, because volume IO is blocked until it is complete. Larger cells have better tolerance of tail latency (e.g. A guiding principle.

Database 106

Database AWS Network Design 106

The Netflix TechBlog

AUGUST 16, 2023

System Metrics Given the significant reduction in connections, we saw reduced CPU utilization (~4%), heap usage (~15%), and latency (~3%) on Zuul, as well. In this case, we went from a subset size of 100 for 400 servers (a division of 4) to 50 (a division of 8).

Traffic 170

Traffic Servers Google Metrics 170

O'Reilly

NOVEMBER 12, 2019

More than a fifth of the respondents work in the software industry—skewing results toward the concerns of software companies, and helping explain the preponderance of those with software engineering roles. As noted earlier, the majority of survey respondents are software engineers.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

The Netflix TechBlog

FEBRUARY 9, 2021

Server-generated assets, since client-side generation would require the retrieval of many individual images, which would increase latency and time-to-render. To reduce latency, assets should be generated in an offline fashion and not in real time. Let’s put it all together and review the system interaction diagram.

Engineering 165

Engineering Storage Latency Entertainment 165

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN304 Learnings from migrating a service from JDK 8 to JDK 11 AWS Lambda improved latency by migrating to JDK 11 with Amazon Corretto.

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN304 Learnings from migrating a service from JDK 8 to JDK 11 AWS Lambda improved latency by migrating to JDK 11 with Amazon Corretto.

Open Source 40

Open Source AWS Lambda Serverless 40

O'Reilly

JULY 20, 2021

There may be alarm systems. There have been cases of harassment, intimidation, and domestic abuse by people whose access should have been revoked: for example, an ex-partner turning off the heating system. This is why people have made a business out of setting up home stereo and video systems.

Google 122

Google Games Technology Technology 122

The Morning Paper

JANUARY 26, 2020

In the context of the papers we’ve been looking at recently, and for a constrained environment, Reverb is helping its users to form an accurate mental model of the system state, and to form and evaluate hypotheses in-situ. candidate bug-fixes) during replay.

Programming 86

Programming Servers Network Latency 86