The Netflix TechBlog

OCTOBER 27, 2021

Requirements for a Metaflow GUI Metaflow is a human-centered system by design. Instead of a fancy GUI for demos and presentations, we wanted a serious productivity tool with carefully thought-out user workflows that would fit seamlessly into our toolchain of data science. The GUI would be targeted at professional data scientists.

Open Source 222

Open Source Monitoring Scalability Code 222

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Wednesday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Wednesday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

Rigor

SEPTEMBER 11, 2019

Presented here alphabetically by last name: Rachel Andrew. Rick is a software engineer on the Google Chrome team, “leading an effort to make the web just work for developers.” Here is our (ever-growing!) list of those who are making a significant impact on speeding up the web today. Rachel Andrew. Rick Byers.

Performance 89

Performance Education Google Website 89

Strategic Tech

JANUARY 19, 2020

How we spend our time and apply our resources when developing software systems is possibly the most fundamental and difficult challenge. A natural tendency for software engineers is to gravitate towards the most technically interesting challenges. Time and resources are limited. A concept known as the Core Domain.

Open Source 87

Open Source Games Innovation Strategy 87

ACM Sigarch

APRIL 8, 2019

Below we present some thoughts on the why and one solution to address the issue. Her current work focuses on hardware/software co-design for extremely large-scale deep learning training. Lena Olson is a Software Engineer at Google. . Why is that? Should we be alarmed as a community? Lack of Exposure.

Architecture 74

Architecture Open Source Hardware Software Engineering 74

The Morning Paper

JANUARY 21, 2020

All the criteria are present for such systems to be classified as complex adaptive systems : connectedness, diversity, adaptation, and interdependence. In summary, software engineers find themselves in a very unenviable position when attempting to resolve an outage with their service. Preventative design is not enough.

Internet 98

Internet Internet Strategy Energy 98

The Morning Paper

DECEMBER 3, 2019

component-based software development, reusable software catalogues etc. Then there’s the changing demographic of software building. Another data point from Miller’s presentation: in 2018 57-70% of the average software project comprises open source components, and that fraction is rising fast.

Open Source 52

Open Source Programming Speed Software 52

Alex Russell

JUNE 22, 2022

Contrary to claims of Apple partisans, iOS engine restrictions are not preventing a "takeover" by Chromium — at least that's not the primary effect. Apple uses its power over browsers to strip-mine and sabotage the web, hurting all engine projects and draining the web of future potential. Cui bono? — who benefits?

Engineering 110

Engineering Traffic Government Google 110

SQL Server According to Bob

OCTOBER 17, 2019

The iterator is designed to remain lock free whenever possible to avoid contention with active queries. Iteration key: It is more complicated than this (background gc capabilities) but for this discussion focus on the fact that the Lock object is kept alive in the hash table as long as one lock owner is present. link] [link].

Servers 40

Servers Software Engineering Database Monitoring 40

SQL Server According to Bob

JULY 4, 2016

This design allows the health monitor thread to provide rich output even when the overall server is under stress. The loop waits for the designated interval to elapse, outputs information and continues the loop. By design, any interval, which could have occurred during the shutdown, are not present in the XEL health session output.

Software Engineering 40

Software Engineering Servers Design Monitoring 40

The Morning Paper

NOVEMBER 21, 2019

It’s easy to make experimental design mistakes that invalidate your online controlled experiments. PlanAlyzer acts a bit like a linter for online experiment designs, where those designs are specified in the PlanOut language. We present the first approach for statically checking the internal validity of online experiments.

Programming 52

Programming Internet Internet Design 52

Tasktop

DECEMBER 10, 2019

Having a designated place to work, where I see other human beings, works much better for me, so I rent an office in the local town. It also makes it easier to be present with my children when I get home, and to separate work from family time (I try to leave the laptop in the car until they are in bed).

Games 40

Games Software Engineering DevOps Google 40

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. Wednesday?—?December

AWS 37

AWS Entertainment Open Source Benchmarking 37

ACM Sigarch

FEBRUARY 23, 2018

The rest of the article presents some examples of this behavior and how it impacts our field, whether we care to admit it or not. Present Bias. In Thaler’s book, he uses the term “present bias” to describe how people discount the future and give larger weight or value to decisions that are closer in time. Discounting the Past.

Architecture 47

Architecture Cache Hardware Virtualization 47

Tasktop

JULY 2, 2019

In my past career as a Software Engineer at a giant Fintech company, I seldom got the chance to talk with the Product Manager of my team. Execution is the work that happens once the development team starts working on an Epic from design to development & testing. Shared credit and downfall.

Development 40

Development FinTech Engineering Software Engineering 40

The Morning Paper

JULY 9, 2019

Last time out we looked at machine learning at Microsoft , where we learned among other things that using an online controlled experiment (OCE) approach to rolling out changes to ML-centric software is important. Online Controlled Experiments (OCEs) are becoming a standard operating procedure in data-driven software companies.

Metrics 42

Metrics Design Testing Software 42

Smashing Magazine

JUNE 28, 2021

Designing With Code: A Modern Approach To Design. The Rise Of Design Thinking As A Problem Solving Strategy. Drew McLellan: He’s a Senior Software Engineer for an agency called Echobind. But for me, I don’t have design sense. A little bit, but I’m not a designer by any means. Mike on Twitter.

Website 62

Website Design Open Source Software 62

SQL Server According to Bob

JANUARY 29, 2017

Recently I blogged about the design of of the debugger bridge and the use of LLDB. When loading it with LLDB the core dump would open but we could not symbolize stacks or see specific memory that was expected to be present in the dump. Bob Dorr – Principal Software Engineer SQL Server. Debugging Steps.

Servers 40

Servers Programming Media Open Source 40

SQL Performance

APRIL 17, 2020

SQL Server will need to check all the existing values to ensure no nulls are present. Special thanks to Panagiotis Antonopoulos (Principal Software Engineer) and Mirek Sztajno (Senior Program Manager) from the SQL Server product team for their assistance and guidance during the research and writing of this article.

Servers 63

Servers Database Testing Efficiency 63

SQL Server According to Bob

MARCH 27, 2017

When SQL Server is started on Linux the monitoring process (/opt/mssql/bin/sqlservr) is present. When designing the core-mini dump capabilities we looked at several solutions before landing on Breakpad. The following diagram could be a high level view of the Windows, ELF or Breakpad file as they are similar in design.

Servers 40

Servers Virtualization Storage Design 40

The Morning Paper

APRIL 16, 2019

We would like students to consider the performance characteristics of their systems, and our tests check that their designs attain reasonable run-time performance. In addition, another lesson here is that once we accept model checking is necessary, we need to design systems that are amenable to model checking.

Systems 43

Systems Efficiency Testing Strategy 43

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. All ML projects are software projects.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

SQL Server According to Bob

NOVEMBER 8, 2016

The primary design behind SQL Server’s, NVDIMM, Direct Access capabilities is to use the NVDIMM as a stable cache for the tail of log write activities. SQL Server 2016 is designed to detect non-volatile storage devices. The NV LC is only a cache so backup takes the LCs and restores the LCs just as it is designed to do today.

Cache 40

Cache Servers Media Storage 40

The Netflix TechBlog

FEBRUARY 9, 2021

In the old architecture, each of these requests resulted in an API call to authenticate the claims presented with the request, as shown: EdgePaas Enters the Picture To further complicate the situation, the Edge Engineering team was in the middle of migrating from an old API server architecture to a new PaaS-based approach.

Architecture 221

Architecture Latency Servers Website 221

Strategic Tech

JANUARY 22, 2024

The Warehousing Modernization Enabling Team (an AMET) has been established to guide modernization in the Warehousing domain, which consists of 100+ software engineers and a monolithic codebase. They need a more loosely coupled architecture and empowered teams. You can also experiment with various workshop formats.

Architecture 98

Architecture Metrics Logistics Strategy 98

Strategic Tech

AUGUST 26, 2018

A trip to the supermarket can teach you a lot about designing software systems and shaping teams to build them… I was recently in need of some chocolate sauce. The Chocolate Sauce Heuristic for Software Design There are a few lessons about software development we can learn from this story, but I want to focus on design.

Design 40

Design Software Architecture Architecture Software 40

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. The infrastructure should allow them to exercise their freedom as data scientists but it should provide enough guardrails and scaffolding, so they don’t have to worry about software architecture too much. It leverages elasticity of the cloud by design?—?both

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

FEBRUARY 9, 2021

The Growth Engineering team is responsible for executing growth initiatives that help us anticipate and adapt to this change. In particular, it’s our job to design and build the systems and protocols that enable customers from all over the world to sign up for Netflix with the plan features and incentives that best suit their needs.

Engineering 142

Engineering Scalability Architecture Innovation 142

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. The infrastructure should allow them to exercise their freedom as data scientists but it should provide enough guardrails and scaffolding, so they don’t have to worry about software architecture too much. It leverages elasticity of the cloud by design?—?both

Open Source 134

Open Source AWS Infrastructure Energy 134

The Netflix TechBlog

FEBRUARY 9, 2021

Before designing a solution it’s important to understand the main product requirements for such a feature: The content needs to be new, relevant, and regional (not all countries have the same catalogue). We need to be able to easily determine what imagery is present for a given platform, region, and language.

Engineering 165

Engineering Storage Latency Entertainment 165

O'Reilly

JULY 20, 2021

But we, as technologists, have typically ignored our own expectations when designing and building those devices. If the devices aren’t designed with those expectations in mind, they’re destined for the landfill. When designing an experience, you need to consider the identity context and where the experience will take place.

Google 121

Google Games Technology Technology 121

The Netflix TechBlog

MAY 30, 2019

Our very first mobile app is called Prodicle and was built for Android & iOS using the same reactive architecture in both platforms, which allowed us to build 2 apps from scratch in 3 months with 4 software engineers. the presentation logic should be inside a component or a class that represents a view and not in the Fragment.

Architecture 155

Architecture Mobile Code Testing 155

SQL Server According to Bob

DECEMBER 31, 2018

Present this help information”); LogToConsole(“”); LogToConsole(“EXAMPLE: MergeXEvents.exe <input spec> <output spec>”); LogToConsole(” MergeXEvents.exe ”C:tempXEvent_0_130494871596480000.xel” Bob Dorr – Principal Software Engineer SQL Server. Designer.cs.

Efficiency 40

Efficiency Code Servers Cache 40

The Netflix TechBlog

NOVEMBER 3, 2022

In the time since it was first presented as an advanced Mesos framework, Titus has transparently evolved from being built on top of Mesos to Kubernetes, handling an ever-increasing volume of containers. This blog post presents how our current iteration of Titus deals with high API call volumes by scaling out horizontally.

Cache 224

Cache Latency Traffic Systems 224

Abhishek Tiwari

NOVEMBER 3, 2018

When it comes to innovation, most of CMS solutions are constrained by their legacy architecture (read strong coupling between content management and content presentation) which makes it difficult to serve content to new types of emerging channels such as apps and devices. Without knowing we created a JAMstack for our client.

Systems 63

Systems Cache Website Network 63

The Netflix TechBlog

OCTOBER 16, 2019

This post is a high level overview of the design and architecture of Gutenberg. An important point to note is that Gutenberg is not designed as an eventing system?—?it Gutenberg, however, is designed for publishing and consuming an entire immutable view of a dataset. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187