The Netflix TechBlog

SEPTEMBER 18, 2020

How does a decision of this scale affect the efficiency of our globally distributed content delivery network, Open Connect ? Streaming DSE is at the center of it all, bringing together different teams at Netflix and using data to drive decisions that impact our members around the world. benefit more?

Analytics 223

Analytics Education Innovation Engineering 223

O'Reilly

FEBRUARY 2, 2020

Is it important to observe what happens on each layer of a neural network? Collaboration between AI developers and operations teams will lead to growing pains on both sides, especially since many data scientists and AI researchers have had limited exposure to, or knowledge of, software engineering.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

The Netflix TechBlog

DECEMBER 18, 2020

I completed a combined math bachelors + masters program, but without any professional guidance, networking, or internships, I was entirely lost. When it was time to pick a major, I was struggling in every subject except one: Math. I had the piece of paper, but what next? I held plenty of jobs as a student, but now I needed a career.

Analytics 138

Analytics Retail Education Engineering 138

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Abhishek Tiwari

JULY 12, 2018

Unfortunately, building data pipelines remains a daunting, time-consuming, and costly activity. Not everyone is operating at Netflix or Spotify scale data engineering function. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines.

Latency 63

Latency Analytics Scalability Engineering 63

The Netflix TechBlog

NOVEMBER 22, 2019

4pm-5pm OPN 303-R BPF Performance Analysis Brendan Gregg , Senior Performance Engineer Abstract : Extended BPF (eBPF) is an open-source Linux technology that powers a whole new class of software: mini programs that run on events. We share everything attendees need to implement CloudTrail in their own organizations.

AWS 37

AWS Entertainment Open Source Benchmarking 37

The Symphonia

JUNE 19, 2018

This summer also marks the 4-yearly event that is La Copa Mundial (we only get Telemundo in my apartment, not Fox Sports Network) but since the good old US of A are absent from the men’s World Cup this year, football fever is distinctly frigid. Unlike the temperature outside. John and I (Mike) have had a fun three months.

Serverless 40

Serverless AWS DevOps Open Source 40

O'Reilly

FEBRUARY 18, 2020

The results for data-related topics are both predictable and—there’s no other way to put it—confusing. Starting with data engineering, the backbone of all data work (the category includes titles covering data management, i.e., relational databases, Spark, Hadoop, SQL, NoSQL, etc.). This follows a 3% drop in 2018.

Software Architecture 134

Software Architecture DevOps Data Engineering Architecture 134

The Netflix TechBlog

DECEMBER 21, 2020

Some of the optimizations are prerequisites for a high-performance data warehouse. Sometimes Data Engineers write downstream ETLs on ingested data to optimize the data/metadata layouts to make other ETL processes cheaper and faster. AutoOptimize reduces end to end lag in data processing by optimizing as we go.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

OCTOBER 30, 2020

Through a combination of my existing network, the interview process, and other online resources about the company, I found that folks are actually very friendly and helpful! Phrases like “high performance” and “dream team” made me imagine an almost gladiator-style workplace. But I quickly learned this wasn’t the case.

Analytics 136

Analytics Engineering Innovation Metrics 136

Abhishek Tiwari

DECEMBER 27, 2017

Data solution vendors like SnapLogic and Informatica are already developing machine learning and artificial intelligence (AI) based smart data integration assistants. These assistants can recommend next-best-action or suggest datasets, transforms, and rules to a data engineer working on a data integration project.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

O'Reilly

MARCH 2, 2020

Frequent contact with customers, good in-person communications between team members, along with practices like source control and testing, would just be in the air, like our Wi-Fi networks. Radar data points: Recent research and analysis. Key survey results: The C-suite is engaged with data quality. They’d simply be what we do.

Artificial Intelligence 21

Artificial Intelligence Software Architecture Programming Infrastructure 21

The Netflix TechBlog

JANUARY 11, 2022

To learn about Analytics and Viz Engineering, have a look at Analytics at Netflix: Who We Are and What We Do by Molly Jackman & Meghana Reddy and How Our Paths Brought Us to Data and Netflix by Julie Beckley & Chris Pham. Curious to learn about what it’s like to be a Data Engineer at Netflix?

Innovation 243

Innovation Metrics Engineering Testing 243