The Netflix TechBlog

MARCH 5, 2019

While our engineering teams have and continue to build solutions to lighten this cognitive load (better guardrails, improved tooling, …), data and its derived products are critical elements to understanding, optimizing and abstracting our infrastructure. In the Reliability space, our data teams focus on two main approaches.

Infrastructure 182

Infrastructure Cloud Scalability AWS 182

The Netflix TechBlog

MARCH 7, 2024

Berg , Romain Cledat , Kayla Seeley , Shashank Srikanth , Chaoying Wang , Darin Yu Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications from our internal infrastructure and content demand modeling to media understanding.

Systems 226

Systems Media Cache Open Source 226

The Netflix TechBlog

MARCH 2, 2021

Stephanie Lane , Wenjing Zheng , Mihir Tendulkar Source credit: Netflix Within the rapid expansion of data-related roles in the last decade, the title Data Scientist has emerged as an umbrella term for myriad skills and areas of business focus. Learning through data is in Netflix’s DNA. It can be hard to know from the outside.

Analytics 207

Analytics C++ Innovation Engineering 207

Abhishek Tiwari

JULY 12, 2018

Building data pipelines can offer strategic advantages to the business. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines. Data pipeline initiatives are generally unfinished projects. In this post, we will discuss why you should avoid building data pipelines in first place.

Latency 63

Latency Analytics Scalability Engineering 63

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. Key challenges. Performance.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Dynatrace

JULY 6, 2022

Scripts and procedures usually focus on a particular task, such as deploying a new microservice to a Kubernetes cluster, implementing data retention policies on archived files in the cloud, or running a vulnerability scanner over code before it’s deployed. Big data automation tools. Batch process automation.

Artificial Intelligence 188

Artificial Intelligence Tuning Strategy Big Data 188

High Scalability

APRIL 14, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Try out their platform. Please apply here.

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! Who's Hiring? Please apply here. Apply here.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 30, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! . this is going to be a challenging journey for any backend engineer! They break down interview prep into fundamental building blocks. Try out their platform. Please apply here.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 3, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how world-class tech companies crush the hiring game! Apply here.

Education 114

Education Engineering Games Java 114

High Scalability

MARCH 17, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

FEBRUARY 18, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how world-class tech companies crush the hiring game! Apply here.

Education 113

Education Engineering Games Java 113

High Scalability

FEBRUARY 9, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how world-class tech companies crush the hiring game! Apply here.

Education 108

Education Engineering Games Java 108

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 100

AWS Entertainment Open Source Benchmarking 100

O'Reilly

FEBRUARY 2, 2020

First, the behavior of an AI application depends on a model , which is built from source code and training data. A model isn’t source code, and it isn’t data; it’s an artifact built from the two. You need a repository for models and for the training data. Models almost certainly react to incoming data; that’s their point.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

The Agile Manager

JULY 31, 2022

There are shadow IT teams of developers or data engineers that spring up in areas like operations or marketing because the captive IT function is slow, if not outright incapable, of responding to internal customer demand. There are also shadow activities of large software delivery programs.

Programming 59

Programming Engineering Data Engineering Innovation 59

Adrian Cockcroft

FEBRUARY 13, 2023

STP213 Scaling global carbon footprint management — Blake Blackwell Persefoni Manager Data Engineering and Michael Floyd AWS Head of Sustainability Solutions. SUS312 How innovators are driving more sustainable manufacturing  — Marcus Ulmefors Northvolt Director Data and ML Platforms and Muhammad Sajid AWS SA. AWS Inferentia 2.6x

AWS 64

AWS Energy Architecture Programming 64

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 202

Java Scalability Traffic Architecture 202

Testsigma

NOVEMBER 28, 2019

International Institute For Software Testing. Udacity Udacity provides nanodegree programs on all automation languages like C++, Machine Learning, Data engineer, Robotics and more. They have diploma courses in Software Testing, Introduction to Python, Diploma in Design Thinking, Diploma in Python Programming.

Website 53

Website Testing Programming Automotive 53

O'Reilly

FEBRUARY 18, 2020

It’s also the data source for our annual usage study, which examines the most-used topics and the top search terms. [1]. This year’s growth in Python usage was buoyed by its increasing popularity among data scientists and machine learning (ML) and artificial intelligence (AI) engineers.

Software Architecture 134

Software Architecture DevOps Data Engineering Architecture 134

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint. Wednesday?—?December

AWS 37

AWS Entertainment Open Source Benchmarking 37

High Scalability

JANUARY 22, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

FEBRUARY 6, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

NOVEMBER 12, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

OCTOBER 29, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

JANUARY 7, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

DECEMBER 12, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Who's Hiring? Apply here.

Education 21

Education Java Software Engineering Engineering 21

The Netflix TechBlog

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

Processing 210

Processing Big Data Efficiency Engineering 210

The Netflix TechBlog

JUNE 6, 2022

By Astha Singhal , Lakshmi Sudheer , Julia Knecht The Application Security teams at Netflix are responsible for securing the software footprint that we create to run the Netflix product, the Netflix studio, and the business. Our customers are product and engineering teams at Netflix that build these software services and platforms.

Software Engineering 169

Software Engineering Scalability Education Engineering 169

The Netflix TechBlog

JULY 21, 2022

We at Netflix, as a streaming service running on millions of devices, have a tremendous amount of data about device capabilities/characteristics and runtime data in our big data platform. With large data, comes the opportunity to leverage the data for predictive and classification based analysis.

Big Data 179

Big Data Cache Engineering Data Engineering 179

O'Reilly

JULY 15, 2020

Success with microservices means owning the software lifecycle. Most (74%) respondents say their teams own the build-test-deploy-maintain phases of the software lifecycle. Software engineers comprise the survey audience’s single largest cluster, over one quarter (27%) of respondents (Figure 1). Success with containers.

Database 136

Database Architecture Education Systems 136

All Things Distributed

OCTOBER 7, 2015

We live in a world where massive volumes of data are generated from websites, connected devices and mobile apps. In such a data intensive environment, making key business decisions such as running marketing and sales campaigns, logistic planning, financial analysis and ad targeting require deriving insights from these data.

Cloud 137

Cloud Big Data AWS Analytics 137