The Netflix TechBlog

JULY 15, 2021

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin, what drew you to data engineering?

Data Engineering 189

Data Engineering Engineering Entertainment Big Data 189

The Netflix TechBlog

JUNE 1, 2021

Data Engineers of Netflix?—?Interview Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. What drew you to Netflix?

Data Engineering 130

Data Engineering Engineering Big Data Healthcare 130

The Netflix TechBlog

NOVEMBER 20, 2023

For example, a job would reprocess aggregates for the past 3 days because it assumes that there would be late arriving data, but data prior to 3 days isn’t worth the cost of reprocessing. Backfill: Backfilling datasets is a common operation in big data processing. data arrives too late to be useful).

Processing 210

Processing Big Data Efficiency Engineering 210

The Netflix TechBlog

MAY 26, 2020

These characteristics allow for an on-call response time that is relaxed and more in line with traditional big data analytical pipelines. Spark could look up and retrieve the data in the s3 files that the Mouthful represented. And excellent logging is needed for debugging purposes and supportability.

Network 150

Network Tuning AWS Big Data 150

All Things Distributed

OCTOBER 7, 2015

Today, I am excited to share with you a brand new service called Amazon QuickSight that aims to simplify the process of deriving insights from a wide variety of data sources in a fast and affordable manner. Big data challenges. We believe this is one of the critical parts of our big data offerings.

Cloud 137

Cloud Big Data AWS Analytics 137

Abhishek Tiwari

DECEMBER 27, 2017

In 2018, we will see new data integration patterns those rely either on a shared high-performance distributed storage interface ( Alluxio ) or a common data format ( Apache Arrow ) sitting between compute and storage. For instance, Alluxio, originally known as Tachyon, can potentially use Arrow as its in-memory data structure.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

The Netflix TechBlog

SEPTEMBER 10, 2019

This enables us to optimize their experience at speed. Instead of relying on engineers to productionize scientific contributions, we’ve made a strategic bet to build an architecture that enables data scientists to easily contribute. Our data scientists faced numerous challenges in our previous infrastructure.

Metrics 215

Metrics Architecture Infrastructure Innovation 215