article thumbnail

Data Engineers of Netflix?—?Interview with Kevin Wylie

The Netflix TechBlog

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin, what drew you to data engineering?

article thumbnail

Data Engineers of Netflix?—?Interview with Samuel Setegne

The Netflix TechBlog

Data Engineers of Netflix?—?Interview Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. What drew you to Netflix?

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Ready-to-go sample data pipelines with Dataflow

The Netflix TechBlog

The most commonly used one is dataflow project , which helps folks in managing their data pipeline repositories through creation, testing, deployment and few other activities. It lets you create YAML formatted mock data files based on selected tables, columns and a few rows of data from the Netflix data warehouse.

article thumbnail

Data pipeline asset management with Dataflow

The Netflix TechBlog

see “data pipeline” Intro The problem of managing scheduled workflows and their assets is as old as the use of cron daemon in early Unix operating systems. The design of a cron job is simple, you take some system command, you pick the schedule to run it on and you are done. workflow ?—?see Example: 0 0 * * MON /home/alice/backup.sh

Storage 201
article thumbnail

Experimentation is a major focus of Data Science across Netflix

The Netflix TechBlog

Here we describe the role of Experimentation and A/B testing within the larger Data Science and Engineering organization at Netflix, including how our platform investments support running tests at scale while enabling innovation. Curious to learn about what it’s like to be a Data Engineer at Netflix?

article thumbnail

A Day in the Life of an Experimentation and Causal Inference Scientist @ Netflix

The Netflix TechBlog

Together with data analytics and data engineering, we comprise the larger, centralized Data Science and Engineering group. Learning through data is in Netflix’s DNA. We use A/B tests to introduce new product features, such as our daily Top 10 row that help our members discover their next favorite show.

Analytics 207
article thumbnail

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

IPS enables users to continue to use the data processing patterns with minimal changes. Introduction Netflix relies on data to power its business in all phases. Users configure the workflow to read the data in a window (e.g. data arrives too late to be useful). past 3 hours or 10 days).