article thumbnail

3. Psyberg: Automated end to end catch up

The Netflix TechBlog

This helps overwrite data only when required and minimizes unnecessary reprocessing. As seen above, by chaining these Psyberg workflows, we could automate the catchup for late-arriving data from hours 2 and 6. The Data Engineer does not need to perform any manual intervention in this case and can thus focus on more important things!

Tuning 244
article thumbnail

A Day in the Life of an Experimentation and Causal Inference Scientist @ Netflix

The Netflix TechBlog

At Netflix, our data scientists span many areas of technical specialization, including experimentation, causal inference, machine learning, NLP, modeling, and optimization. Together with data analytics and data engineering, we comprise the larger, centralized Data Science and Engineering group.

Analytics 207
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

How Our Paths Brought Us to Data and Netflix

The Netflix TechBlog

Julie] Chris and I have the same primary stakeholders (or engineering team that we support): Encoding Technologies. They are continuously innovating compression algorithms to efficiently send high quality audio and video files to our customers over the internet. Tell me about some of the exciting projects you’re a part of.

Analytics 223
article thumbnail

Sustainability at AWS re:Invent 2022 All the talks and videos I could find…

Adrian Cockcroft

I asked around and heard that they are still working on it, but the AWS hiring freeze means that they don’t have the headcount they expected and are making slow progress on an API, more detailed metrics, and scope 3, which everyone is waiting for. Portfolio is currently reducing Amazons carbon footprint by 19 Million Metric Tons of CO2e.

AWS 64
article thumbnail

A Day in the Life of a Content Analytics Engineer

The Netflix TechBlog

I’m a Senior Analytics Engineer on the Content and Marketing Analytics Research team. My team focuses on innovating and maintaining the metrics Netflix uses to understand performance of our shows and films on the service. One of my favorite things about being an Analytics Engineer is the variety.

Analytics 136
article thumbnail

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

article thumbnail

Organise your engineering teams around the work by reteaming

Abhishek Tiwari

For instance, if you are fast-growing VC funded e-commerce startup and your number one business priority is multiplying current growth and performing exceptionally well on key financial metrics charted out by your investors. I also have a strong feeling that long-lived teams are not good for innovation and disruption.