article thumbnail

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

article thumbnail

Sequential A/B Testing Keeps the World Streaming Netflix Part 1: Continuous Data

The Netflix TechBlog

These observations are from a particular type of A/B test that Netflix runs called a software canary or regression-driven experiment. An example data stream for an A/B test where each observation represents play-delay for the control (left) and treatment (right). Such changes should not be ignored by our testing procedure.

Testing 239
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Our First Netflix Data Engineering Summit

The Netflix TechBlog

Engineers from across the company came together to share best practices on everything from Data Processing Patterns to Building Reliable Data Pipelines. The result was a series of talks which we are now sharing with the rest of the Data Engineering community!

article thumbnail

Nurturing Design in Your Software Engineering Culture

Strategic Tech

There are a few qualities that differentiate average from high performing software engineering organisations. In my experience, the culture is better and the results are better in orgs where engineers and architects obsess over the design of code and architecture. They prefer to work in isolation and just deliver.

article thumbnail

Common Challenges in Digital Experience Testing and How to Overcome Them

DZone

Testing your digital platforms as part of a digital experience program is a vital element of ensuring that your customers have a seamless and user-friendly experience as they interact with your digital platforms. Of course, as with any other aspect of the testing process, some challenges can arise, but this is to be expected.

Testing 109
article thumbnail

What is platform engineering?

Dynatrace

Platform engineering is a practice that outlines how development teams build internal platforms to create self-service capabilities for software engineering teams. The result is a cloud-native approach to software delivery.

article thumbnail

All of Netflix’s HDR video streaming is now dynamically optimized

The Netflix TechBlog

A vital aspect of such development is subjective testing with HDR encodes in order to generate training data. The pandemic, however, posed unique challenges in conducting a conventional in-lab subjective test with HDR encodes. A/B testing also allows us to get a read on the improvement in quality of experience (QoE).