article thumbnail

Rebuilding Netflix Video Processing Pipeline with Microservices

The Netflix TechBlog

This introductory blog focuses on an overview of our journey. Future blogs will provide deeper dives into each service, sharing insights and lessons learned from this process. Future blogs will provide deeper dives into each service, sharing insights and lessons learned from this process.

article thumbnail

What is platform engineering?

Dynatrace

In response to this shift, platform engineering is growing in popularity. The practice of platform engineering has evolved alongside the increasing complexity of cloud environments. Platform engineers design and implement these platforms, as well as ensure their security, scalability, and reliability.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

article thumbnail

Accelerate and empower Site Reliability Engineering with Dynatrace observability

Dynatrace

In this blog post, we’ll delve deeper into these categories to gain a comprehensive understanding of their significance and the challenges they present. Process Improvements (50%) The allocation for process improvements is devoted to automation and continuous improvement SREs help to ensure that systems are scalable, reliable, and efficient.

article thumbnail

2. Diving Deeper into Psyberg: Stateless vs Stateful Data Processing

The Netflix TechBlog

By Abhinaya Shetty , Bharath Mummadisetty In the inaugural blog post of this series, we introduced you to the state of our pipelines before Psyberg and the challenges with incremental processing that led us to create the Psyberg framework within Netflix’s Membership and Finance data engineering team.

article thumbnail

1. Streamlining Membership Data Engineering at Netflix with Psyberg

The Netflix TechBlog

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions. Let’s dive in! What is late-arriving data? Some techniques we used were: 1.

article thumbnail

Netflix Studio Engineering Overview

The Netflix TechBlog

Our mission in Studio Engineering is to build a unified, global, and digital studio that powers the effective production of amazing content. In an effort to effectively and efficiently produce this content we are looking to improve and automate many areas of the production process. link] Why Does Studio Engineering Exist?