article thumbnail

Rebuilding Netflix Video Processing Pipeline with Microservices

The Netflix TechBlog

Future blogs will provide deeper dives into each service, sharing insights and lessons learned from this process. The Netflix video processing pipeline went live with the launch of our streaming service in 2007. The Netflix video processing pipeline went live with the launch of our streaming service in 2007.

article thumbnail

Data Mesh?—?A Data Movement and Processing Platform @ Netflix

The Netflix TechBlog

A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A stream processing) is one of the key factors that enable Netflix to maintain its leading position in the competition of entertaining our users.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Evolving from Rule-based Classifier: Machine Learning Powered Auto Remediation in Netflix Data…

The Netflix TechBlog

We have deployed Auto Remediation in production for handling memory configuration errors and unclassified errors of Spark jobs and observed its efficiency and effectiveness (e.g., For efficient error handling, Netflix developed an error classification service, called Pensive, which leverages a rule-based classifier for error classification.

Tuning 210
article thumbnail

The right person at the right time makes all the difference: Best practices for ownership information

Dynatrace

Secondly, knowing who is responsible is essential but not sufficient, especially if you want to automate your triage process. How to efficiently introduce team ownerships Dynatrace provides different ways of associating team ownership with entities and adding desired team metadata, such as contact details, to your environments.

article thumbnail

Conducting log analysis with an observability platform and full data context

Dynatrace

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. ” A data warehouse, on the other hand, is an efficient and fast option for querying data.

Analytics 195
article thumbnail

Dynatrace observability is now available for Red Hat OpenShift on the IBM® Power® architecture

Dynatrace

By leveraging the Dynatrace Operator and Dynatrace capabilities on Red Hat OpenShift on IBM Power, customers can accelerate their modernization to hybrid cloud and increase operational efficiencies with greater visibility across the full stack from hardware through application processes.

article thumbnail

Practical API Design at Netflix, Part 1: Using Protobuf FieldMask

The Netflix TechBlog

When we process a request it is often beneficial to know which fields the caller is interested in and which ones they ignore. By default, gRPC uses protobuf as its IDL (interface definition language) and data serialization protocol. Our protobuf message definition (.proto FieldMask is a protobuf message.

Design 245