article thumbnail

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

article thumbnail

What is a data lakehouse? Combining data lakes and warehouses for the best of both worlds

Dynatrace

Data lakehouses typically provide support for data ingestion through a variety of methods. This data lands in its original, raw form without requiring schema definition. A data lakehouse provides a cost-effective storage layer for both structured and unstructured data. Data warehouses.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Conducting log analysis with an observability platform and full data context

Dynatrace

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. They enable IT teams to identify and address the precise cause of application and infrastructure issues.

Analytics 193
article thumbnail

Data Movement in Netflix Studio via Data Mesh

The Netflix TechBlog

Data Mesh provides a drag-and-drop, self-service user interface for exploring sources and creating pipelines so that users can focus on delivering business value without having to worry about managing and scaling complex data streaming infrastructure. The audits check for equality (i.e.

Big Data 253
article thumbnail

Building a Rule-Based Platform to Manage Netflix Membership SKUs at Scale

The Netflix TechBlog

These rule definitions are hosted in a separate git repository and Hendrix module within SKUService load and refresh it periodically. The changes are administered by the regular git pull request flow and guarded by the validation infrastructure. Consumers pass a map of context to be used as rules evaluation criteria.

Mobile 209
article thumbnail

Reimagining Experimentation Analysis at Netflix

The Netflix TechBlog

Our A/B tests range across UI, algorithms, messaging, marketing, operations, and infrastructure changes. Instead of relying on engineers to productionize scientific contributions, we’ve made a strategic bet to build an architecture that enables data scientists to easily contribute. Getting Data with the Metrics Repo 2.

Metrics 215
article thumbnail

Ensuring Performance, Efficiency, and Scalability of Digital Transformation

Alex Podelko

So you need to understand what is going on there – and Debbie is definitely an authority in that area. How to select appropriate IT Infrastructure to support Digital Transformation by Boris Zibitsker, BEZNext. – Optimizing IT infrastructure – with specific use cases. Something we all struggle with.