Remove Big Data Remove Infrastructure Remove Transportation Remove Tuning
article thumbnail

How Netflix uses eBPF flow logs at scale for network insight

The Netflix TechBlog

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 325
article thumbnail

Delta: A Data Synchronization and Enrichment Platform

The Netflix TechBlog

Data Pipeline using Delta In the following sections, we are going to describe the Delta-Connector that connects to a datastore and publishes CDC events to the Transport Layer, which is a real-time data transportation infrastructure routing CDC events to Kafka topics. Please stay tuned.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

article thumbnail

Data Movement in Netflix Studio via Data Mesh

The Netflix TechBlog

Data Mesh provides a drag-and-drop, self-service user interface for exploring sources and creating pipelines so that users can focus on delivering business value without having to worry about managing and scaling complex data streaming infrastructure. Please stay tuned! Endnotes ¹ Inmon, Bill. Dehghani, Zhamak.

Big Data 253