Remove AWS Remove Cloud Remove Transportation Remove Tuning
article thumbnail

How Netflix uses eBPF flow logs at scale for network insight

The Netflix TechBlog

Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices. The choice of transport protocols like GRPC, HTTPS & UDP is runtime dependent on characteristics of the instance placement.

Network 325
article thumbnail

Building Netflix’s Distributed Tracing Infrastructure

The Netflix TechBlog

This was the most important question we considered when building our infrastructure because data sampling policy dictates the amount of traces that are recorded, transported, and stored. Our engineering teams tuned their services for performance after factoring in increased resource utilization due to tracing.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Measuring Carbon is Not Enough?—?Unintended Consequences

Adrian Cockcroft

This is basically the position of the major cloud providers, they will all take care of it for you by 2030 or so. Buildings, food and transport have a much bigger carbon footprint than IT globally. I’ve written before about how to tune out retry storms. But you want to *do something* to help sooner.

Energy 52
article thumbnail

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

In the rest of this blog, we will a) touch on the complexity of Netflix cloud landscape, b) discuss lineage design goals, ingestion architecture and the corresponding data model, c) share the challenges we faced and the learnings we picked up along the way, and d) close it out with “what’s next” on this journey.

article thumbnail

DBLog: A Generic Change-Data-Capture Framework

The Netflix TechBlog

There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS. The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. Proceedings of the Third ACM Symposium on Cloud Computing. Supporting Relational Databases.

Database 197
article thumbnail

DBLog: A Generic Change-Data-Capture Framework

The Netflix TechBlog

There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS. The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. Proceedings of the Third ACM Symposium on Cloud Computing. Supporting Relational Databases.

Database 212