The Netflix TechBlog

JUNE 7, 2021

Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices. The choice of transport protocols like GRPC, HTTPS & UDP is runtime dependent on characteristics of the instance placement.

Network 325

Network Transportation AWS Cloud 325

The Netflix TechBlog

OCTOBER 19, 2020

This was the most important question we considered when building our infrastructure because data sampling policy dictates the amount of traces that are recorded, transported, and stored. Our engineering teams tuned their services for performance after factoring in increased resource utilization due to tracing.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

OCTOBER 15, 2019

Data Pipeline using Delta In the following sections, we are going to describe the Delta-Connector that connects to a datastore and publishes CDC events to the Transport Layer, which is a real-time data transportation infrastructure routing CDC events to Kafka topics. Please stay tuned.

Transportation 192

Transportation Architecture Processing Storage 192

Adrian Cockcroft

NOVEMBER 13, 2023

Buildings, food and transport have a much bigger carbon footprint than IT globally. I’ve written before about how to tune out retry storms. For a sense of proportion, a single economy flight between the USA and Europe is order-of-magnitude a ton of carbon.

Energy 52

Energy Cloud Traffic Transportation 52

The Netflix TechBlog

MARCH 25, 2019

In the push model paradigm, various platform tools such as the data transportation layer, reporting tools, and Presto will publish lineage events to a set of lineage related Kafka topics, therefore, making data ingestion relatively easy to scale improving scalability for the data lineage system.

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

DECEMBER 17, 2019

There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS. The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. The destination may be a datastore or an external API. Supporting Relational Databases.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS. The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. The destination may be a datastore or an external API. Supporting Relational Databases.

Database 212

Database Traffic Transportation Open Source 212