Efficiency, Transportation and Tuning - Technology Performance Pulse

How Netflix uses eBPF flow logs at scale for network insight

The Netflix TechBlog

JUNE 7, 2021

The sidecar has been implemented by leveraging the highly performant eBPF along with carefully chosen transport protocols to consume less than 1% of CPU and memory on any instance in our fleet. The choice of transport protocols like GRPC, HTTPS & UDP is runtime dependent on characteristics of the instance placement.

Network

Network Transportation AWS Cloud

Building Netflix’s Distributed Tracing Infrastructure

The Netflix TechBlog

OCTOBER 19, 2020

We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments. Our engineering teams tuned their services for performance after factoring in increased resource utilization due to tracing. Storage: don’t break the bank!

Infrastructure

Infrastructure Transportation Storage Open Source

Plan Your Multi Cloud Strategy

Scalegrid

MARCH 22, 2024

Choosing the Right Cloud Services Choosing the right cloud services is crucial in developing an efficient multi cloud strategy. Adopting spot instances for less critical tasks, which are less expensive than on-demand or reserved instances, is an efficient way of managing expenses.

Strategy

Strategy Cloud Government Innovation

Kubernetes vs Docker: What’s the difference?

Dynatrace

SEPTEMBER 29, 2021

Just like shipping containers revolutionized the transportation industry, Docker containers disrupted software. For a deeper look into how to gain end-to-end observability into Kubernetes environments, tune into the on-demand webinar Harness the Power of Kubernetes Observability. Watch webinar now! Networking.

Open Source

Open Source Traffic DevOps Cloud

Snap: a microkernel approach to host networking

The Morning Paper

NOVEMBER 10, 2019

I’m jumping ahead a bit here, but the component of Snap which provides the transport and communications stack is called Pony Express. Here are the bombshell paragraphs: Our datacenter applications seek ever more CPU-efficient and lower-latency communication, which Pony Express delivers. Enter Google! Emphasis mine).

Network

Network Transportation Latency Entertainment

Measuring Carbon is Not Enough?—?Unintended Consequences

Adrian Cockcroft

NOVEMBER 13, 2023

Buildings, food and transport have a much bigger carbon footprint than IT globally. In the simplest case, you have a growing workload, and you optimize it to run more efficiently so that you don’t need to buy or rent additional hardware, so your carbon footprint stays the same, but the carbon per transaction or operation is going down.

Energy

Energy Cloud Traffic Transportation

Data Mesh?—?A Data Movement and Processing Platform @ Netflix

The Netflix TechBlog

AUGUST 1, 2022

It reads events from the upstream transports and applies some business logic to each of them. An intermediate processor writes data to another transport. More Processing Patterns And Better Efficiency People use Data Mesh not only to move data. Please stay tuned! Processors A processor is a Flink Job.

Processing

Processing Transportation Entertainment Tuning

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure

Infrastructure Big Data Transportation Architecture

Data Movement in Netflix Studio via Data Mesh

The Netflix TechBlog

JULY 26, 2021

With dependable near real-time data, Studio teams are able to track and react better to the ever-changing pace of productions and improve efficiency of global business operations using the most up-to-date information. Please stay tuned! The audits check for equality (i.e. We will have follow up blog posts on these topics in future.

Big Data

Big Data Government Analytics Processing

HTTP/3 From A To Z: Core Concepts (Part 1)

Smashing Magazine

AUGUST 9, 2021

For example, while HTTP deals with URLs and data interpretation, Transport Layer Security (TLS) ensures security by encryption, TCP enables reliable data transport by retransmitting lost packets, and Internet Protocol (IP) routes packets from one endpoint to another across different devices in between (middleboxes). What Is QUIC?

Transportation

Transportation Internet Internet Network

DBLog: A Generic Change-Data-Capture Framework

The Netflix TechBlog

DECEMBER 17, 2019

The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. It is required for a database to execute this query efficiently, which typically applies for systems that implement range scans over primary keys. A transaction log is assumed to be the origin of those events.

Database

Database Traffic Transportation Open Source

DBLog: A Generic Change-Data-Capture Framework

The Netflix TechBlog

DECEMBER 17, 2019

The database is sending them to a transport that DBLog can consume. We use the term ‘ change log’ for that transport. It is required for a database to execute this query efficiently, which typically applies for systems that implement range scans over primary keys. A transaction log is assumed to be the origin of those events.

Database

Database Traffic Transportation Open Source

HTTP/3: Performance Improvements (Part 2)

Smashing Magazine

AUGUST 22, 2021

An often used metaphor is that of a pipe used to transport water. One aspect of performance is about how efficiently a transport protocol can use a network’s full (physical) bandwidth (i.e. As such, tuning congestion logic is usually only done by a select few developers, and evolution is slow. Congestion Control.

Performance

Performance Network Latency Servers

HTTP/3: Practical Deployment Options (Part 3)

Smashing Magazine

SEPTEMBER 6, 2021

For example, you could reduce compression efficiency , because that works better with more data. As such, a micro-optimization is, again, how you probably need to fine-tune things on a low level to really benefit from it. This is true, but only to a relatively limited extent. What Does It All Mean? Clients and QUIC Discovery.

Network

Network Servers Cache Traffic

Technology Performance Pulse

How Netflix uses eBPF flow logs at scale for network insight

Building Netflix’s Distributed Tracing Infrastructure

Trending Sources

Plan Your Multi Cloud Strategy

Kubernetes vs Docker: What’s the difference?

Snap: a microkernel approach to host networking

Measuring Carbon is Not Enough?—?Unintended Consequences

Data Mesh?—?A Data Movement and Processing Platform @ Netflix

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

Data Movement in Netflix Studio via Data Mesh

HTTP/3 From A To Z: Core Concepts (Part 1)

DBLog: A Generic Change-Data-Capture Framework

DBLog: A Generic Change-Data-Capture Framework

HTTP/3: Performance Improvements (Part 2)

HTTP/3: Practical Deployment Options (Part 3)

Stay Connected