The Netflix TechBlog

NOVEMBER 22, 2019

Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges. We showcase our case studies, open-source tools in benchmarking, and how we ensure that AWS cloud services are serving our needs without compromising on tail latencies.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges. We showcase our case studies, open-source tools in benchmarking, and how we ensure that AWS cloud services are serving our needs without compromising on tail latencies.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

This talk explores the journey, learnings, and improvements to performance analysis, efficiency, reliability, and security. Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges. In 2019, Netflix moved thousands of container hosts to bare metal.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Brendan Gregg

OCTOBER 8, 2018

Screenshot: tracing read latency for PID 181: # bpftrace -e 'kprobe:vfs_read /pid == 30153/ { @start[tid] = nsecs; } kretprobe:vfs_read /@start[tid]/ { @ns = hist(nsecs - @start[tid]); delete(@start[tid]); }'. Back then I could already tell if disks were seeking by interpreting iostat(1) output: seeing high disk latency but small I/O.

C++ 110

C++ Virtualization Programming Latency 110

Brendan Gregg

OCTOBER 7, 2018

Screenshot: tracing read latency for PID 181: # bpftrace -e 'kprobe:vfs_read /pid == 30153/ { @start[tid] = nsecs; } kretprobe:vfs_read /@start[tid]/ { @ns = hist(nsecs - @start[tid]); delete(@start[tid]); }'. Back then I could already tell if disks were seeking by interpreting iostat(1) output: seeing high disk latency but small I/O.

C++ 40

C++ Virtualization Programming Latency 40

Smashing Magazine

AUGUST 3, 2023

Each stage has its unique challenges and potential pitfalls, as other case studies show. You need to beware that slow server response times can significantly increase TTFB, often due to server overload, network issues, or un-optimized logic on the server side. Here’s a breakdown of the moving pieces.

Performance 75

Performance Cache Traffic Metrics 75

Highly Scalable

MAY 1, 2012

This approach often leads to heavyweight high-latency analytical processes and poor applicability to realtime use cases. This opportunity is leveraged in the following case study. Case Study. Case Study. Case Study. bits per unique value.

Analytics 191

Analytics Traffic Big Data Efficiency 191