The Netflix TechBlog

AUGUST 30, 2021

Complementing the hardware is the software on the RAE and in the cloud, and bridging the software on both ends is a bi-directional control plane. When a new hardware device is connected, the Local Registry detects and collects a set of information about it, such as networking information and ESN.

Latency 213

Latency Traffic Transportation Hardware 213

The Morning Paper

JANUARY 30, 2020

Seamless offloading of web app computations from mobile device to edge clouds via HTML5 web worker migration , Jeong et al., Edge servers are the middle ground – more compute power than a mobile device, but with latency of just a few ms. in the cloud). SoCC’19 [^1].

Mobile 104

Mobile Cloud Latency Games 104

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. Ford, et al., “TCP

Performance 140

Performance Latency Hardware Storage 140

The Morning Paper

OCTOBER 4, 2020

We are standing on the eve of the 5G era… 5G, as a monumental shift in cellular communication technology, holds tremendous potential for spurring innovations across many vertical industries, with its promised multi-Gbps speed, sub-10 ms low latency, and massive connectivity. The first 5G networks are now deployed and operational.

Energy 130

Energy Latency Performance Network 130

Sutter's Mill

FEBRUARY 13, 2017

Tue-Thu Apr 25-27: High-Performance and Low-Latency C++ (Stockholm). On April 25-27, I’ll be in Stockholm (Kista) giving a three-day seminar on “High-Performance and Low-Latency C++.”

Latency 51

Latency C++ Hardware Performance 51

The Netflix TechBlog

JUNE 4, 2019

When you’re running in the cloud your containers are in a shared space; in particular they share the CPU’s memory hierarchy of the host instance. These applications range from critical low-latency services powering our customer-facing video streaming service, to batch jobs for encoding or machine learning.

Cache 251

Cache Latency Airlines Logistics 251

The Morning Paper

NOVEMBER 15, 2020

So while scale out has seen the majority of attention in the cloud era, it’s good to remind ourselves periodically just what we really can do on a single box or even a single thread. An IDS/IPS monitors network flows and matches incoming packets (or more strictly, Protocol Data Units, PDUs) against a set of rules. IDS/IPS requirements.

Servers 128

Servers Hardware Latency Design 128

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Dotcom-Montior

NOVEMBER 24, 2021

There was a time when standing up a website or application was simple and straightforward and not the complex networks they are today. With the rapid advancements in web application technologies, programming languages, cloud computing services, microservices, hybrid environments, etc., The recipe was straightforward. Peer-to-Peer.

Systems 74

Systems Monitoring Hardware Network 74

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur. ASPLOS’19.

Big Data 45

Big Data Cloud Performance Hardware 45

Dotcom-Montior

MAY 12, 2020

Network or connection error. Network latency. Hardware resources. Network Latency. With the evolution of cloud technologies, such as Single Page Applications (SPAs), Web APIs, and Model View Controller (MVC), network latency has become a crucial factor to be monitored. HTTP error.

Monitoring 63

Monitoring Entertainment Hardware Latency 63

The Morning Paper

JANUARY 28, 2020

Shredder is " a low-latency multi-tenant cloud store that allows small units of computation to be performed directly within storage nodes. " Shredder is " a low-latency multi-tenant cloud store that allows small units of computation to be performed directly within storage nodes. " Shredder in action.

Serverless 99

Serverless Storage Latency Cloud 99

All Things Distributed

JULY 13, 2010

Expanding the Cloud - Cluster Compute Instances for Amazon EC2. Today, Amazon Web Services took very an important step in unlocking the advantages of cloud computing for a very important application area. Unlocking the benefits of the cloud for the HPC community. By Werner Vogels on 12 July 2010 05:00 PM. Comments ().

Cloud 118

Cloud AWS Automotive Latency 118

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. Ford, et al., “TCP

Performance 52

Performance Latency Hardware Storage 52

The Morning Paper

JUNE 13, 2019

As we saw with the SOAP paper last time out, even with a fixed model variant and hardware there are a lot of different ways to map a training workload over the available hardware. The following figure highlights how just one of these variables, batch size, impacts throughput and latency on ResNet50.

Hardware 58

Hardware Latency Serverless Energy 58

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

HammerDB

OCTOBER 23, 2023

With a simple example such as this, it would not necessarily be expected for the additional network traffic to be significant between the 2 approaches. However, with more complex application logic this network round trip soon becomes a key focus area for improving performance. On MySQL, we saw a 1.5X performance advantage. client (i.e.

Benchmarking 62

Benchmarking Database Network C++ 62

Dynatrace

AUGUST 5, 2019

Across the board, the topics cloud migration, application modernization, breaking the monolith or hybrid cloud re-platforming have been a center point in many of our discussions with our joint enterprise customers. If you can answer all these questions fine – if not: get your own Dynatrace Trial and start installing OneAgents.

Cloud 31

Cloud Traffic Database Network 31

The Morning Paper

MARCH 3, 2020

This work is latency critical, because volume IO is blocked until it is complete. Physalia is designed to offer consistency and high-availability, even under network partitions. Larger cells have better tolerance of tail latency (e.g. Thus the configuration master is under stress just when you need it the most.

Database 106

Database AWS Network Design 106

VoltDB

JUNE 1, 2017

It was – like the hypothetical movie I describe above – more than a little bit odd, as you could leave a session discussing ever more abstract layers of virtualization and walk into one where they emphasized the critical importance of pinning a network interface to a specific VM for optimal performance.

Hardware 56

Hardware Latency Virtualization Cloud 56

Alex Russell

MARCH 6, 2021

Thanks to progress in networks and browsers (but not devices), a more generous global budget cap has emerged for sites constructed the "modern" way: ~100KiB of HTML/CSS/fonts and ~300-350KiB of JS (compressed) is the new rule-of-thumb limit for at least the next year or two. Modern network performance and availability.

Performance 124

Performance Network Cache Metrics 124

VoltDB

JUNE 1, 2017

It was – like the hypothetical movie I describe above – more than a little bit odd, as you could leave a session discussing ever more abstract layers of virtualization and walk into one where they emphasized the critical importance of pinning a network interface to a specific VM for optimal performance.

Hardware 40

Hardware Latency Virtualization Cloud 40

The Morning Paper

APRIL 14, 2019

The paper sets out what we can do in software given today’s hardware, and along the way also highlights areas where cooperation from hardware will be needed in the future. There are also stateless interconnects including buses and on-chip networks. Microarchitectural channels. Threat scenarios.

Hardware 44

Hardware Cache Latency Speed 44

The Morning Paper

JULY 4, 2019

Netflix’s system is deployed on the public cloud as complex set of interacting microservices. degraded hardware, transient networking problem) or, more often, because of some change deployed by Netflix engineers that did not have the intended effect. On error rates.

Latency 77

Latency Engineering Metrics Traffic 77

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable.

Storage 115

Storage AWS Virtualization Cloud 115

Brendan Gregg

AUGUST 8, 2017

Load averages are an industry-critical metric – my company spends millions auto-scaling cloud instances based on them and other metrics – but on Linux there's some mystery around them. They can be also useful when a single value of demand is desired, such as for a cloud auto scaling rule. Network I/O?

Latency 111

Latency Metrics C++ Systems 111

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. A data pipeline is a software which runs on hardware. The software is error-prone and hardware failures are inevitable.

Latency 63

Latency Analytics Scalability Engineering 63

Sutter's Mill

JUNE 21, 2012

In the twilight of Moore’s Law, the transitions to multicore processors, GPU computing, and HaaS cloud computing are not separate trends, but aspects of a single trend—mainstream computers from desktops to ‘smartphones’ are being permanently transformed into heterogeneous supercomputer clusters.

Latency 40

Latency Transportation Hardware Education 40

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. Try to measure your mean time to respond (MTTR) for incidents.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. Try to measure your mean time to respond (MTTR) for incidents.

Latency 53

Latency Engineering Systems Hardware 53

O'Reilly Software

APRIL 4, 2018

A message-oriented implementation requires an efficient messaging backbone that facilitates the exchange of data in a reliable and secure way with the lowest latency possible. It enables unbounded scalability as more commodity or specialized hardware can be seamlessly added to existing clusters.

Architecture 80

Architecture Scalability Google Operating System 80

Abhishek Tiwari

DECEMBER 27, 2017

portability between cloud and on-premises. optimised container networking and security. Using default scheduler's node affinity feature you can ensure that certain pods only schedule on nodes with specialized hardware like GPU, memory-optimised, I/O optimised etc. better cluster resource utilization.

Big Data 63

Big Data Storage Benchmarking Hardware 63

The Morning Paper

SEPTEMBER 12, 2019

SQL provides a declarative programming interface, below which the system itself can figure out the most effective execution plans based on data size and statistics, layout, compute hardware etc. For the feed-forward neural network, TensorFlow using GPUs is still significantly faster with smaller hidden layers, but cannot scale.

Network 44

Network Database Programming Hardware 44

Brendan Gregg

SEPTEMBER 5, 2017

Here's some output from my zfsdist tool, in bcc/BPF, which measures ZFS latency as a histogram on Linux: # zfsdist. Tracing ZFS operation latency. Xen's biggest user is the Amazon EC2 cloud, which has over one million customers, and appears to be a custom version (it self identifies as version "3.4.3.amazon").

Virtualization 76

Virtualization AWS Engineering Hardware 76

Smashing Magazine

JANUARY 11, 2021

Networking, HTTP/2, HTTP/3 OCSP stapling, EV/DV certificates, packaging, IPv6, QUIC, HTTP/3. If you don’t have a device at hand, emulate mobile experience on desktop by testing on a throttled 3G network (e.g. Estimated Input Latency tells us if we are hitting that threshold, and ideally, it should be below 50ms. 300ms RTT, 1.6

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

If you don’t have a device at hand, emulate mobile experience on desktop by testing on a throttled 3G network (e.g. To make the performance impact more visible, you could even introduce 2G Tuesdays or set up a throttled 3G/4G network in your office for faster testing. 300ms RTT, 1.6 Mbps down, 0.8

Performance 145

Performance Cache Network Metrics 145

KeyCDN

AUGUST 15, 2019

Wait time: Sometimes called average latency, wait time refers the amount of time a request spends in a queue before it gets processed. The cloud-based service sends massive numbers of requests per second to simulate how your website holds up. Peak response time: The longest response time.

Testing Tools 55

Testing Tools Best Practices Performance Testing Testing 55