Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

Percona

NOVEMBER 3, 2023

Rather than listing the concepts, function calls, etc, available in Citus, which frankly is a bit boring, I’m going to explore scaling out a database system starting with a single host. I won’t cover all the features but show just enough that you’ll want to see more of what you can learn to accomplish for yourself.

Database 107

Database Benchmarking Latency C++ 107

Dynatrace

DECEMBER 22, 2021

Complex information systems fail in unexpected ways. Observability gives developers and system operators real-time awareness of a highly distributed system’s current state based on the data it generates. With observability, teams can understand what part of a system is performing poorly and how to correct the problem.

Artificial Intelligence 175

Artificial Intelligence Metrics Architecture DevOps 175

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Amazon Elastic File System (EFS). Amazon Simple Storage Service (S3). Metrics for each service instance are presented in detailed charts—see the example for ECS below. Dynatrace news.

AWS 135

AWS Metrics IoT Storage 135

The Morning Paper

NOVEMBER 8, 2020

USENIX is a nonprofit organisation committed to making content and research freely available – both conference proceedings and the recorded presentations of their events. It’s such a powerful idea that I can imagine distributed systems implementers everywhere adopting it from now on. What does the VirtualLog give us?

Virtualization 126

Virtualization Latency Storage Systems 126

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Amazon Elastic File System (EFS). Amazon Simple Storage Service (S3). Metrics for each service instance are presented in detailed charts—see the example for ECS below. Dynatrace news.

AWS 132

AWS Metrics IoT Storage 132

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Percona

DECEMBER 26, 2023

The collected data is aggregated and presented in a single view. Dashboard example 2: Monitoring in real time These two panels show read/write IO performance to the persistent storage while benchmarking a live run. pg_stat_monitor About pg_stat_monitor is a Query Performance Monitoring tool for PostgreSQL.

Benchmarking 124

Benchmarking Metrics C++ Database 124

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

The Morning Paper

MARCH 17, 2020

Based on these findings, the authors present four guidelines to get the best performance out of this memory today. higher latency and lower bandwidth)… We have found the actual behavior of Optane DIMMs to be more complicated and nuanced than the "slower, persistent DRAM" label would suggest. Optane DIMMs are here!

Scalability 59

Scalability Latency Cache Media 59

Dotcom-Montior

MAY 12, 2020

Websites are now more than just the storage and retrieval of information to present content to users. They now allow users to interact more with the company in the form of online forms, shopping carts, Content Management Systems (CMS), online courses, etc. Network latency. Network Latency. The list goes on and on.

Monitoring 63

Monitoring Entertainment Hardware Latency 63

The Morning Paper

JUNE 13, 2019

HotOS’19 is presenting me with something of a problem as there are so many interesting looking papers in the proceedings this year it’s going to be hard to cover them all! Perhaps inspired by serverless in spirit and in terminology, the path forward proposed in this paper is towards a managed and model-less inference serving system.

Hardware 58

Hardware Latency Serverless Energy 58

The Morning Paper

DECEMBER 8, 2019

At the start of November I was privileged to attend HPTS (the High Performance Transaction Systems) conference in Asilomar. On the last morning of the conference Daniel Bittman presented some of the work being done in the context of the Twizzler OS project to explore new programming models for NVM.

Hardware 56

Hardware Programming Media Storage 56

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Operating System (OS) settings Swappiness Swappiness is a Linux kernel setting that influences the behavior of the Virtual Memory manager when it needs to allocate a swap, ranging from 0-100.

Best Practices 52

Best Practices Design Tuning Database 52

Percona

MAY 15, 2023

The main reason behind this is that MySQL is a relational database system (RDBMS), and any data that is going to be written in it must respect the RDBMS rules. In short, any data that is written must be consistent with the data present. In this common case, we do not need to implement a full sharding system such as Vitess.

Traffic 125

Traffic Scalability Database Servers 125

Adrian Cockcroft

APRIL 5, 2023

This proposal seeks to define a standard for real-time carbon and energy data as time-series data that would be accessed alongside and synchronized with the existing throughput, utilization and latency metrics that are provided for the components and applications in computing environments.

Energy 52

Energy Metrics Cloud Operating System 52

The Netflix TechBlog

JUNE 13, 2023

This is where large-scale system migrations come into play. Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. Canaries and sticky canaries are valuable tools in the system migration process.

Traffic 279

Traffic Metrics Systems Strategy 279

Testsigma

FEBRUARY 12, 2021

Presently we know it is far from easy to forecast the future – all of us have discovered this in 2020 through major ups and downs. The Internet of Things is generally referred to as IoT which encompasses computers, cars, houses or some other technological system related. IoT Test Automation. billion in 2016. billion in 2019 to $40.74

Artificial Intelligence 92

Artificial Intelligence Software Software IoT 92

O'Reilly Software

APRIL 30, 2018

If your organization relies on a single point of failure in terms of DNS, you’re open to system failure due to disasters of both technical and natural origins from power outages to sophisticated attacks. Modern hybrid applications typically utilize public cloud components, including content delivery networks and cloud storage.

Cloud 68

Cloud Traffic Internet Internet 68

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. It’s now time to put our race-logic encoded decision trees in the context of an end-to-end system. Boosted race trees for low energy classification Tzimpragos et al.,

Energy 52

Energy Hardware Efficiency Architecture 52

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Adrian Cockcroft

NOVEMBER 11, 2019

A resilient system continues to operate successfully in the presence of failures. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks. The first technique is the most generally useful.

Latency 52

Latency Engineering Systems Hardware 52

Alex Russell

MARCH 6, 2021

A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. This 2GiB RAM, Android 9 stalwart features the all-too classic lines of a Quad-core A53 (1.4GHz, small mercies) CPU, tastefully presented in a charming 5.5" The Moto G4 , for example.

Performance 124

Performance Network Cache Metrics 124

Alex Russell

JULY 14, 2021

The predominant desktop situation is relatively straightforward: Browsers handle links, and non-browsers defer loading http and https URLs to the system, which in turn invokes the default browser. Instead, they load content referred by their hosting native app in system-provided WebViews. The Baseline Scenario #.

Google 121

Google Mobile Engineering Internet 121

Adrian Cockcroft

NOVEMBER 11, 2019

A resilient system continues to operate successfully in the presence of failures. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks. The first technique is the most generally useful.

Latency 53

Latency Engineering Systems Hardware 53

HammerDB

OCTOBER 12, 2018

As with the previous guides as an Intel employee (#IAMINTEL) the examples are taken from a MySQL 8 on Linux on Intel system and the approach is the same for whatever system you are testing although some of the settings you see may be different. For this reason sysbench presents a vastly more simplified workload than HammerDB. .

Best Practices 40

Best Practices Scalability Performance C++ 40

The Netflix TechBlog

SEPTEMBER 2, 2020

Edgar helps Netflix teams troubleshoot distributed systems efficiently with the help of a summarized presentation of request tracing, logs, analysis, and metadata. The more complex a system, the more places to look for clues. In an earlier blog post, we discussed Telltale , our health monitoring system. What is Edgar?

Latency 296

Latency Transportation Engineering Traffic 296

HammerDB

DECEMBER 5, 2020

For TPC-C this meant enough available spindles to reduce I/O latency and for TPC-H enough bandwidth for data throughput. I.e. if system A generated 1.5X more transactions than system B in the fully audited benchmark then the HammerDB result was also 1.5X This was both expensive and time consuming to configure.

C++ 40

C++ Benchmarking Virtualization Open Source 40

Alex Russell

APRIL 29, 2021

After 20 years of neck-in-neck competition, often starting from common code lineages, there just isn't that much left to wring out of the system. Consistent improvement is the name of the game, and it can still have positive impacts, particularly as users lean on the system more heavily over time. Where Chrome Has Lagged.

Media 145

Media Games Education Engineering 145

Dotcom-Montior

MARCH 6, 2022

Some of the phases are presented below: Receiving. Cached Mode (Repeat Visit): The user access the website for the second time, emulating a second visit from the user’s perspective, which includes all the files now cached in a local storage. The time it takes to download files. This is the first phase of the timeline.

Performance 52

Performance Cache Website Speed 52

The Netflix TechBlog

NOVEMBER 3, 2022

In the time since it was first presented as an advanced Mesos framework, Titus has transparently evolved from being built on top of Mesos to Kubernetes, handling an ever-increasing volume of containers. As the number of Titus users increased over the years, the load and pressure on the system increased substantially.

Cache 224

Cache Latency Traffic Systems 224

O'Reilly

NOVEMBER 12, 2019

These are not system administrators whose job it is to worry about keeping software patched, secured, and managed. latency, startup, mocking, etc.) Serverless presents a conceptually simpler path to deploying software for those in development roles with no need to manage servers and storage.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. Driving Storage Costs Down for AWS Customers. All Things Distributed.

AWS 111

AWS Government Big Data Cloud 111