The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

APRIL 26, 2022

Understanding why a user is experiencing transactional or performance issues enables organizations to achieve greater observability that goes beyond metrics, traces and logs. Digital experience monitoring (DEM) allows an organization to optimize customer experiences by taking into account the context surrounding digital experience metrics.

Monitoring 200

Monitoring Social Media IoT Metrics 200

Percona

AUGUST 10, 2018

The performance of a PostgreSQL database can be compromised by dead tuples, since they continue to occupy space and can lead to bloat. Now, though, it’s time to look at autovacuum for postgres, and the internals you to know to maintain a high performance PostgreSQL database needed by demanding applications.

Tuning 45

Tuning Cache Database Storage 45

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

HammerDB

MAY 5, 2019

So the PAUSE instruction is an optimization over NOP when a thread is waiting to acquire a lock and is particularly important in spin-locks on x86 CPUs for power and performance. ” and “as the PAUSE latency has been increased significantly, workloads that are sensitive to PAUSE latency will suffer some performance loss.”

Testing 48

Testing Tuning Latency Storage 48

The Morning Paper

JUNE 13, 2019

Making queries to an inference engine has many of the same throughput, latency, and cost considerations as making queries to a datastore, and more and more applications are coming to depend on such queries. The following figure highlights how just one of these variables, batch size, impacts throughput and latency on ResNet50.

Hardware 58

Hardware Latency Serverless Energy 58

Percona

MARCH 23, 2023

This blog is in reference to our previous ones for ‘Innodb Performance Optimizations Basics’ 2007 and 2013. Although there have been many blogs about adjusting MySQL variables for better performance since then, I think this topic deserves a blog update since the last update was a decade ago, and MySQL 5.7

Performance 127

Performance Hardware Tuning Storage 127

The Morning Paper

MARCH 10, 2020

So if you want to design a system that will offer good real-world performance, it’s really useful to have benchmarks that accurately represent real-world workloads. This benchmark can synthetically generate more precise key-value queries that represent the reads and writes of key-value stores to the underlying storage system.

Benchmarking 59

Benchmarking Storage Cache Open Source 59

Scalegrid

JUNE 4, 2020

In this post, we are going to compare the performance and pricing of DigitalOcean PostgreSQL vs. ScaleGrid PostgreSQL to help you determine the best PostgreSQL hosting service on DigitalOcean. Compare Latency. lower latency compared to DigitalOcean for PostgreSQL. PostgreSQL DigitalOcean Performance Test. Compare Pricing.

Database 230

Database Latency Benchmarking Performance 230

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. Pipeline reliability and performance go hand in hand. If tuned for performance, there is a good change reliability is compromised - and vice versa.

Latency 63

Latency Analytics Scalability Engineering 63

The Netflix TechBlog

SEPTEMBER 24, 2021

The ProRes codec family provides great editing performance and image quality. From chunk encoding to assembly and packaging, the result of each previous processing step must be uploaded to cloud storage and then downloaded by the next processing step. Uploading and downloading data always come with a penalty, namely latency.

Cloud 237

Cloud Media Storage Cache 237

The Netflix TechBlog

JUNE 13, 2023

A small percentage of production traffic is redirected to the two new clusters, allowing us to monitor the new version’s performance and compare it against the current version. Canaries also provide an opportunity to measure system performance under different load conditions, allowing us to identify and resolve any performance bottlenecks.

Traffic 279

Traffic Metrics Systems Strategy 279

The Netflix TechBlog

OCTOBER 27, 2020

Iceberg is widely adopted in Netflix as a data warehouse table format that addresses many of the usability and performance problems with Hive tables. The data warehouse is not designed to serve point requests from microservices with low latency. As most key-value storage engines support efficiently deleting a namespace (e.g.

Latency 243

Latency Storage Big Data Tuning 243

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? To be fair, that is also true with PostgreSQL; it hasn’t been tuned either, and it, too, can also perform much better.

Tuning 137

Tuning Cache Servers Benchmarking 137

The Netflix TechBlog

MAY 14, 2021

We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases. The watermarking functionality, at the start, was a simple offering with various Google Drive integrations for storage and links.

Traffic 160

Traffic Latency Java Google 160

The Netflix TechBlog

NOVEMBER 2, 2021

Perceptual quality measurements are used to drive video encoding optimizations , perform video codec comparisons , carry out A/B testing and optimize streaming QoE decisions to mention a few. This enables us to use our scale to increase throughput and reduce latencies. VQS is called using the measureQuality endpoint.

Media 171

Media Innovation Metrics Latency 171

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. The CFQ works well for many general use cases but lacks latency guarantees. The goal is to reduce memory usage without impacting query performance negatively.

Best Practices 52

Best Practices Design Tuning Database 52

SQL Performance

APRIL 27, 2020

The basic tier provides up to 5 DTUs with standard storage. The standard tier supports from 10 up to 3000 DTUs with standard storage and the premium tier supports 125 up to 4000 DTUs with premium storage, which is orders of magnitude faster than standard storage. vCore Pricing Tier. GB per vCore. HyperScale Database.

Azure 55

Azure Database Serverless Hardware 55

All Things Distributed

OCTOBER 2, 2017

We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded. Performant – The service would need to be able to maintain consistent performance in the face of diverse customer workloads.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

APRIL 9, 2015

The Amazon ML console and API provide data and model visualization tools, as well as wizards to guide you through the process of creating machine learning models, measuring their quality and fine-tuning the predictions to match your application requirements. Details on the AWS Blog. The Amazon Elastic File System. for a while already.

Lambda 122

Lambda Cloud IoT AWS 122

Brendan Gregg

JUNE 30, 2018

A co-worker introduced me to Craig Hanson and Pat Crain's performance mantras, which neatly summarize much of what we do in performance analysis and tuning. They are: **Performance mantras**. These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Don't do it. Do it less.

Benchmarking 54

Benchmarking Latency Cache Network 54

The Morning Paper

MARCH 17, 2020

Based on these findings, the authors present four guidelines to get the best performance out of this memory today. The data we have collected demonstrate that many of the assumptions that researchers have made about how NVDIMMs would behave and perform are incorrect. Understanding Optane DIMM performance. Optane DIMMs are here!

Scalability 59

Scalability Latency Cache Media 59

HammerDB

OCTOBER 12, 2018

This post complements the previous best practice guides this time with the focus on MySQL and MariaDB and achieving top levels of performance with the HammerDB MySQL TPC-C test. As is exactly the same with PostgreSQL for system choice a 2 socket system is optimal for MySQL OLTP performance. HammerDB difference from Sysbench.

Best Practices 40

Best Practices Scalability Performance C++ 40

Brendan Gregg

JUNE 29, 2018

A co-worker introduced me to Craig Hanson and Pat Crain's performance mantras, which neatly summarize much of what we do in performance analysis and tuning. They are: **Performance mantras**. These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Don't do it. Do it less.

Benchmarking 40

Benchmarking Latency Cache Network 40

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. Built-in, high-performance ML algorithms, re-engineered for greater, speed, accuracy, and data-throughput.

Tuning 94

Tuning AWS Scalability Infrastructure 94

HammerDB

NOVEMBER 5, 2018

If you are new to running Oracle, SQL Server, MySQL and PostgreSQL TPC-C workloads with HammerDB and have needed to investigate I/O performance the chances are that you have experienced waits on writing to the Redo, Transaction Log or WAL depending on the database you are testing. A good example of how tuning is an iterative process.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

Brendan Gregg

SEPTEMBER 5, 2017

Here's some output from my zfsdist tool, in bcc/BPF, which measures ZFS latency as a histogram on Linux: # zfsdist. Tracing ZFS operation latency. As a performance engineer who does kernel tracing, I find the lack of an ID I can trace in the kernel to be pretty annoying. Hit Ctrl-C to end. ^C People rarely create them manually.

Virtualization 76

Virtualization AWS Engineering Hardware 76

SQL Server According to Bob

JANUARY 11, 2020

This will help you increase system performance and avoid I/O environment errors. Atomicity A transaction must be an atomic unit of work; either all of its data modifications are performed or none of them are performed. Stable Media Stable media is often confused with physical storage.

Servers 40

Servers Cache Media Hardware 40

SQL Server According to Bob

JANUARY 11, 2020

​​ This is similar to the ​​ suspend and resume ​​ operations performed by ​​ the operating system. Streams ​​ NTFS volumes ​​ enable ​​ data files to have one or more secondary storage streams.

Servers 40

Servers Cache Database Media 40

Percona

NOVEMBER 3, 2023

xlarge 4vCPU 8GB-RAM Storage: EBS volume (root) 80GB gp2 (IOPS 240/3000) As well, high availability will be integrated, guaranteeing cluster viability in the case that one worker node goes down. Repeating the benchmark once again demonstrates that performance has degraded somewhat.

Database 107

Database Benchmarking Latency C++ 107