The Netflix TechBlog

JUNE 14, 2023

The GraphQL shim enabled client engineers to move quickly onto GraphQL, figure out client-side concerns like cache normalization, experiment with different GraphQL clients, and investigate client performance without being blocked by server-side migrations. In such cases, we were not testing for response data but overall behavior.

Traffic 353

Traffic Latency Cache Metrics 353

The Netflix TechBlog

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Transportation 192

Transportation Architecture Processing Storage 192

SQL Performance

SEPTEMBER 17, 2019

This article will cover many areas that database administrators need to be aware of in order to properly license, recover, and tune a Reporting Services installation. Unless the key is backed up, any protected data, such as connection strings and passwords, can't be decrypted. Tuning Options. Reporting Services Infrastructure.

Tuning 67

Tuning Servers Database Best Practices 67

Percona

APRIL 24, 2023

Indexes are generally considered to be the panacea when it comes to SQL performance tuning, and PostgreSQL supports different types of indexes catering to different use cases. I keep seeing many articles and talks on “tuning” discussing how creating new indexes speeds up SQL but rarely ones discussing removing them.

Tuning 126

Tuning Cache Storage Database 126

The Netflix TechBlog

MAY 4, 2023

Migrating Critical Traffic At Scale with No Downtime — Part 1 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. This technique facilitates validation on multiple fronts.

Traffic 339

Traffic Latency Tuning Systems 339

Percona

AUGUST 31, 2018

Out of the box, the default PostgreSQL configuration is not tuned for any particular workload. It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload. That means data is stored in memory twice, first in PostgreSQL buffer and then kernel buffer.

Tuning 56

Tuning Database Performance Benchmarking 56

Percona

APRIL 17, 2023

We’ll also go over some best practices for MongoDB security as well as MongoDB data modeling. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. Spoiler alert: This post focuses on MongoDB 3.6.X

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. Requirements In a previous blog post, we discussed Delta , a data enrichment and synchronization platform.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. Requirements In a previous blog post, we discussed Delta , a data enrichment and synchronization platform.

Database 197

Database Traffic Transportation Open Source 197

Percona

FEBRUARY 15, 2023

With MyRocks/RocksDB, data is stored in a set of large SST files. Tuning In terms of tuning, two parameters can be tuned, the size of the bitmap and the number of bits set by every value. Bloom filters are an essential component of an LSM-based database engine like MyRocks.

Storage 128

Storage Tuning Cache Engineering 128

Dynatrace

AUGUST 1, 2022

Kubernetes was initially designed with a strong focus on stateless workloads, meaning these workloads do not need to store any persistent data. Interestingly, our partner RedHat reported in 2021 that around 80% of deployed workloads are databases or data caches, storing data in persistent volume claims (PVCs). Stay tuned.

Storage 196

Storage Database Network Metrics 196

The Netflix TechBlog

JULY 21, 2022

We at Netflix, as a streaming service running on millions of devices, have a tremendous amount of data about device capabilities/characteristics and runtime data in our big data platform. With large data, comes the opportunity to leverage the data for predictive and classification based analysis.

Big Data 179

Big Data Cache Engineering Data Engineering 179

n0derunner

JULY 23, 2019

Effect of removing CPU constraints and maintaining data locality on a running DB instance. As the DB continues to run on the new host – the Nutanix storage detects the access patterns and “localizes” the data that the DB is accessing. The Postgres DB is totally un-tuned and contains purely default settings.

Benchmarking 76

Benchmarking Storage Tuning Cache 76

The Netflix TechBlog

SEPTEMBER 24, 2021

As described by the white paper Apple ProRes ( link ), the target data rate of the Apple ProRes HQ for 1920x1080 at 29.97 Uploading and downloading data always come with a penalty, namely latency. The problematic pattern of packagers is that they do not always generate data linearly. is 220 Mbps.

Cloud 237

Cloud Media Storage Cache 237

Percona

MARCH 23, 2023

By caching hot datasets, indexes, and ongoing changes, InnoDB can provide faster response times and utilize disk IO in a much more optimal way. If you see concurrency issues, you can tune this variable. As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases.

Performance 125

Performance Hardware Tuning Storage 125

Percona

AUGUST 10, 2018

avi@percona:~$ps -eaf | egrep "/post|autovacuum" postgres 2862 1 0 Jun17 pts/0 00:00:11 /usr/pgsql-10/bin/postgres -D /var/lib/pgsql/10/data postgres 2868 2862 0 Jun17 ? This may help tune your table level autovacuum settings appropriately. Tuning Autovacuum in PostgreSQL. Why is autovacuum needed ?

Tuning 44

Tuning Cache Database Storage 44

Percona

MARCH 27, 2023

having to open each table.frm (and in which my test runs, I have purposely read a very high number of tables compared to “Table-open-cache” variable). When running only sysbench, there is a high number of cache hits (~99.98%) and a low number of “misses,” which means the table_cache is big enough for the workload.

Cache 102

Cache Hardware Servers Benchmarking 102

Percona

NOVEMBER 9, 2023

With this data, we’ll be able to better understand and tune our Postgres servers. The first step is to include a new query that will gather data on WAL usage because this is not yet collected by default on the latest version of PMM. PMM already collects the configurations, so we will only miss data from the graphs.

Monitoring 90

Monitoring Tuning Metrics Servers 90

Smashing Magazine

NOVEMBER 8, 2021

From connecting back-office operations to front-of-the-house A/B testing and dynamic personalization for each customer, the shared foundation is fast server-side rendering powered by fast storefront data access. On top of this foundation, we add layers of caching, prerendering and edge delivery optimizations — not the other way around.

Cache 136

Cache Best Practices Strategy Servers 136

Dynatrace

JULY 16, 2020

I wanted to leverage Dynatrace’s Environment APIs, for example to export timeseries data, get problem stats, or change configuration settings, like enforcing a certain data privacy setting. TenantCache: a cache to store tenant information and API token information and semi-permanent data to avoid unnecessary roundtrips. ?

Cache 235

Cache Serverless Efficiency Tuning 235

John McCalpin

APRIL 25, 2023

For most high-end processors these values have remained in the range of 75% to 85% of the peak DRAM bandwidth of the system over the past 15-20 years — an amazing accomplishment given the increase in core count (with its associated cache coherence issues), number of DRAM channels, and ever-increasing pipelining of the DRAMs themselves.

Benchmarking 97

Benchmarking Cache Latency Tuning 97

Brendan Gregg

SEPTEMBER 5, 2021

The ZFS Adapative Replacement Cache (ARC) is the main memory cache for the file system. The arc_reclaim_thread periodically runs arc_adjust() to evict memory from the cache to keep it from growing too large, and to maintain a threshold of free memory that applications can quickly use. I worked on this code back at Sun.

Cache 145

Cache Cloud Java Tuning 145

HammerDB

APRIL 19, 2023

Please refer to this tuning guide to tune the system for HammerDB: Open Source Database Tuning Guide on 3rd Generation Intel® Xeon® Scalable Processors Based Platform. For the experiments in this blog, we did not tune the system. We are using the most basic, default, configurations for a database of about 100GB.

Performance 54

Performance Benchmarking Cache Storage 54

The Morning Paper

OCTOBER 18, 2020

Suppose you had a model that given a data item from a list, could predict its position in a sorted version of that list. But maybe we can sample a subset of the data and get a model that is a useful approximation, by learning an approximation to the CDF (cumulative distribution function). The big idea. Sympathy for the machine.

Cache 137

Cache C++ Java Tuning 137

IO River

NOVEMBER 2, 2023

‍Just as a well-coordinated airport directs flights to multiple runways based on traffic and weather conditions, a CDN with Multiple Origins Load Balancing ensures that web traffic is distributed across various data centers, optimizing performance and reliability. What is a CDN Multiple Origins Load Balancer?‍ What is “Origin”?

Traffic 52

Traffic Cache Servers Latency 52

Tim Kadlec

APRIL 12, 2020

The cache hit was very low (a topic for another day), and whenever the server had to provide a fresh version of the page, it was taking an unacceptably long time more often than not. Interestingly, the synthetic data was pretty much smoothing over the TTFB issues entirely. We didn’t have to go through the trouble, though.

Tuning 69

Tuning Cache Network Servers 69

IO River

NOVEMBER 2, 2023

Just as a well-coordinated airport directs flights to multiple runways based on traffic and weather conditions, a CDN with Multiple Origins Load Balancing ensures that web traffic is distributed across various data centers, optimizing performance and reliability. What is a CDN Multiple Origins Load Balancer?‍ That’s where rules come in.

Traffic 40

Traffic Cache Network Servers 40

Percona

JULY 27, 2023

This will be handy for auditing data. has EXPLAIN FORMAT=JSON that can output data into a user variable. This could be handy for tracking query tuning and performance auditing. Seems like a lot of work to do while servers are failing, but it should save the data in a better state. has 131 bug fixes and 8.1.0

Servers 83

Servers Open Source Tuning Cache 83

Dynatrace

NOVEMBER 13, 2020

Be sure to fine-tune the anomaly detection settings for your mobile app so that you can focus on those anomalies that are most relevant to the experience of your end users. This single-page app offers a variety of options for manually generating monitoring data and observing the data in the Dynatrace web UI. How to get started.

Code 181

Code Mobile Monitoring Infrastructure 181

Dynatrace

OCTOBER 15, 2020

Optimize Go applications based on real-time data from production environments. From http.Transport documentation : By default, Transport caches connections for future reuse. So stay tuned! Accelerate problem resolution and troubleshooting with our automatic root cause analysis.

Transportation 187

Transportation Monitoring Metrics Code 187

Percona

JULY 6, 2023

An index can save a lot of time in data access and lead the queries to gather the results the fastest way possible. then true else false end FROM generate_series(1, 10000000); Now we have our test table with some dummy data so we can practice. This column data type is boolean, so only two different values are possible, true and false.

Best Practices 90

Best Practices Tuning Database Testing 90

SQL Performance

JANUARY 11, 2021

We have a lot of data. And a lot of applications running against that data. So when their compiled application sends ad hoc queries to SQL Server, particularly as a prepared statement, and when we don't have the freedom to add or change indexes, several tuning opportunities are immediately off the table. Background.

Cache 59

Cache Tuning Database Performance 59

Brendan Gregg

SEPTEMBER 5, 2021

The ZFS Adapative Replacement Cache (ARC) is the main memory cache for the file system. The arc_reclaim_thread runs arc_adjust() to evict memory from the cache to keep it from growing too large, and to maintain a threshold of free memory that applications can quickly use. I worked on this code back at Sun. These were new to me.

Cache 52

Cache Cloud Java Tuning 52

The Morning Paper

MARCH 17, 2020

The data we have collected demonstrate that many of the assumptions that researchers have made about how NVDIMMs would behave and perform are incorrect. Once data has reached these queues it is durable. Use non-temporal stores for large transfers, and control cache evictions. Understanding Optane DIMM performance.

Scalability 59

Scalability Latency Cache Media 59

Brendan Gregg

FEBRUARY 9, 2018

In this post I'll look at the Linux kernel page table isolation (KPTI) patches that workaround Meltdown: what overheads to expect, and ways to tune them. The last two columns show cycles where a data TLB or instruction TLB walk was active in at least one PMH (page miss handler). Much of my testing was on Linux 4.14.11 and 4.14.12

Performance 75

Performance Benchmarking Cache Tuning 75

The Morning Paper

MARCH 10, 2020

That matters if your store reads data in (multi-key) blocks. The paper examines three different uses of RocksDB at Facebook: UDB , the underlying storage engine for the MySQL databases storing the social graph data. All cache read misses and all writes go through UDB servers, with SQL queries being converted into RocksDB queries.

Benchmarking 59

Benchmarking Storage Cache Open Source 59

Brendan Gregg

SEPTEMBER 5, 2021

The ZFS Adapative Replacement Cache (ARC) is the main memory cache for the file system. The arc_reclaim_thread runs arc_adjust() to evict memory from the cache to keep it from growing too large, and to maintain a threshold of free memory that applications can quickly use. I worked on this code back at Sun. These were new to me.

Cache 40

Cache Cloud Java Tuning 40