The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Brendan Gregg

JUNE 30, 2018

These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Accurate benchmarking rewards engineering investment that actually improves performance, but, unfortunately, inaccurate benchmarking is more common. If the benchmark reported 20k ops/sec, you should ask: why not 40k ops/sec?

Benchmarking 54

Benchmarking Latency Cache Network 54

Smashing Magazine

SEPTEMBER 28, 2022

Five Data-Loading Patterns To Improve Frontend Performance. Five Data-Loading Patterns To Improve Frontend Performance. Data loading patterns are an essential part of your application as they will determine which parts of your application are directly usable by visitors. But isn’t waiting for the data the point?

Performance 125

Performance Cache Servers Social Media 125

Scalegrid

JANUARY 12, 2024

Key Takeaways A cloud workload encompasses any application or service running on a cloud infrastructure, facilitating tasks ranging from basic functions to advanced data analysis with the help of resources like databases, collaboration tools, and disaster recovery systems. This applies to both virtual machines and container-based deployments.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Brendan Gregg

JUNE 29, 2018

These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Accurate benchmarking rewards engineering investment that actually improves performance, but, unfortunately, inaccurate benchmarking is more common. If the benchmark reported 20k ops/sec, you should ask: why not 40k ops/sec?

Benchmarking 40

Benchmarking Latency Cache Network 40

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Benchmark before you decide. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. Transparent huge pages (THP) disabled.

Performance 129

Performance Hardware Tuning Storage 129

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. better cluster resource utilization.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Percona

JUNE 22, 2023

Database uptime and availability Monitoring database uptime and availability is crucial as it directly impacts the availability of critical data and the performance of applications or websites that rely on the MySQL database. Disk space usage Monitor the disk space usage of MySQL data files, log files, and temporary files.

Performance 107

Performance Monitoring Traffic Database 107

Percona

APRIL 7, 2023

To illustrate this, I ran the Sysbench-TPCC synthetic benchmark against two different GCP instances running a freshly installed Percona Server for MySQL version 8.0.31 In MySQL, considering the standard storage engine, InnoDB , the data cache is called Buffer Pool. MySQL (B) 2517529 2610323 389048 5516900 194140 11523.48

Tuning 138

Tuning Cache Servers Benchmarking 138

Particular Software

OCTOBER 26, 2020

Backed by Cosmos DB, a fully managed, globally distributed, elastically scaled, pay-as-you-go service, your NServiceBus-based systems can benefit from guaranteed single-digit-millisecond latency with 99.999% availability. How does this compare with Azure Storage Persistence? CosmosClient(new CosmosClient("ConnectionString")).DefaultContainer(

Azure 52

Azure Storage Benchmarking Latency 52

The Morning Paper

NOVEMBER 8, 2020

If you think of this a bit like mapping memory addresses to data in memory, then another parallel comes to mind: the virtual address space. replacing Paxos with Raft), or they could be shims over external storage systems. replacing Paxos with Raft), or they could be shims over external storage systems. The NativeLoglet.

Virtualization 126

Virtualization Latency Storage Systems 126

The Morning Paper

AUGUST 29, 2019

There’s a broad set of choices including where you store the data, whether you run your own DBMS nodes or use a service, the kinds of instance types to go for if you do run your own, and so forth. We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems.

Architecture 76

Architecture Cloud Storage Serverless 76

n0derunner

SEPTEMBER 17, 2018

Creating a HCI benchmark to simulate multi-tennent workloads. As with traditional storage, applications are writing to a shared storage environment which is necessary to support VM movement. As with traditional storage, applications are writing to a shared storage environment which is necessary to support VM movement.

Performance Testing 40

Performance Testing Benchmarking Testing Performance 40

Percona

JULY 1, 2023

It efficiently manages read and write operations, optimizes data access, and minimizes contention, resulting in high throughput and low latency to ensure that applications perform at their best. This flexibility helps businesses seamlessly transition their databases to the AWS cloud without significant disruptions.

AWS 52

AWS Database Serverless Storage 52

Percona

JANUARY 24, 2023

Reduced WAL segment size has further gains in replication and backup because less data need to be transmitted. Individual processes generate WAL records, and latency is very crucial for transactions. What is Full Page Writes? getconf PAGESIZE 4096 PostgreSQL deals with 8k pages as the “atomic unit” for reading and writing.

Database 120

Database Benchmarking Open Source Latency 120

HammerDB

NOVEMBER 5, 2018

This post at an entry-level discusses the options you have to improve log throughput in your benchmark environment. . Before the database is started your data should be in a consistent state on disk. When the database is started a component of the in-memory database instance is the data buffer.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

SQL Performance

APRIL 8, 2019

Back on December 5, 2017, Microsoft announced that they were using AMD EPYC 7551 processors in their storage-optimized Lv2-Series virtual machines. I wrote about using CPU-Z to benchmark the Intel Xeon E5-2673 v3 processor in an Azure VM in this article. Figure 1: CPU-Z Benchmark Results for LS16v2. Memory (GiB).

Azure 42

Azure Virtualization Storage Benchmarking 42

Percona

SEPTEMBER 1, 2023

This reduction in latency ensures that applications and websites provide a more rapid and responsive user experience. Scalability As your data volume and user base expand, a finely tuned database can seamlessly accommodate increased workloads without compromising performance. Query plan analysis can also provide valuable insights.

Tuning 52

Tuning Database Performance Hardware 52

HammerDB

DECEMBER 5, 2020

The HammerDB TPROC-C workload by design intended as CPU and memory intensive workload derived from TPC-C – so that we get to benchmark at maximum CPU performance at a much smaller database footprint. For TPC-C this meant enough available spindles to reduce I/O latency and for TPC-H enough bandwidth for data throughput.

C++ 40

C++ Benchmarking Virtualization Open Source 40

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Alex Russell

MARCH 6, 2021

A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. In 2016, Jio swept over the subcontinent like a monsoon dropping a torrent of 4G infrastructure and free data rather than rain. The Moto G4 , for example. A 2021 Global Baseline.

Performance 124

Performance Network Cache Metrics 124

Alex Russell

APRIL 29, 2021

This post mines publicly available data on the pace of compatibility fixes and feature additions to assess the claim. As an engineer on a browser team, I'm privy to the blow-by-blow of various performance projects, benchmark fire drills, and the ways performance marketing (deeply) impacts engineering priorities. Converging Views.

Media 145

Media Games Education Engineering 145

SQL Server According to Bob

SEPTEMBER 26, 2016

When we released Always On Availability Groups in SQL Server 2012 as a new and powerful way to achieve high availability, hardware environments included NUMA machines with low-end multi-core processors and SATA and SAN drives for storage (some SSDs). This chart shows our scaled results using a OLTP workload derived from TPC benchmarks.

Availability 40

Availability Servers Hardware Benchmarking 40

Alex Russell

OCTOBER 22, 2017

This rework delays launch which, in turn, delays gathering data about the viability of a PWA strategy. Budgets are scaled to a benchmark network & device. Deciding what benchmark to use for a performance budget is crucial. Given the conflicted data we see across our other systems, this seems about right as a baseline.

Performance 68

Performance Benchmarking Network Mobile 68

All Things Distributed

NOVEMBER 19, 2010

There was an excellent first benchmarking report of the Cluster GPU Instances by the folks at Cycle Computing - " A Couple More Nails in the Coffin of the Private Compute Cluster " The Top500 supercomputer list. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. At werner.ly

AWS 67

AWS Cloud Benchmarking Storage 67

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . maximum transition latency: Cannot determine or is not supported. . large-pages.

Best Practices 40

Best Practices Scalability Performance C++ 40

The Netflix TechBlog

SEPTEMBER 2, 2020

With request tracing and additional data from logs, events, metadata, and analysis, Edgar is able to show the flow of a request through our distributed system?—?what But, there are a few substantial differences in how Edgar approaches its data and its users. Adding logs to the picture can help a great deal.

Latency 296

Latency Transportation Engineering Traffic 296

SQL Server According to Bob

JANUARY 11, 2020

To ensure data integrity for our customers who are running SQL Server, it is important that I/O environments provide appropriate functionality. Atomicity A transaction must be an atomic unit of work; either all of its data modifications are performed or none of them are performed.

Servers 40

Servers Cache Media Hardware 40

ScaleOut Software

MAY 15, 2014

We have spent a great deal of time at ScaleOut Software re-architecting our in-memory data grid (IMDG)’s code base to make best use of many cores and large memory. For example, the IMDG must be able to efficiently create millions of objects in each server to make use of its huge storage capacity. Testing Scale-Up Performance.

Cache 40

Cache Servers Network Design 40

SQL Server According to Bob

JANUARY 11, 2020

If ​​ it is ​​ not, ​​ the ​​ data ​​ is considered ​​ to be ​​ changed ​​ or ​​ damaged. In ​​ other words, ​​ a formula is applied ​​ to a given set of bytes.

Servers 40

Servers Cache Database Media 40

Smashing Magazine

JANUARY 11, 2021

It will help you build up a company-tailored case study with real data. Furthermore, using data from case studies and experiments published on WPO Stats will help increase sensitivity for business about why performance matters, and what impact it has on user experience and business metrics. Large preview ). Large preview ).

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

It will help you build up a company-tailored case study with real data. Furthermore, using data from case studies and experiments published on WPO Stats will help increase sensitivity for business about why performance matters, and what impact it has on user experience and business metrics. Large preview ). Large preview ).

Performance 145

Performance Cache Network Metrics 145