Brendan Gregg

JUNE 29, 2018

These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Accurate benchmarking rewards engineering investment that actually improves performance, but, unfortunately, inaccurate benchmarking is more common. If the benchmark reported 20k ops/sec, you should ask: why not 40k ops/sec?

Benchmarking 40

Benchmarking Latency Cache Network 40

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For storage, FIO is generally used. Benchmarking the target Two of the more popular database benchmarks for MySQL are HammerDB and sysbench. 0.42 %sys 9.52

Performance 54

Performance Benchmarking Cache Storage 54

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! In this talk, we share how Netflix deploys systems to meet its demands, Ceph’s design for high availability, and results from our benchmarking. We look forward to seeing you there!

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! In this talk, we share how Netflix deploys systems to meet its demands, Ceph’s design for high availability, and results from our benchmarking. We look forward to seeing you there!

AWS 100

AWS Entertainment Open Source Benchmarking 100

Scalegrid

JANUARY 12, 2024

These vendors serve data center players and offer advanced options, such as ScaleGrid’s engine, which ensures that different elements work well together automatically, eliminating the need for manual effort in managing heterogeneous environments. Storage is a critical aspect to consider when working with cloud workloads.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

DECEMBER 23, 2022

I wrote this post on MyRocks because I believe it is the most interesting new MySQL storage engine to have appeared over the last few years. The use case is the TPC-C benchmark but executed not on a high-end server but on a lower-spec virtual machine that is I/O limited like for example, with AWS EBS volumes. Conclusion.

Storage 55

Storage Cache C++ Virtualization 55

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Benchmark before you decide. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. Transparent huge pages (THP) disabled.

Performance 123

Performance Hardware Tuning Storage 123

HammerDB

JUNE 27, 2022

HammerDB is a software application for database benchmarking. Databases are highly sophisticated software, and to design and run a fair benchmark workload is a complex undertaking. The Transaction Processing Performance Council (TPC) was founded to bring standards to database benchmarking, and the history of the TPC can be found here.

Benchmarking 62

Benchmarking Open Source C++ Cache 62

Percona

JULY 1, 2023

What we should really compare is the MySQL and Aurora database engines provided by Amazon RDS. It also supports auto-scaling of compute and storage resources, dynamically adjusting capacity based on chosen utilization thresholds to optimize performance and costs. Considering a Fully Managed DBaaS Offering For Your Business?

AWS 52

AWS Database Serverless Storage 52

Percona

MARCH 17, 2023

PMM2 uses VictoriaMetrics (VM) as its metrics storage engine. Please note that the focus of these tests was around standard metrics gathering and display, we’ll use a future blog post to benchmark some of the more intensive query analytics (QAN) performance numbers.

Monitoring 95

Monitoring Scalability Cache Database 95

Percona

JULY 25, 2023

After the “data dictionary” (DD) engine and DD cache are initialized on a server, the Storage Engines can ask for a table definition. Initializing a DD engine and the cache adds complexity and other server dependencies. XtraBackup does not simply behave like a server to access a table object.

Cache 83

Cache Benchmarking Servers Design 83

Percona

JUNE 22, 2023

Indexing efficiency Monitoring indexing efficiency in MySQL involves analyzing query performance, using EXPLAIN statements, utilizing performance monitoring tools, reviewing error logs, performing regular index maintenance, and benchmarking/testing. This KPI is also directly related to Query Performance and helps improve it.

Performance 102

Performance Monitoring Traffic Database 102

Adrian Cockcroft

JUNE 21, 2022

So in 1988 I joined the Sun sales force and became a Systems Engineer (nowadays we would say Solutions Architect). Another big jump, but now it was my job to run benchmarks in the lab, and write white papers that explained the new products to the world, as they were launched.

Tuning 52

Tuning Benchmarking Engineering C++ 52

HammerDB

NOVEMBER 5, 2018

This post at an entry-level discusses the options you have to improve log throughput in your benchmark environment. . For MySQL the dependency is at the storage engine level and in this case for the InnoDB storage engine the parameter is self-explanatory. innodb_flush_log_at_trx_commit=0. PostgreSQL.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

Percona

APRIL 7, 2023

To illustrate this, I ran the Sysbench-TPCC synthetic benchmark against two different GCP instances running a freshly installed Percona Server for MySQL version 8.0.31 In MySQL, considering the standard storage engine, InnoDB , the data cache is called Buffer Pool. In PostgreSQL, it is called shared buffers.

Tuning 133

Tuning Cache Servers Benchmarking 133

Percona

JANUARY 19, 2023

It can help us to save costs on storage and backup times. While MySQL can handle large data sets, it is always recommended to keep only the used data in the databases, as this will make data access more efficient, and also will help to save costs on storage and backups. 1 mysql mysql 704M Dec 30 02:28 employees.ibd -rw-r --.

Open Source 85

Open Source Storage Database Big Data 85

Percona

JANUARY 9, 2023

Also, we find a difference in execution on InnoDB and MyISAM engines. sec) In this InnoDB engine, we can see that it requires some time to get COUNT(*) and COUNT(val_no_null) of rows for the table, and as we will see further, MyiSAM is significantly faster compared to InnoDB table in the sense of getting an answer for COUNT(*).

Cache 74

Cache Engineering Storage Speed 74

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. faster access to external storage and data locality (I/O, bandwidth). Heron is a real-time, distributed stream processing engine developed at Twitter. Storage provisioning. But Kubernetes storage is evolving quite quickly.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Percona

AUGUST 5, 2023

Otherwise, the storage engine does a scatter-gather and queries ALL partitions in a UNION that is not concurrent. This method distributes data evenly across partitions to achieve balanced storage and optimal query performance. You want to ensure that table lookups go to the correct partition or group of partitions.

Strategy 40

Strategy Storage Database Best Practices 40

The Morning Paper

NOVEMBER 8, 2020

Back in 2017 the engineering team at Facebook had a problem. replacing Paxos with Raft), or they could be shims over external storage systems. A minimal Loglet needs to provide totally ordered, durable storage via the shared log API. Every little helps! The NativeLoglet. Delos in production.

Virtualization 126

Virtualization Latency Storage Systems 126

Alex Russell

APRIL 29, 2021

Apple forces developers of competing browsers to use their engine for all browsers on iOS , restricting their ability to deliver a better version of the web platform. They are, pound for pound, some of the best engine developers globally and genuinely want good things for the web. With each team, benchmarks lost are understood as bugs.

Media 145

Media Games Education Engineering 145

Percona

JUNE 22, 2023

” Here are additional metrics used to determine the reliability of a database, make adjustments that minimize downtime, and set benchmarks for meeting business continuity requirements. Depending on your setup, costs can include: Hardware devices (servers, storage devices, network switches, etc.)

Availability 104

Availability Database Open Source Hardware 104

HammerDB

DECEMBER 5, 2020

From an engineering perspective this may be all that you need to know. The HammerDB TPROC-C workload by design intended as CPU and memory intensive workload derived from TPC-C – so that we get to benchmark at maximum CPU performance at a much smaller database footprint. This was both expensive and time consuming to configure.

C++ 40

C++ Benchmarking Virtualization Open Source 40

HammerDB

DECEMBER 1, 2022

Jobs Configuration and Storage. The base configuration for the jobs storage can be found in the generic.xml file in the commandline section. <commandline> If the storage of job related data is not desired, then it is possible to disable/enable this feature with the jobs disable command. Use a different Browser.

Virtualization 57

Virtualization C++ Storage Database 57

Percona

SEPTEMBER 1, 2023

Inadequate CPU, memory, or storage can lead to bottlenecks and performance degradation, so remedying these issues involves upgrading hardware or optimizing resource utilization through query and server configuration adjustments. Avoid over-indexing, which can bloat storage and slow writes. This does not apply to read (SELECT) traffic.

Tuning 52

Tuning Database Performance Hardware 52

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! In this talk, we share how Netflix deploys systems to meet its demands, Ceph’s design for high availability, and results from our benchmarking. We look forward to seeing you there!

AWS 37

AWS Entertainment Open Source Benchmarking 37

HammerDB

DECEMBER 14, 2022

Benchmark set to TPC-C for MariaDB. in host REDPOLL:3306 under user ROOT in database TPCC with storage engine INNODB? Benchmark set to TPC-C for MariaDB. Benchmark set to TPC-C for MariaDB. Results storage. Type "help" for a list of commands. Initialized SQLite on-disk database /home/hammerdb/HammerDB-4.6/TMP/hammer.DB

C++ 62

C++ Virtualization Database Benchmarking 62

Smashing Magazine

SEPTEMBER 28, 2022

On your first try, you can use it as a benchmark for optimizations later. An SSR application will generally have templating engines that inject the variables into an HTML when given to the client. Caching partially stores your data and is not used as permanent storage. Using the cache as permanent storage is an anti-pattern.

Cache 125

Cache Performance Servers Social Media 125

n0derunner

NOVEMBER 24, 2021

I have worked for Nutanix in the performance engineering group since 2013. My opinions are likely biased, but that also gives me a decent amount of context when it comes to the performance of Nutanix storage over time. We’ve come a long way, baby. Full disclosure.

Database 52

Database Performance Storage Engineering 52

n0derunner

NOVEMBER 24, 2021

I have worked for Nutanix in the performance engineering group since 2013. My opinions are likely biased, but that also gives me a decent amount of context when it comes to the performance of Nutanix storage over time. We’ve come a long way, baby. Full disclosure.

Database 52

Database Performance Storage Engineering 52

Percona

DECEMBER 20, 2018

As the MyRocks storage engine (based on the RocksDB key-value store [link] ) is now available as part of Percona Server for MySQL 5.7 , I wanted to take a look at how it performs on a relatively high-end server and SSD storage. Fsync Performance on Storage Devices. A Look at MyRocks Performance. Interested?

Database 40

Database Tuning Performance AWS 40

The Netflix TechBlog

MAY 21, 2019

Netflix engineers run a series of tests and benchmarks to validate the device across multiple dimensions including compatibility of the device with the Netflix SDK, device performance, audio-video playback quality, license handling, encryption and security. It could help us design and implement more targeted reward functions.

Testing 163

Testing AWS Lambda Network 163

Scalegrid

JULY 13, 2020

Recognized as the fastest growing database by popularity, PostgreSQL was named the DBMS of the year in both 2018 and 2017 by DB-Engines, and continues to grow in popularity in 2019. pg_repack – reorganizes tables online to reclaim storage. In fact, PostgreSQL is so popular, 11.5%

Open Source 244

Open Source Tuning C++ Database 244

The Morning Paper

JUNE 16, 2019

If we do that naively though, we’re going to end up with a lot of universes to store and maintain and the storage requirements alone will be prohibitive. The authors have built a prototype implementation of these ideas based on the Noria dataflow engine. For good query performance we’d like to pre-compute these per-user universes.

Database 90

Database Cache Benchmarking Efficiency 90

High Scalability

APRIL 3, 2019

For the 2nd year in a row, PostgreSQL has kept the title of #1 fastest growing database in the world according to the DBMS of the Year report by the experts at DB-Engines. PostgreSQL is an open source object-relational database system that has soared in popularity over the past 30 years from its active, loyal, and growing community.

Database 177

Database Cloud Open Source Systems 177

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . HammerDB difference from Sysbench. Otherwise find the file ut0ut.cc

Best Practices 40

Best Practices Scalability Performance C++ 40