The Netflix TechBlog

APRIL 26, 2022

The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. Then we started hitting roadblocks while optimizing the query performance. Pay the storage, network, and compute cost. However, Spark query execution remained slow.

Storage 187

Storage Design Scalability Latency 187

n0derunner

NOVEMBER 9, 2018

We see often see this sort of test in bake-offs, and such a test does answer an important question – “what’s the lowest possible response time I can expect from the storage” However, this test only gives a single data point. The single VM test is using only a fraction of the cluster capacity.

Performance Testing 40

Performance Testing Testing Performance Storage 40

Scalegrid

JUNE 22, 2020

ScaleGrid provides 30% more storage on average vs. DigitalOcean for MySQL at the same affordable price. MySQL DigitalOcean Performance Benchmark. As you can see above, ScaleGrid and DigitalOcean offer the same plan configurations across this plan size, apart from SSD where ScaleGrid provides over 20% more storage for the same price.

Database 217

Database Benchmarking Latency Performance 217

Scalegrid

JULY 8, 2019

From there, you can do some performance testing to see if you require more CPU, in which case, balanced instance types or CPU-intensive instance types might better serve your MySQL needs, such as the Dv3 instance types. The Es2 series is typically a good starting point for most applications MySQL workloads.

Azure 219

Azure Cloud Virtualization Database 219

Scalegrid

APRIL 28, 2020

Since database hosting is more dependent on memory (RAM) than storage, we are going to compare various instance sizes ranging from just 1GB of RAM up to 64GB of RAM so you can see how costs vary across different application workloads. Meltdown Performance Impact on MongoDB: AWS, Azure & DigitalOcean. EC2 instances. VM instances.

Azure 344

Azure AWS Database Latency 344

Testsigma

JULY 3, 2021

If you’ve been performing on-premise testing in your organization, you know the rules already. But for the uninitiated, on-premise testing is a form of testing where testers perform tests on local machines, systems, or devices set up at an office. On-premise testing comes with a lot of responsibility.

Cloud 57

Cloud Testing Storage DevOps 57

Scalegrid

JANUARY 20, 2020

The ‘follower’ system is writable, so you can use it as a staging environment to test your application changes. Note: ScaleGrid implements follower clusters using storage snapshots. Database Dev/Test Setup. By using follower clusters, you can periodically import data from your production database into the dev/test database.

Analytics 185

Analytics Storage Database Systems 185

Percona

MAY 9, 2023

Many MongoDB clusters use storage-level snapshots to provide fast and reliable backups. To start realistic compatibility and performance testing, we needed to restore the filesystem snapshot backup that was stored on NetApp volume in GCP into a Google Kubernetes Engine (GKE) cluster running Percona Operator for MongoDB.

Storage 107

Storage Cloud Servers Google 107

Testsigma

DECEMBER 4, 2020

Web app testing : Optimized for mobile browsing and server-driven, the web apps should be tested for UI to ensure that all the elements of the app are aligned for different screens and resolutions. Since the mobile apps have limited storage space and power supply, test for memory utilization as well as battery usage.

Mobile 65

Mobile Testing Operating System Storage 65

Scalegrid

DECEMBER 16, 2019

They offer SSD-based cloud hosting with straightforward pricing as well starting at just $5/month , which makes it ideal (and affordable) for developers to build, test and deploy their new applications seamlessly in the cloud. What’s most impressive is that you’re not compromising performance for cost.

Azure 187

Azure AWS Latency Database 187

Dynatrace

SEPTEMBER 17, 2019

PostgreSQL & Elastic for data storage. Robert’s AWS & EKS admin team are monitoring most services with that capability but found it beneficial for them to have Dynatrace monitor Elastic File Storage (EFS). 2: Automate Quality aka “Performance as a Self-Service”. NGINX as an API Gateway. REDIS for caching.

AWS 127

AWS Monitoring Ecommerce Lambda 127

n0derunner

JULY 23, 2019

As the DB continues to run on the new host – the Nutanix storage detects the access patterns and “localizes” the data that the DB is accessing. Many different queries are executing in parallel, some hitting RAM cache, some hitting storage. Over the subsequent minutes the transaction rate increases to ~30,000 TPS.

Benchmarking 76

Benchmarking Storage Tuning Cache 76

The Netflix TechBlog

NOVEMBER 3, 2022

When a new leader is elected it loads all data from external storage. We ran empirical tests for scenarios of low and high request volumes, and the results are presented in the next section. Overview The figure below depicts a simplified high-level architecture of a single Titus cluster (a.k.a

Cache 224

Cache Latency Traffic Systems 224

n0derunner

OCTOBER 1, 2019

Nutanix X-Ray is well known for being able to model IO/Storage workloads, but what about workloads that are CPU bound? This time though the metric is Database transactions per second not IOPS or Storage throughput. X-Ray can run Ansible scripts on the X-Ray worker VMs, and by doing so we are able to provision almost any application.

Performance 75

Performance Benchmarking Storage Database 75

Percona

DECEMBER 20, 2018

A Look at MyRocks Performance. As the MyRocks storage engine (based on the RocksDB key-value store [link] ) is now available as part of Percona Server for MySQL 5.7 , I wanted to take a look at how it performs on a relatively high-end server and SSD storage. Fsync Performance on Storage Devices.

Database 40

Database Tuning Performance AWS 40

Apica

AUGUST 12, 2020

Additionally, many operational groups would like to be able to leverage functional testing scripts into production, saving hundreds, and potentially 1000’s of hours of rescripting time annually. Ensuring proper access and secured storage of the scripts, endpoints and user access credentials.

Monitoring 52

Monitoring Testing Scalability Hardware 52

Apica

OCTOBER 17, 2022

Changes happen much slower at the middle tier and integration layers than the UI, so testing third-party services and back-ends directly can offer longer-lasting reassurances about the integrity of individual components – even if the components don’t behave as expected once merged in production. Metrics and observability data hoarding.

Retail 52

Retail Website Infrastructure Software Architecture 52

n0derunner

SEPTEMBER 6, 2019

The vertica vioperf tool is used to determine whether the storage you are planning on using is fast enough to feed the vertica database. The amount of data written is simply the IO rate achieved by the storage * runtime. The vioperf tool runs on a linux host or VM and can be pointed at any filesystem just like fio or vdbench.

Storage 40

Storage Database Metrics Systems 40

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52

HammerDB

OCTOBER 12, 2018

As is exactly the same with PostgreSQL for system choice a 2 socket system is optimal for MySQL OLTP performance. As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. After each test completed HammerDB then increased the virtual user count and repeated the test.

Best Practices 40

Best Practices Scalability Performance C++ 40

Highly Scalable

APRIL 2, 2012

The deployment schema includes three types of nodes – processing nodes, storage nodes, and maintenance nodes. Storage nodes are basically Coherence storage nodes. Both Storage and Maintenance nodes do not serve client requests. Both Storage and Maintenance nodes do not serve client requests.

Ecommerce 100

Ecommerce Cache Storage Architecture 100

Adrian Cockcroft

JUNE 21, 2022

Our team also produced The Guide to High Availability by storage expert Jeannie Johnstone Kobert, Sun Cluster Environment by Enrique Vargas, Solaris PC Netlink by Don Devitt (we had a great team outing near Boston on his sailboat), and several other books.

Tuning 52

Tuning Benchmarking Engineering C++ 52

SQL Server According to Bob

JANUARY 11, 2020

Stable Media Stable media is often confused with physical storage. SQL Server defines stable media as storage that can survive system restart or common failure. Stable media is commonly physical disk storage, but other devices and certain caching facilities qualify as well. See the article for more details. SQL Server 7.0

Servers 40

Servers Cache Media Hardware 40

Smashing Magazine

JANUARY 7, 2019

To optimize storage interally, you could use Dropbox’s new Lepton format for losslessly compressing JPEGs by an average of 22%. Test with network throttling, and emulate a high-DPI device. BrowserStack is fantastic, but test on real devices as well. k6 allows you write unit tests-alike performance tests.

Performance 103

Performance Cache Metrics Network 103

Smashing Magazine

JANUARY 6, 2020

To optimize storage interally, you could use Dropbox’s new Lepton format for losslessly compressing JPEGs by an average of 22%. Test with network throttling, and emulate a high-DPI device. BrowserStack is fantastic, but test on real devices as well. k6 allows you write unit tests-alike performance tests.

Performance 145

Performance Cache Network Metrics 145