Percona

FEBRUARY 15, 2023

A bloom filter is a space-efficient way of storing information about a list of keys. For good performance, the filter blocks are cached in the RocksDB block cache and normally stay there since they are accessed frequently. Bloom filter is an important feature improving performance of LSM storage engines.

Storage 125

Storage Tuning Cache Engineering 125

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19. This is not surprising in hindsight.

Storage 64

Storage Systems Hardware Efficiency 64

Percona

DECEMBER 23, 2022

I wrote this post on MyRocks because I believe it is the most interesting new MySQL storage engine to have appeared over the last few years. Although MyRocks is very efficient for writes, I chose a more generic workload that will provide a different MyRocks use case. RocksDB is a very efficient key-value store based on LSM trees.

Storage 55

Storage Cache C++ Virtualization 55

Smashing Magazine

SEPTEMBER 28, 2022

Active Memory Caching. When you want to get data that you already had quickly, you need to do caching — caching stores data that a user recently retrieved. Caching partially stores your data and is not used as permanent storage. Caching partially stores your data and is not used as permanent storage.

Cache 125

Cache Performance Servers Social Media 125

Percona

JUNE 22, 2023

We will also discuss related configuration variables to consider that can impact these KPIs, helping you gain a comprehensive understanding of your MySQL server’s performance and efficiency. Query performance Query performance is a key performance indicator (KPI) in MySQL, as it measures the efficiency and speed of query execution.

Performance 102

Performance Monitoring Traffic Database 102

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Percona

SEPTEMBER 1, 2023

Enhanced Database Efficiency By adjusting configuration settings, you can markedly enhance the overall efficiency of your MySQL database. This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. Let’s explore these benefits in more detail.

Tuning 52

Tuning Database Performance Hardware 52

Percona

JULY 25, 2023

After the “data dictionary” (DD) engine and DD cache are initialized on a server, the Storage Engines can ask for a table definition. Initializing a DD engine and the cache adds complexity and other server dependencies. Essentially LRU cache is disabled by loading the tables as non-evictable. ibd > t1.sdi

Cache 83

Cache Benchmarking Servers Design 83

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

Scalegrid

MARCH 28, 2024

Key Takeaways Redis offers complex data structures and additional features for versatile data handling, while Memcached excels in simplicity with a fast, multi-threaded architecture for basic caching needs. Redis is better suited for complex data models, and Memcached is better suited for high-throughput, string-based caching scenarios.

Cache 130

Cache Storage Scalability Architecture 130

Scalegrid

NOVEMBER 23, 2023

Configure the PostgreSQL hostname by editing configuration files and restarting the server, with secure storage of connection details to enhance security. Clearing your local machine’s DNS cache and setting up conditional forwarders on Hub VNET for customizing the DNS Server may help solve these issues.

Database 130

Database Servers Storage Network 130

Smashing Magazine

OCTOBER 26, 2021

Nx is an open-source build framework that helps you architect, test, and build at any scale — integrating seamlessly with modern technologies and libraries, while providing a robust command-line interface (CLI), caching, and dependency management. Nx uses distributed graph-based task execution and computation caching to speed up tasks.

Cache 109

Cache Servers Code Storage 109

The Morning Paper

JUNE 16, 2019

The challenging thing of course, is efficiently maintaining all of these parallel universes. If we do that naively though, we’re going to end up with a lot of universes to store and maintain and the storage requirements alone will be prohibitive. These ideas make an efficient multiverse database possible.

Database 90

Database Cache Benchmarking Efficiency 90

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Percona

JUNE 4, 2023

MariaDB retains compatibility with MySQL, offers support for different programming languages, including Python, PHP, Java, and Perl, and works with all major open source storage engines such as MyRocks, Aria, and InnoDB. Efficient data processing and indexing, making it optimal for executing queries and handling large datasets.

Open Source 52

Open Source Servers Storage Scalability 52

ScaleOut Software

JULY 21, 2020

For more than fifteen years, ScaleOut StateServer® has demonstrated technology leadership as an in-memory data grid (IMDG) and distributed cache. They also can perform analysis quickly and efficiently — where the data lives. In-Memory Data Grids for Fast-Changing Data. The Challenges with Parallel Queries.

Analytics 52

Analytics Airlines Cache Scalability 52

ScaleOut Software

JULY 21, 2020

For more than fifteen years, ScaleOut StateServer® has demonstrated technology leadership as an in-memory data grid (IMDG) and distributed cache. They also can perform analysis quickly and efficiently — where the data lives. In-Memory Data Grids for Fast-Changing Data. The Challenges with Parallel Queries.

Analytics 52

Analytics Airlines Cache Scalability 52

All Things Distributed

DECEMBER 18, 2012

Jul 4 - Leases: An efficient fault-tolerant mechanism for distributed file cache consistency , Gray, Cary, and David Cheriton, Vol. Aug 24 - The Five-Minute Rule Ten Years Later, and Other Computer Storage Rules of Thumb , Jim Gray and Goetz Graefe, ACM SIGMOD Record 26 (4): 63â??68, Saltzer, D. Reed, and D. Gray and David R.

Operating System 144

Operating System Cache AWS Scalability 144

Smashing Magazine

NOVEMBER 2, 2021

Cost-Efficient And Effort-Saving Development. On the contrary, a native application of an e-commerce store can come at 30, 50, or even 100 MB and up, consuming internal device storage. To specify, users can continue using those parts of the site that were cached if the Internet connection is unstable or when offline.

Code 74

Code Cache Retail Mobile 74

All Things Distributed

MARCH 5, 2012

Amazon ElastiCache customers will see their prices drop by up to 10%, depending on their cache node types. We continuously apply all our innovative skills to the design of datacenters, servers, storage, network, etc. to drive new efficiencies and higher reliability.

AWS 111

AWS Innovation Scalability Cache 111

Scalegrid

DECEMBER 21, 2023

To monitor Redis® instances effectively, collect Redis metrics focusing on cache hit ratio, memory allocated, and latency threshold. This ensures each Redis® instance optimally uses the in-memory data store and aligns with the operating system’s efficiency.

Strategy 130

Strategy Monitoring Latency DevOps 130

The Netflix TechBlog

SEPTEMBER 24, 2021

Figure 1: A Simplified Video Processing Pipeline With this architecture, chunk encoding is very efficient and processed in distributed cloud computing instances. From chunk encoding to assembly and packaging, the result of each previous processing step must be uploaded to cloud storage and then downloaded by the next processing step.

Cloud 237

Cloud Media Storage Cache 237

Dynatrace

AUGUST 1, 2022

Interestingly, our partner RedHat reported in 2021 that around 80% of deployed workloads are databases or data caches, storing data in persistent volume claims (PVCs). You quickly realize that it will take ages to fill up the overprovisioned database storage. Two days later, your database runs out of storage in the middle of the night.

Storage 192

Storage Database Network Tuning 192

Percona

APRIL 24, 2023

The more indexes, the more the requirement of memory for effective caching. Indexes need more cache than tables Due to random writes and reads, indexes need more pages to be in the cache. Cache requirements for indexes are generally much higher than associated tables.

Tuning 123

Tuning Cache Storage Database 123

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 286

Serverless AWS Lambda Storage 286

Percona

FEBRUARY 1, 2023

This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability. Therefore, TOAST is a storage technique used in PostgreSQL to handle large data objects such as images, videos, and audio files.

Storage 87

Storage Scalability Strategy Performance 87

The Netflix TechBlog

MARCH 8, 2021

They could need a GPU when doing graphics-intensive work or extra large storage to handle file management. Instead, we created a service to take the most popular configurations and cache them. We rely on our internal partner teams to support components installed on the workstation, such as storage and artist tools.

Entertainment 306

Entertainment Storage Hardware Open Source 306

Dynatrace

OCTOBER 4, 2022

While data lakes and data warehousing architectures are commonly used modes for storing and analyzing data, a data lakehouse is an efficient third way to store and analyze data that unifies the two architectures while preserving the benefits of both. Unlike data warehouses, however, data is not transformed before landing in storage.

Artificial Intelligence 214

Artificial Intelligence Storage Analytics Government 214

Highly Scalable

AUGUST 20, 2013

The pipelines can be stateful and the engine’s middleware should provide a persistent storage to enable state checkpointing. The engine should be compact and efficient, so one can deploy it in multiple datacenters on small clusters. Efficient lineage tracking is not a trivial problem. Interoperability with Hadoop. Pipelining.

Big Data 154

Big Data Processing Lambda Database 154

The Netflix TechBlog

MARCH 10, 2023

are stored in secure storage layers. Amsterdam is built on top of three storage layers. To avoid the ES query for the list of indices for every indexing request, we keep the list of indices in a distributed cache. It is also responsible for asset discovery, validation, sharing, and for triggering workflows.

Strategy 258

Strategy Cache Analytics Storage 258

Smashing Magazine

JANUARY 15, 2019

An obvious metric here is CPU usage, but memory usage and other forms of data storage also play their part. I predominantly use Nginx, and I have a particular fondness for FastCGI cache and have found it to be especially efficient. A particular favorite in the WordPress space is WP Super Cache.

Website Performance 73

Website Performance Website Energy Performance 73

ACM Sigarch

NOVEMBER 4, 2019

Using genomics and deep learning as two motivating examples, he highlighted the importance of algorithm-architecture co-design to improve end-to-end efficiency. The best paper runner-up was “ Dynamic Multi-Resolution Data Storage ”. . Albonesi.

Hardware 18

Hardware Architecture Programming Innovation 18

Percona

JULY 5, 2023

The principle of locality While the principle of locality is usually related to processors and cache access patterns, it also applies to data access in general. If your data is organized following the principle of locality, data access will be more efficient. This type of fragmentation can improve performance and efficiency.

Open Source 123

Open Source Database Efficiency Testing 123

The Morning Paper

FEBRUARY 6, 2020

Last week we looked at a function shipping solution to the problem; Cloudburst uses the more common data shipping to bring data to caches next to function runtimes (though you could also make a case that the scheduling algorithm placing function execution in locations where the data is cached a flavour of function-shipping too).

Lambda 98

Lambda Serverless Cache Latency 98

IO River

NOVEMBER 2, 2023

They need to deliver impeccable performance without breaking the bank.According to recent industry statistics, global streaming has seen an uptick of 30% in the past year, underscoring the importance of efficient CDN architecture strategies. Given its unchanging nature, static content is ideal for caching.

Architecture 52

Architecture Performance Internet Internet 52

HammerDB

JUNE 27, 2022

Cached vs Scaled Workloads. A key difference between cached and scaled workloads is the implementation of keying and thinking time to introduce a pause of time between transactions. Instead, most users prefer to implement a cached workload. Using TPC terminology for non-audited benchmarks violate TPC trademarks.

Benchmarking 62

Benchmarking Open Source C++ Cache 62

ScaleOut Software

MAY 15, 2014

For example, the IMDG must be able to efficiently create millions of objects in each server to make use of its huge storage capacity. During load-balancing, the client gets the following exception when accessing the cache: ErrorCode<ERRCA0017>:SubStatus<ES0006>:There is a temporary failure. Please retry later.

Cache 40

Cache Servers Network Design 40