Percona

APRIL 17, 2023

Make sure the drives are mounted with noatime and also if the drives are behind a RAID controller with appropriate battery-backed cache. Unfortunately, MongoDB is not NUMA-aware, and because of this, MongoDB can allocate memory unevenly, leading to the swap issue even with memory available. 25.84 - Total 21.04 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

Percona

MARCH 28, 2023

Journal It’s a process where every write operation gets written (appended) from Memory to a Journal file, AKA transaction log that exists on disk at a specific interval configured using “ journalCommitIntervalMs.” The same data, in the form of pages inside the Wiredtiger cache, are also marked dirty. wt and index-*.wt).

Hardware 85

Hardware C++ Storage Cache 85

Percona

SEPTEMBER 1, 2023

This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. A finely tuned database processes queries more efficiently, leading to swifter results. MySQL relies heavily on the availability of hardware resources to perform at its best.

Tuning 52

Tuning Database Performance Hardware 52

Percona

JANUARY 3, 2024

ProxySQL is a high-performance SQL proxy that runs as a daemon watched by a monitoring process. The process monitors the daemon and restarts it in case of a crash to minimize downtime. The proxy is designed to run continuously without needing to be restarted. Reach out to us today to schedule your instructor-led class!

Traffic 106

Traffic Hardware Cache Servers 106

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It became clear that real-time query processing and in-stream processing is the immediate need in many practical applications. Fault-tolerance.

Big Data 154

Big Data Processing Lambda Database 154

Randon ASCII

SEPTEMBER 30, 2022

Modern software loves to use additional processes so I wasn’t surprised when the busiest process wasn’t SoundRec.exe , but the verbosely named “RuntimeBroker.exe <Microsoft.WindowsSoundRecorder>”. Is there some RuntimeBroker caching? Is that a WinRT flaw, or a Voice Recorder flaw?

Software 130

Software Software Cache Hardware 130

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., Microservices fundamentally change a lot of assumptions current cloud systems are designed with, and present both opportunities and challenges when optimizing for quality of service (QoS) and utilization.

Open Source 61

Open Source Hardware Benchmarking Systems 61

Scalegrid

JANUARY 5, 2024

Despite initial investment costs, DBMS presents long-term savings and improved efficiency through automated processes, efficient query optimizations, and scalability, contributing to enhanced decision-making and end-user productivity. Since its introduction in the 1960s, the concept of DBMS has undergone significant evolution.

Efficiency 147

Efficiency Storage Database Scalability 147

HammerDB

JUNE 27, 2022

It enables the user to measure database performance and make comparative judgements about database hardware and software. Databases are highly sophisticated software, and to design and run a fair benchmark workload is a complex undertaking. Cached vs Scaled Workloads. Why HammerDB was developed.

Benchmarking 62

Benchmarking Open Source C++ Cache 62

Percona

MAY 1, 2023

Key areas include: Configuration parameter tuning : This tuning involves altering variables such as memory allocation, disk I/O settings, and concurrent connections based on specific hardware and requirements. Throughput: The throughput of an application is directly influenced by the speed at which the database can process and serve queries.

Tuning 52

Tuning Efficiency Database Performance 52

The Morning Paper

MARCH 8, 2020

This paper describes the design decisions behind the Snowflake cloud-based data warehouse. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) joins) during query processing. Workload characteristics.

Storage 112

Storage Engineering Cache Serverless 112

Percona

JUNE 22, 2023

Defining high availability In general terms, high availability refers to the continuous operation of a system with little to no interruption to end users in the event of hardware or software failures, power outages, or other disruptions. In addition to the disaster recovery site, this design includes an external layer of nodes.

Availability 105

Availability Database Open Source Hardware 105

Testsigma

JUNE 3, 2021

A few of the errors which can be faced due to a poorly designed UI are: i. Example: An e-commerce website is unable to process the payment part. Otherwise, it may result in the processing of garbage data and erroneous results. Hardware error. We focus on software so much that we forget about the hardware failures.

Software 82

Software Software Testing Hardware 82

Brendan Gregg

FEBRUARY 28, 2023

DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

ACM Sigarch

JULY 17, 2023

We then discuss defects observed, the role of test and design to address this problem, and identify research opportunities. An example of a specification is the correct operation of the hardware of a microprocessor. We first frame the problem statement and highlight the importance of fault root causes. Each fault has a root cause.

Energy 52

Energy Hardware Best Practices Architecture 52

Percona

MAY 2, 2023

Let’s look at some questions I did that a MySQL DBA usually needs to answer in an interview process. ChatGPT: The InnoDB buffer pool is used by MySQL to cache frequently accessed data in memory. If we expand the cache concept more, the buffer pool could be even less if the working set (hot data) is smaller.

Social Media 119

Social Media Database Servers Cache 119

John McCalpin

APRIL 25, 2023

For most high-end processors these values have remained in the range of 75% to 85% of the peak DRAM bandwidth of the system over the past 15-20 years — an amazing accomplishment given the increase in core count (with its associated cache coherence issues), number of DRAM channels, and ever-increasing pipelining of the DRAMs themselves.

Benchmarking 97

Benchmarking Cache Latency Tuning 97

HammerDB

APRIL 19, 2023

sysbench is a widely used open-source benchmarking tool that is designed to evaluate the performance of CPU, memory, disk I/O, and database systems. In the context of MySQL performance evaluation, it simulates a typical online transaction processing (OLTP) workload on your MySQL database. and 8.0.32 HammerDB 4.5 to 8.0.32.

Performance 54

Performance Benchmarking Cache Storage 54

The Morning Paper

NOVEMBER 5, 2019

Breaking that assumption allowed Ceph to introduce a new storage backend called BlueStore with much better performance and predictability, and the ability to support the changing storage hardware landscape. But let’s take a quick look at the changing hardware landscape before we go on… The changing hardware landscape.

Storage 64

Storage Systems Hardware Efficiency 64

Alex Russell

MARCH 6, 2021

A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. Advances in browser content processing. Hardware Past As Performance Prologue. The Moto G4 , for example. Modern network performance and availability. Content Is Dead, Long Live Content.

Performance 124

Performance Network Cache Metrics 124

The Morning Paper

DECEMBER 10, 2019

. …software operating on persistent data structures requires "global" pointers that remain valid after a process terminates, while hardware requires that a diverse set of devices all have the same mappings they need for bulk transfers to and from memory, and that they be able to do so for a potentially heterogeneous memory system.

Hardware 52

Hardware Virtualization Operating System Programming 52

SQL Server According to Bob

JANUARY 11, 2020

Stable media is commonly physical disk storage, but other devices and certain caching facilities qualify as well. Many high-end disk subsystems provide high-speed cache facilities to reduce the latency of read and write operations. This cache is often supported by a battery-powered backup facility.

Servers 40

Servers Cache Media Hardware 40

Brendan Gregg

FEBRUARY 28, 2023

DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Smashing Magazine

JANUARY 11, 2021

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. Establish a performance culture.

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

AUGUST 15, 2021

Cache Merril. In an Agile approach, a technology roadmap feeds the sprint and grooming processes, providing insight into how the product will travel from start to finish. In a way, they’re designed to communicate uncertainty and provide transparency into what stops along the way are likely to remain as is and which might be in flux.

Technology 139

Technology Technology Development Strategy 139

The Morning Paper

DECEMBER 8, 2019

Byte-addressable non-volatile memory,) NVM will fundamentally change the way hardware interacts, the way operating systems are designed, and the way applications operate on data. The beauty of persistent memory is that we can use memory layouts for persistent data (with some considerations for volatile caches etc.

Hardware 56

Hardware Programming Media Storage 56

Brendan Gregg

AUGUST 18, 2019

bpftrace uses BPF (Berkeley Packet Filter), an in-kernel execution engine that processes a virtual instruction set. hardware Hardware counter-based instrumentation. pid process ID. comm Process or command name. Syscall counts by process bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count() }'.

Latency 68

Latency C++ Cache Programming 68

Smashing Magazine

JANUARY 6, 2020

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. Establish a performance culture.

Performance 145

Performance Cache Network Metrics 145

SQL Performance

DECEMBER 17, 2019

This removes the burden of purchasing and maintaining your hardware, storage and networking infrastructure, while still giving you a very familiar experience with Windows and SQL Server itself. Microsoft currently has eight main types of virtual machines designed for different types of workloads. GHz, 128MB of L3 cache, 128 PCIe 4.0

Azure 72

Azure Virtualization Servers Storage 72

Smashing Magazine

JANUARY 7, 2019

Performance isn’t just a technical concern: it matters, and when baking it into the workflow, design decisions have to be informed by their performance implications. Without a strong alignment between dev/design and business/marketing teams, performance isn’t going to sustain long-term. Establish a performance culture.

Performance 104

Performance Cache Network Metrics 104

Highly Scalable

APRIL 2, 2012

Some time ago I participated in design of a backend for one large online retailer company. In particular, we built this system on top of Oracle Coherence and designed our own data structures and indexes. In particular, we built this system on top of Oracle Coherence and designed our own data structures and indexes.

Ecommerce 100

Ecommerce Cache Storage Architecture 100

Abhishek Tiwari

DECEMBER 27, 2017

In 2018, we anticipate that ETL will either lose relevance or the ETL process will disintegrate and be consumed by new data architectures. Apache Arrow's in-memory columnar layout is specifically optimized for data locality for better performance on modern hardware like CPUs and GPUs. Leveraging the recent hardware advances.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

Alex Russell

APRIL 29, 2021

Apple's policy against browser engine choice adds years of delays beyond the (expected) delay of design iteration, specification authoring, and browser feature development. Helps media apps on the web save battery when doing video processing. is access to hardware devices. They may have shipped in iOS 14.5 Compression Streams.

Media 145

Media Games Education Engineering 145

The Morning Paper

SEPTEMBER 19, 2019

It handles an order of magnitude more throughput than a prototype built on a stream processing engine. Could it be Analyzing efficient stream processing on modern hardware ? Embedded functional dependencies looks at the problem of robust schema design to eliminate redundant data values and handle missing data values.

Blockchain 46

Blockchain Hardware Google Analytics 46

ACM Sigarch

FEBRUARY 23, 2018

In industry, generally due to time-to-market restrictions, we tend to think extremely short term with evolutionary design changes rather than riskier revolutionary ideas that have a longer timeline for returns. I recall heated discussions 20+ years ago on whether speculative data in caches improves or degrades cache performance.

Architecture 47

Architecture Cache Hardware Virtualization 47

High Scalability

NOVEMBER 25, 2019

Your Laptop has a filesystem used by hundreds of processes. It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. On prem data processing. Edge caching. Nginx for disk based caching. Disk based caching.

Architecture 52

Architecture Cache Azure Storage 52

Smashing Magazine

JANUARY 21, 2021

We were designing and building things, setting up new products, refactoring the components, and publishing articles. Plus a service worker that caches all static assets and serves them for repeat views, along with cached versions of articles that a reader has already visited. Image source: Lighthouse Metrics ) ( Large preview ).

Performance 145

Performance Cache Mobile Network 145