Scalegrid

FEBRUARY 8, 2024

Key Takeaways Distributed storage systems benefit organizations by enhancing data availability, fault tolerance, and system scalability, leading to cost savings from reduced hardware needs, energy consumption, and personnel. This process effectively duplicates essential parts of information to safeguard against potential loss.

Storage 130

Storage Systems Big Data Azure 130

The Netflix TechBlog

JUNE 4, 2019

Because microprocessors are so fast, computer architecture design has evolved towards adding various levels of caching between compute units and the main memory, in order to hide the latency of bringing the bits to the brains. Its goal is to assign running processes to time slices of the CPU in a “fair” way. So why mess with it?

Cache 251

Cache Latency Airlines Logistics 251

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases. Refer to innodb_redo_log_capacity below.

Performance 129

Performance Hardware Tuning Storage 129

The Morning Paper

DECEMBER 10, 2019

. …software operating on persistent data structures requires "global" pointers that remain valid after a process terminates, while hardware requires that a diverse set of devices all have the same mappings they need for bulk transfers to and from memory, and that they be able to do so for a potentially heterogeneous memory system.

Hardware 52

Hardware Virtualization Operating System Programming 52

John McCalpin

APRIL 25, 2023

This metric is interesting because we don’t always have the luxury of parallelizing every application we run, and our operating systems almost always process each call (e.g., This post is about a secondary performance characteristic — sustained memory bandwidth for a single thread running on a single core.

Benchmarking 97

Benchmarking Cache Latency Tuning 97

Dotcom-Montior

NOVEMBER 24, 2021

Each process has its own independent memory that it works with. State is distributed through the system. Software and hardware components are autonomous and execute tasks concurrently. Concurrency refers to the system’s ability to carry out multiple tasks in parallel and manage the access and usage of shared resources.

Systems 74

Systems Monitoring Hardware Network 74

The Morning Paper

DECEMBER 8, 2019

Byte-addressable non-volatile memory,) NVM will fundamentally change the way hardware interacts, the way operating systems are designed, and the way applications operate on data. This means that the overheads of system calls become much more noticeable. in front of that memory , as we saw last week).

Hardware 56

Hardware Programming Media Storage 56

The Morning Paper

APRIL 14, 2019

The paper sets out what we can do in software given today’s hardware, and along the way also highlights areas where cooperation from hardware will be needed in the future. The paper focuses on two key use cases: A confined component running in its own security domain, connected to the rest of the system by explicit (e.g.

Hardware 44

Hardware Cache Latency Speed 44

The Morning Paper

MAY 21, 2019

This boils down to a single digit µs latency toleration in the tail for far memory, and in addition to security and privacy concerns, rules out remote memory solutions. Thus we’re fundamentally trading (de)-compression latency at access time for the ability to pack more data in memory.

Software 78

Software Software Google Hardware 78

Brendan Gregg

AUGUST 8, 2017

Linux load averages are "system load averages" that show the running thread (task) demand on the system as an average number of running plus waiting threads. This measures demand, which can be greater than what the system is currently processing. Nowadays, the source code to old operating systems can also be found online.

Latency 111

Latency Metrics C++ Systems 111

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Operating System (OS) settings Swappiness Swappiness is a Linux kernel setting that influences the behavior of the Virtual Memory manager when it needs to allocate a swap, ranging from 0-100.

Best Practices 52

Best Practices Design Tuning Database 52

O'Reilly Software

APRIL 4, 2018

We live in the era of the connected experience, where our daily interactions with the world can be digitized, collected, processed, and analyzed to generate valuable insights. Accumulating all this data to process overnight is not an option anymore. Do we need to process each record individually? Process tolerance.

Architecture 80

Architecture Scalability Google Operating System 80

SQL Server According to Bob

JANUARY 11, 2020

Many high-end disk subsystems provide high-speed cache facilities to reduce the latency of read and write operations. Most manufacturers' implementations immediately flush pending writes to physical disk during the restart operations. Important Always consult with your hardware manufacturer for proper stable media strategies.

Servers 40

Servers Cache Media Hardware 40

Percona

JULY 1, 2023

Understanding DBaaS DBaaS cloud services allow users to use databases without configuring physical hardware and infrastructure or installing software. These may be performance, high availability, operational cost, management, capacity planning, scalability, security, monitoring, etc.

AWS 52

AWS Database Serverless Storage 52

Adrian Cockcroft

NOVEMBER 11, 2019

Staff should be familiar with recovery processes and the behavior of the system when it’s working hard to mitigate failures. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system. Book: Engineering a Safer World by Nancy G.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

Staff should be familiar with recovery processes and the behavior of the system when it’s working hard to mitigate failures. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system. Book: Engineering a Safer World by Nancy G.

Latency 53

Latency Engineering Systems Hardware 53

SQL Server According to Bob

JANUARY 11, 2020

Subsystem / Path The ​​ I/O subsystem or path ​​ includes ​​ those ​​ components ​​ that are ​​ used to support an I/O operation. DBCC Page Auditing DBCC page auditing is the process of executing common DBCC consistency checks as pages are read from disk.

Servers 40

Servers Cache Database Media 40

Smashing Magazine

JANUARY 7, 2019

So, if we created an overview of all the things we have to keep in mind when improving performance — from the very start of the process until the final release of the website — what would that list look like? Also, as Patrick Meenan suggested, it’s worth to plan out a loading sequence and trade-offs during the design process.

Performance 103

Performance Cache Metrics Network 103

All Things Distributed

MARCH 11, 2016

Failures are a given and everything will eventually fail over time: from routers to hard disks, from operating systems to memory units corrupting TCP packets, from transient errors to permanent failures. This is a given, whether you are using the highest quality hardware or lowest cost components. Primitives not frameworks.

AWS 144

AWS Hardware Retail Virtualization 144

High Scalability

SEPTEMBER 14, 2018

events processed to date 300k+ users globally 50% of the Fortune 100 use @pagerduty 10,500+ customers of every size 300+ integrations ?? editingemily : "In peacetime, a ticketing system is great. Because of that big servers and other memory systems need to have another kind of memory in the hierarchy. They'll love you even more.

Internet 30

Internet Internet Scalability Education 30