Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Operating System (OS) settings Swappiness Swappiness is a Linux kernel setting that influences the behavior of the Virtual Memory manager when it needs to allocate a swap, ranging from 0-100.

Best Practices 52

Best Practices Design Tuning Database 52

Dynatrace

SEPTEMBER 13, 2023

This is a set of best practices and guidelines that help you design and operate reliable, secure, efficient, cost-effective, and sustainable systems in the cloud. Storing frequently accessed data in faster storage, usually in-memory caching, improves data retrieval speed and overall system performance. Beyond

AWS 243

AWS Efficiency Azure Cloud 243

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. Such demanding use cases place a great value on systems capable of fast and reliable execution, a need that spans across various industry segments.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

Dynatrace

DECEMBER 15, 2022

This transition to public, private, and hybrid cloud is driving organizations to automate and virtualize IT operations to lower costs and optimize cloud processes and systems. Besides the traditional system hardware, storage, routers, and software, ITOps also includes virtual components of the network and cloud infrastructure.

Artificial Intelligence 198

Artificial Intelligence DevOps Hardware Virtualization 198

Percona

NOVEMBER 3, 2023

Rather than listing the concepts, function calls, etc, available in Citus, which frankly is a bit boring, I’m going to explore scaling out a database system starting with a single host. I won’t cover all the features but show just enough that you’ll want to see more of what you can learn to accomplish for yourself.

Database 104

Database Benchmarking Latency C++ 104

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. Managing Costs Lower storage expenses in the public cloud and the ability to scale resources as needed enhance the cost-effectiveness of a hybrid cloud environment compared to on-premises infrastructure.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Randon ASCII

SEPTEMBER 8, 2019

The fact that this shows up as CPU time suggests that the reads were all hitting in the system cache and the CPU time was the kernel overhead (note ntoskrnl.exe on the first sampled call stack) of grabbing data from the cache. Now that we suspect file I/O it’s necessary to go to Graph Explorer-> Storage-> File I/O.

Latency 79

Latency Cache Programming Operating System 79

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 96

AWS Benchmarking Performance Traffic 96

Dynatrace

JANUARY 31, 2024

million” – Gartner Data observability is a practice that helps organizations understand the full lifecycle of data, from ingestion to storage and usage, to ensure data health and reliability. Downstream dashboards and system automations were using outdated data, leading to incorrect statuses of flights in reports.

DevOps 191

DevOps Analytics Airlines Metrics 191

All Things Distributed

JANUARY 23, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? s storage infrastructure. Comments ().

Storage 115

Storage AWS Virtualization Cloud 115

Scalegrid

JANUARY 8, 2024

Artificial intelligence can automate tasks ranging from: data analysis resource provisioning system maintenance decision-making natural language processing This not only improves accuracy and reliability but also frees up valuable time for IT teams to focus on strategic tasks, such as resource management on platforms like Google Cloud.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Dynatrace

JUNE 25, 2020

The network latency between cluster nodes should be around 10 ms or less. For Premium HA, this has been extended from 10 ms latency (in the same network region) to around 100 ms network latency due to asynchronous data replication between regions. In the image below, three downed nodes make an entire cluster unavailable.

Availability 244

Availability Hardware Latency Traffic 244

The Morning Paper

FEBRUARY 6, 2020

’ Stateless is fine until you need state, at which point the coarse-grained solutions offered by current platforms limit the kinds of application designs that work well. On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access.

Lambda 98

Lambda Serverless Cache Latency 98

Percona

MAY 1, 2023

This message is normally a side effect of a storage subsystem that is not capable of keeping up with the number of writes (e.g., However, gp2 suffers from a tricky way of calculating credits and usage that may drive erroneous conclusions about the real capacity of the storage. The settings might not be optimal. IOPs) required by MySQL.

AWS 119

AWS Hardware Storage Tuning 119

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Amazon Elastic File System (EFS). Amazon Simple Storage Service (S3). The example below visualizes average latency by API name and stage for a specific AWS API Gateway. Amazon Aurora.

AWS 135

AWS Metrics IoT Storage 135

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Amazon Elastic File System (EFS). Amazon Simple Storage Service (S3). The example below visualizes average latency by API name and stage for a specific AWS API Gateway. Amazon Aurora.

AWS 132

AWS Metrics IoT Storage 132

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Morning Paper

OCTOBER 26, 2020

As a production system within Microsoft capturing around a quadrillion events and indexing 16 trillion search keys per day it would be interesting in its own right, but there’s a lot more to it than that. These two narratives of reference architecture and ingestion/indexing system are interwoven throughout the paper.

Cloud 104

Cloud Big Data Latency Architecture 104

The Morning Paper

JANUARY 30, 2020

Edge servers are the middle ground – more compute power than a mobile device, but with latency of just a few ms. The Mobile Web Worker (MWW) System. The current system assumes an application specific regression model is available on the servers which can predict processing time given the current parameters of the job (e.g.

Mobile 104

Mobile Cloud Latency Games 104

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

The Morning Paper

SEPTEMBER 10, 2019

Google already has Dremel , Mesa , Photon , F1 , PowerDrill , and Spanner , so why did they need yet another data processing system? Because they had too many data processing systems! ;). And given the growing scale of YouTube, some of those backend systems were starting to creak. Procella system overview. are divided.

Analytics 75

Analytics Latency Cache Google 75

Dynatrace

APRIL 26, 2022

STM generates traffic that replicates the typical path or behavior of a user on a network to measure performance for example, response times, availability, packet loss, latency, jitter, and other variables). One use case for STM is to model the behavior of a customer in the form of a flow of transactions along the buyer’s journey.

Monitoring 199

Monitoring Social Media IoT Metrics 199

Particular Software

AUGUST 16, 2021

which provides saga and outbox storage for NServiceBus endpoints that is transactionally consistent with the business data you store in Cosmos DB. But where Cosmos DB really shines is for systems that require worldwide low latency with flexible pricing. You might use this if you were designing a new competitor to Facebook. ??

Azure 52

Azure Serverless Storage Database 52

Percona

DECEMBER 26, 2023

Dashboard example 2: Monitoring in real time These two panels show read/write IO performance to the persistent storage while benchmarking a live run. Percona Distribution for PostgreSQL provides the best and most critical enterprise components from the open-source community, in a single distribution, designed and tested to work together.

Benchmarking 121

Benchmarking Metrics C++ Database 121

Percona

JULY 17, 2023

5 among all database management systems and No. 1 among non-relational/document-based systems ( DB-Engines, July 2023 ). Instead of the table-based structure of relational databases, MongoDB stores data in documents and collections, a design for handling large amounts of unstructured data and for real-time web applications.

Open Source 75

Open Source Database Scalability Software 75

Particular Software

SEPTEMBER 20, 2021

Some will claim that any type of RPC communication ends up being faster (meaning it has lower latency) than any equivalent invocation using asynchronous messaging. There are more steps, so the increased latency is easily explained. Garbage collectors are designed under the assumption that memory should be cleaned up reasonably quickly.

Benchmarking 97

Benchmarking Latency Servers Systems 97

ScaleOut Software

JULY 14, 2021

The Microsoft Azure IoT ecosystem offers a rich set of capabilities for processing IoT telemetry, from its arrival in the cloud through its storage in databases and data lakes. A security sensor in a key-card access system might indicate an unusual pattern of building entries for an employee who has given notice of resignation.

IoT 52

IoT Azure Analytics Serverless 52

The Morning Paper

AUGUST 29, 2019

We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. As it is infeasible to test every OLAP system runnable on AWS, we chose widely-used systems that represented a variety of architectures and cost models. The design space.

Architecture 76

Architecture Cloud Storage Serverless 76

The Morning Paper

MARCH 3, 2020

It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. This work is latency critical, because volume IO is blocked until it is complete. A guiding principle.

Database 106

Database AWS Network Design 106

SQL Performance

FEBRUARY 26, 2020

Organizations are taking advantage of having managed backups, lots of built-in security features, an uptime SLA of 99.99%, and an always up-to-date environment where they are no longer responsible for patching SQL Server or the operating system. One size does not always fit all. Managed Instance provides two tiers for performance.

Azure 63

Azure Performance Storage Latency 63

IO River

NOVEMBER 2, 2023

Yes, these might change based on localization settings, but in terms of storage, it’s still a completely baked cookie that the server can choose to serve or not serve.Whenever a user clicks on an episode, these bits remain unchanged. Login & Authentication: Systems that verify user credentials and maintain session-specific data.

Architecture 52

Architecture Performance Internet Internet 52

Percona

JULY 1, 2023

With its support for MySQL and PostgreSQL, and its automated replication and backup capabilities, it’s designed to deliver high performance, scalability, and availability to meet the needs of mission-critical applications.

AWS 52

AWS Database Serverless Storage 52