Percona

APRIL 13, 2023

PostgreSQL is a popular open source relational database management system that is widely used for storing and managing data. Replication lag is the delay between the time when data is written to the primary database and the time when it is replicated to the standby databases. What is replication lag?

Latency 79

Latency Tuning Open Source Network 79

SQL Performance

SEPTEMBER 17, 2019

Many database administrators find themselves having to support instances of SQL Server Reporting Services (SSRS), or at least the backend databases that are required for SSRS. In each of the deployment models, the role of the database administrator is to make sure that SSRS is stable, dependable, and recoverable.

Tuning 67

Tuning Servers Database Best Practices 67

The Netflix TechBlog

JUNE 13, 2023

This is where large-scale system migrations come into play. By collecting and analyzing key performance metrics of the service over time, we can assess the impact of the new changes and determine if they meet the availability, latency, and performance requirements. But what happens when this machinery needs a transformation?

Traffic 279

Traffic Metrics Systems Strategy 279

Scalegrid

DECEMBER 14, 2023

How the MongoDB timeout is set up can significantly affect your application’s performance, no matter if you are an experienced MongoDB user or just starting with NoSQL databases. Typical applications are interacting with different database servers based on the business logic.

Java 130

Java Network Servers Database 130

Dynatrace

MAY 17, 2023

Think about items such as general system metrics (for example, CPU utilization, free memory, number of services), the connectivity status, details of our web server, or even more granular in-application tasks like database queries. Database monitoring Once more, under Applications & Microservices, we’ll also find Databases.

Metrics 176

Metrics Monitoring Database Network 176

Scalegrid

OCTOBER 17, 2019

In that environment, the first PostgreSQL developers decided forking a process for each connection to the database is the safest choice. It would be a shame if your database crashed, after all. On modern Linux systems, the difference in overhead between forking a process and creating a thread is much lesser than it used to be.

Architecture 200

Architecture Database Latency Servers 200

Scalegrid

OCTOBER 24, 2019

AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments. As organizations continue to migrate to the cloud, it’s important to get in front of performance issues, such as high latency, low throughput, and replication lag with higher distances between your users and cloud infrastructure.

AWS 183

AWS Latency Performance Benchmarking 183

Percona

NOVEMBER 2, 2023

MongoDB is a dynamic database system continually evolving to deliver optimized performance, robust security, and limitless scalability. Our new eBook, “ From Planning to Performance: MongoDB Upgrade Best Practices ,” guides you through the entire process to ensure your database’s long-term success.

Best Practices 84

Best Practices Hardware Tuning Scalability 84

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? That’s a heritage of the LAMP model when the same server would host both the database and the web server.

Tuning 134

Tuning Cache Servers Benchmarking 134

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Benchmark before you decide.

Performance 123

Performance Hardware Tuning Storage 123

Dynatrace

JANUARY 26, 2021

Traditional computing models rely on virtual or physical machines, where each instance includes a complete operating system, CPU cycles, and memory. REST APIs, authentication, databases, email, and video processing all have a home on serverless platforms. The provider is essentially your system administrator.

Serverless 245

Serverless Efficiency Lambda Azure 245

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. How Bulldozer leverages Spark, Protobuf and KV DAL for moving the data.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

AUGUST 13, 2020

You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? Our streaming teams need a monitoring system that enables them to quickly diagnose and remediate problems; seconds count! Our Node team needs a system that empowers a small group to operate a large fleet. By Andrei U.,

Monitoring 210

Monitoring Tuning Traffic Metrics 210

Scalegrid

MARCH 22, 2024

They can also bolster uptime and limit latency issues or potential downtimes. Ready to take your database management to the next level with ScaleGrid’s cutting-edge solutions? Register now for free and experience the seamless operation of your databases across multi-cloud and hybrid-cloud systems.

Strategy 130

Strategy Cloud Government Innovation 130

The Netflix TechBlog

MAY 14, 2021

However, they are scattered across multiple systems, and there isn’t an easy way to tie related messages together. Once you finally find useful identifiers, you may begin writing SQL queries against your production database to find out what went wrong. Netflix original series such as La Casa De Papel), as well as with Netflix Studio.

Traffic 160

Traffic Latency Java Google 160

Percona

AUGUST 10, 2018

The performance of a PostgreSQL database can be compromised by dead tuples, since they continue to occupy space and can lead to bloat. Now, though, it’s time to look at autovacuum for postgres, and the internals you to know to maintain a high performance PostgreSQL database needed by demanding applications.

Tuning 44

Tuning Cache Database Storage 44

Percona

MAY 1, 2023

I’ll show you some MySQL settings to tune to get better performance, and cost savings, with AWS RDS. ” Yes, this is true, io2 volumes are expensive, and honestly, I think they should be used only where really high IO capacity at expected latencies is required, and this didn’t seem to be the case.

AWS 117

AWS Hardware Storage Tuning 117

The Netflix TechBlog

DECEMBER 21, 2020

Use cases We found several use cases where a system like AutoOptimize can bring tons of value. Merge As the data lands into the data warehouse through real-time data ingestion systems, it comes in different sizes. Orient: Gather tuning parameters for a particular table that changed. Transparency to end-users.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

NOVEMBER 2, 2021

The coupling problem Until recently, video quality measurements were generated as part of our Reloaded production system. This system is responsible for processing incoming media files, such as video, audio and subtitles, and making them playable on the streaming service. We call this system Cosmos. The workflow is initiated.

Media 171

Media Innovation Metrics Latency 171

The Netflix TechBlog

AUGUST 30, 2021

System Setup Architecture The following diagram summarizes the architecture description: Figure 1: Event-sourcing architecture of the Device Management Platform. Fault Tolerance If the underlying KafkaConsumer crashes due to ephemeral system or network events, it should be automatically restarted. million elements.

Latency 213

Latency Traffic Transportation Hardware 213

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

JULY 26, 2021

This happens at an unprecedented scale and introduces many interesting challenges; one of the challenges is how to provide visibility of Studio data across multiple phases and systems to facilitate operational excellence and empower decision making. End-to-End Schema Evolution Schema is a key component of Data Mesh.

Big Data 253

Big Data Government Analytics Processing 253

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. top(1) showed that only the Cassandra database was consuming CPU. Microbenchmark os::javaTimeMillis() on both systems.

Speed 126

Speed Java AWS Virtualization 126

The Netflix TechBlog

FEBRUARY 9, 2021

In particular, it’s our job to design and build the systems and protocols that enable customers from all over the world to sign up for Netflix with the plan features and incentives that best suit their needs. Let’s take a deeper look at the architecture, protocols, and systems involved.

Engineering 142

Engineering Scalability Architecture Innovation 142

All Things Distributed

OCTOBER 2, 2017

Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

Internet 128

Internet Internet AWS Performance 128

HammerDB

MAY 5, 2019

Note that the main developer of HammerDB is Intel employee (#IAMINTEL) however HammerDB is a personal open source project and HammerDB has no optimization whatsoever for a database running on any particular architecture. So to test I took a system with Skylake CPUs and all storage on a P4800X SSD. In the recent MySQL 8.0.16

Testing 48

Testing Tuning Latency Storage 48

HammerDB

OCTOBER 18, 2019

HammerDB doesn’t publish competitive database benchmarks, instead we always encourage people to be better informed by running their own. So by default, the system boots into powersave. Note that turbostat will show you the maximum frequency your system can run out depending on how many cores are utilised.

Benchmarking 63

Benchmarking Performance Hardware C++ 63

IO River

NOVEMBER 15, 2023

This involves:Threat Intelligence: Keeping an eye on the latest types of attacks, whether it's a novel form of SQL injection or a new phishing tactic.Internal Audits: Regularly scanning your own system for vulnerabilities. Two of them are particularly gnarly: fine-tuning rules to perfection and managing a WAF over a multi-CDN architecture.

Traffic 52

Traffic Network Logistics Architecture 52

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. top(1) showed that only the Cassandra database was consuming CPU. Microbenchmark os::javaTimeMillis() on both systems.

Speed 40

Speed Java AWS Virtualization 40

IO River

NOVEMBER 2, 2023

This involves:Threat Intelligence: Keeping an eye on the latest types of attacks, whether it's a novel form of SQL injection or a new phishing tactic.Internal Audits: Regularly scanning your own system for vulnerabilities. Two of them are particularly gnarly: fine-tuning rules to perfection and managing a WAF over a multi-CDN architecture.

Traffic 40

Traffic Network Logistics Architecture 40

All Things Distributed

APRIL 9, 2015

The Amazon ML console and API provide data and model visualization tools, as well as wizards to guide you through the process of creating machine learning models, measuring their quality and fine-tuning the predictions to match your application requirements. The Amazon Elastic File System. Details on the AWS Blog. for a while already.

Lambda 122

Lambda Cloud IoT AWS 122

O'Reilly Software

APRIL 4, 2018

Google was among the pioneers that created “web scale” architectures to analyze the massive data sets that resulted from “crawling” the web that gave birth to Apache Hadoop, MapReduce, and NoSQL databases. Are there inherent time relationships in the messages that need to be preserved as they travel across the system? At least once?

Architecture 80

Architecture Scalability Google Operating System 80

HammerDB

NOVEMBER 5, 2018

If you are new to running Oracle, SQL Server, MySQL and PostgreSQL TPC-C workloads with HammerDB and have needed to investigate I/O performance the chances are that you have experienced waits on writing to the Redo, Transaction Log or WAL depending on the database you are testing. Oracle Log File Sync.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

The Morning Paper

MARCH 10, 2020

So if you want to design a system that will offer good real-world performance, it’s really useful to have benchmarks that accurately represent real-world workloads. This benchmark can synthetically generate more precise key-value queries that represent the reads and writes of key-value stores to the underlying storage system.

Benchmarking 59

Benchmarking Storage Cache Open Source 59