Scalegrid

DECEMBER 14, 2023

How the MongoDB timeout is set up can significantly affect your application’s performance, no matter if you are an experienced MongoDB user or just starting with NoSQL databases. Typical applications are interacting with different database servers based on the business logic. x+ in Java).

Java 130

Java Network Servers Database 130

Scalegrid

OCTOBER 17, 2019

In that environment, the first PostgreSQL developers decided forking a process for each connection to the database is the safest choice. It would be a shame if your database crashed, after all. On modern Linux systems, the difference in overhead between forking a process and creating a thread is much lesser than it used to be.

Architecture 200

Architecture Database Latency Servers 200

Percona

NOVEMBER 2, 2023

MongoDB is a dynamic database system continually evolving to deliver optimized performance, robust security, and limitless scalability. Our new eBook, “ From Planning to Performance: MongoDB Upgrade Best Practices ,” guides you through the entire process to ensure your database’s long-term success.

Best Practices 91

Best Practices Hardware Tuning Scalability 91

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? That’s a heritage of the LAMP model when the same server would host both the database and the web server.

Tuning 138

Tuning Cache Servers Benchmarking 138

Dynatrace

MAY 17, 2023

Think about items such as general system metrics (for example, CPU utilization, free memory, number of services), the connectivity status, details of our web server, or even more granular in-application tasks like database queries. This leads us to the process page of our specific Apache instance.

Metrics 168

Metrics Monitoring Database Network 168

Dynatrace

JANUARY 26, 2021

REST APIs, authentication, databases, email, and video processing all have a home on serverless platforms. The Serverless Process. When an application is triggered, it can cause latency as the application starts. The average request is handled, processed, and returned quickly. Services scale to meet demand.

Serverless 242

Serverless Efficiency Lambda Azure 242

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases. Benchmark before you decide.

Performance 129

Performance Hardware Tuning Storage 129

Scalegrid

MARCH 22, 2024

Crafting Your Multi-Cloud Blueprint Formulating a comprehensive multi-cloud framework is an intricate process that requires a business’s specific requirements, choosing suitable cloud services, and optimizing for performance and affordability. They can also bolster uptime and limit latency issues or potential downtimes.

Strategy 130

Strategy Cloud Government Innovation 130

Percona

AUGUST 10, 2018

The performance of a PostgreSQL database can be compromised by dead tuples, since they continue to occupy space and can lead to bloat. Now, though, it’s time to look at autovacuum for postgres, and the internals you to know to maintain a high performance PostgreSQL database needed by demanding applications. What is autovacuum ?

Tuning 45

Tuning Cache Database Storage 45

The Netflix TechBlog

AUGUST 30, 2021

The challenge, then, is to be able to ingest and process these events in a scalable manner, i.e., scaling with the number of devices, which will be the focus of this blog post. In-Order Processing The semantics of correct device information updates ingestion requires that messages be consumed in the order that they are produced.

Latency 213

Latency Traffic Transportation Hardware 213

The Netflix TechBlog

OCTOBER 19, 2020

Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices. The process started with manual pull of member account information that was part of the session.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. The Linux default is usually 60 , which is not ideal for database usage. The CFQ works well for many general use cases but lacks latency guarantees.

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

JANUARY 10, 2024

Future blogs will provide deeper dives into each service, sharing insights and lessons learned from this process. The Netflix video processing pipeline went live with the launch of our streaming service in 2007. The Netflix video processing pipeline went live with the launch of our streaming service in 2007.

Processing 229

Processing Media Latency Innovation 229

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. top(1) showed that only the Cassandra database was consuming CPU. What about short-lived processes, like a service restarting in a loop? A quick check of basic performance statistics showed over 30% higher CPU consumption.

Speed 126

Speed Java AWS Virtualization 126

HammerDB

OCTOBER 18, 2019

HammerDB doesn’t publish competitive database benchmarks, instead we always encourage people to be better informed by running their own. hardware limits: 1000 MHz - 4.00 GHz available cpufreq governors: performance powersave current policy: frequency should be within 1000 MHz and 4.00 hardware limits: 1000 MHz - 4.00

Benchmarking 63

Benchmarking Performance Hardware C++ 63

IO River

NOVEMBER 15, 2023

It's an ongoing, iterative process. Two of them are particularly gnarly: fine-tuning rules to perfection and managing a WAF over a multi-CDN architecture. Similarly, a rule that works perfectly on one CDN may trigger false positives or negatives on another due to slight variations in how each CDN processes web traffic.

Traffic 52

Traffic Network Logistics Architecture 52

O'Reilly Software

APRIL 4, 2018

We live in the era of the connected experience, where our daily interactions with the world can be digitized, collected, processed, and analyzed to generate valuable insights. Accumulating all this data to process overnight is not an option anymore. Do we need to process each record individually? Process tolerance.

Architecture 80

Architecture Scalability Google Operating System 80

IO River

NOVEMBER 2, 2023

It's an ongoing, iterative process. Two of them are particularly gnarly: fine-tuning rules to perfection and managing a WAF over a multi-CDN architecture. SQL Injection involves inserting malicious SQL queries into an input field, tricking the database into revealing sensitive information.Imagine a login page.

Traffic 40

Traffic Network Logistics Architecture 40

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. top(1) showed that only the Cassandra database was consuming CPU. What about short-lived processes, like a service restarting in a loop? A quick check of basic performance statistics showed over 30% higher CPU consumption.

Speed 40

Speed Java AWS Virtualization 40

HammerDB

NOVEMBER 5, 2018

If you are new to running Oracle, SQL Server, MySQL and PostgreSQL TPC-C workloads with HammerDB and have needed to investigate I/O performance the chances are that you have experienced waits on writing to the Redo, Transaction Log or WAL depending on the database you are testing.

Benchmarking 40

Benchmarking Database C++ Virtualization 40

The Netflix TechBlog

JUNE 13, 2023

Replay traffic testing gives us the initial foundation of validation, but as our migration process unfolds, we are met with the need for a carefully controlled migration process. A process that doesn’t just minimize risk, but also facilitates a continuous evaluation of the rollout’s impact.

Traffic 279

Traffic Metrics Systems Strategy 279

The Netflix TechBlog

JULY 26, 2021

Operational Reporting is a reporting paradigm specialized in covering high-resolution, low-latency data sets, serving detailed day-to-day activities¹ and processes of a business domain. In the initial stage, data consumers set up ETL pipelines directly pulling data from databases.

Big Data 253

Big Data Government Analytics Processing 253

The Netflix TechBlog

DECEMBER 21, 2020

There are several benefits of such optimizations like saving on storage, faster query time, cheaper downstream processing, and an increase in developer productivity by removing additional ETLs written only for query performance improvement. Orient: Gather tuning parameters for a particular table that changed. Transparency to end-users.

Storage 203

Storage Latency Efficiency Data Engineering 203

SQL Server According to Bob

JANUARY 11, 2020

Microsoft SQL Server I/O Basics Author: ​​ Bob Dorr, Microsoft SQL Server Escalation Published: ​​ December, 2004 SUMMARY: ​​ Learn the I/O requirements for Microsoft SQL Server database file operations. This will help you increase system performance and avoid I/O environment errors.

Servers 40

Servers Cache Media Hardware 40

HammerDB

OCTOBER 12, 2018

Similarly for this guide MySQL can be swapped for a mySQL based databases such as MariaDB. As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. As is exactly the same with PostgreSQL for system choice a 2 socket system is optimal for MySQL OLTP performance.

Best Practices 40

Best Practices Scalability Performance C++ 40

The Netflix TechBlog

OCTOBER 27, 2020

Usually Data scientists and engineers write Extract-Transform-Load (ETL) jobs and pipelines using big data compute technologies, like Spark or Presto , to process this data and periodically compute key information for a member or a video. The processed data is typically stored as data warehouse tables in AWS S3.

Latency 243

Latency Storage Big Data Tuning 243

SQL Server According to Bob

JANUARY 11, 2020

KB sectors run on smaller sectors   14 System and sample databases   15 Determining the formatted sector size of database   15 What sector sizes does SQL Server support? DBCC Page Auditing DBCC page auditing is the process of executing common DBCC consistency checks as pages are read from disk.

Servers 40

Servers Cache Database Media 40

The Netflix TechBlog

MAY 14, 2021

Once you finally find useful identifiers, you may begin writing SQL queries against your production database to find out what went wrong. We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases.

Traffic 160

Traffic Latency Java Google 160

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Triggering repairs at any time.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Triggering repairs at any time.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

AUGUST 13, 2020

You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? Telltale learns what constitutes typical health for an application, no alert tuning required. For example, a latency increase is less critical than error rate increase and some error codes are less critical than others.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

The Netflix TechBlog

FEBRUARY 18, 2022

Event Prioritization Considering the use cases were wide ranging both in terms of their sources and their importance, we built segmentation into the event processing. We thus assigned a priority to each use case and sharded event traffic by routing to priority-specific queues and the corresponding event processing clusters.

Systems 334

Systems Traffic Architecture Mobile 334

The Netflix TechBlog

NOVEMBER 2, 2021

This system is responsible for processing incoming media files, such as video, audio and subtitles, and making them playable on the streaming service. This enables us to use our scale to increase throughput and reduce latencies. Here is a step-by-step description of the processes involved: 1. The workflow is initiated.

Media 171

Media Innovation Metrics Latency 171

The Netflix TechBlog

FEBRUARY 9, 2021

Introduction and account creation Highlight our value propositions and begin the account creation process. Given the flow and mode, the Orchestration Service can then process the request. Retrieve eligible SKUs from SKU Platform The SKU Platform consists of a rules engine, a database, and application logic.

Engineering 142

Engineering Scalability Architecture Innovation 142

All Things Distributed

OCTOBER 2, 2017

Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

APRIL 9, 2015

The Amazon ML console and API provide data and model visualization tools, as well as wizards to guide you through the process of creating machine learning models, measuring their quality and fine-tuning the predictions to match your application requirements. Details on the AWS Blog. The Amazon Elastic File System. for a while already.

Lambda 122

Lambda Cloud IoT AWS 122