Dynatrace

JANUARY 31, 2024

GenAI is prone to erratic behavior due to unforeseen data scenarios or underlying system issues. The RAG process begins by summarizing and converting user prompts into queries that are sent to a search platform that uses semantic similarities to find relevant data in vector databases, semantic caches, or other online data sources.

Cache 209

Cache Azure Infrastructure Monitoring 209

Scalegrid

MARCH 28, 2024

Redis Data Types and Structures The design of Redis’s data structures emphasizes versatility. It is designed to cache plain text values, offering fast read and write access to frequently accessed data. Advanced Redis Features Showdown Big data center concept, cloud database, server power station of the future.

Cache 130

Cache Storage Scalability Architecture 130

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. By integrating with studio content systems, we enabled the pipeline to leverage rich metadata from the creative side and create more engaging member experiences like interactive storytelling.

Processing 229

Processing Media Latency Innovation 229

The Morning Paper

MARCH 3, 2020

Millions of tiny databases , Brooker et al., It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. NSDI’20. This paper is a real joy to read. Requirements.

Database 106

Database AWS Network Design 106

The Morning Paper

SEPTEMBER 3, 2019

SLOG: serializable, low-latency, geo-replicated transactions Ren et al., SLOG is another research system motivated by the needs of the application developer (aka, user!). Building correct applications is much easier when the system provides strict serializability guarantees. VLDB’19. Is my data at home?

Latency 70

Latency Processing Benchmarking Systems 70

HammerDB

OCTOBER 23, 2023

HammerDB uses stored procedures to achieve maximum throughput when benchmarking your database. HammerDB has always used stored procedures as a design decision because the original benchmark was implemented as close as possible to the example workload in the TPC-C specification that uses stored procedures. What is a stored procedure?

Benchmarking 62

Benchmarking Database Network C++ 62

Scalegrid

JANUARY 8, 2024

AI-driven cloud solutions like ScaleGrid offer a diverse range of database hosting options, robust infrastructure optimized for scalability and security, and enable significant cost reductions, supporting businesses in efficient growth and improved ROI. These services are tailored to meet various business requirements.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Note that the intent of tuning the settings is not exclusively about improving performance but also enhancing the high availability and resilience of the MongoDB database. 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

MARCH 7, 2023

We designed a unique concept called Annotation Operations which allows teams to create data pipelines and easily write annotations without worrying about access patterns of their data from different applications. There are many naive solutions possible for this problem for example: Write different runs in different databases.

Media 264

Media Latency Architecture Database 264

The Netflix TechBlog

JUNE 13, 2023

This is where large-scale system migrations come into play. By collecting and analyzing key performance metrics of the service over time, we can assess the impact of the new changes and determine if they meet the availability, latency, and performance requirements. But what happens when this machinery needs a transformation?

Traffic 279

Traffic Metrics Systems Strategy 279

Dynatrace

JANUARY 31, 2024

Data observability involves monitoring and managing the internal state of data systems to gain insight into the data pipeline, understand how data evolves, and identify any issues that could compromise data integrity or reliability. An erroneous change in the database system leads to a subset of the data being categorized incorrectly.

DevOps 197

DevOps Analytics Airlines Metrics 197

Scalegrid

MARCH 22, 2024

They can also bolster uptime and limit latency issues or potential downtimes. Ready to take your database management to the next level with ScaleGrid’s cutting-edge solutions? Register now for free and experience the seamless operation of your databases across multi-cloud and hybrid-cloud systems.

Strategy 130

Strategy Cloud Government Innovation 130

Highly Scalable

SEPTEMBER 18, 2012

As such, it encompasses distributed system coordination, failover, resource management and many other capabilities. These developments gradually highlight a system of relevant database building blocks with proven practical efficiency. System Coordination. Scalability is one of the main drivers of the NoSQL movement.

Database 213

Database Latency C++ Scalability 213

Percona

APRIL 13, 2023

PostgreSQL is a popular open source relational database management system that is widely used for storing and managing data. Replication lag is the delay between the time when data is written to the primary database and the time when it is replicated to the standby databases. What is replication lag?

Latency 79

Latency Tuning Open Source Network 79

Percona

SEPTEMBER 1, 2023

MySQL performance tuning offers several significant advantages for effective database management and optimization. Enhanced Database Efficiency By adjusting configuration settings, you can markedly enhance the overall efficiency of your MySQL database. Experiencing database performance issues?

Tuning 52

Tuning Database Performance Hardware 52

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. Such demanding use cases place a great value on systems capable of fast and reliable execution, a need that spans across various industry segments.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

MAY 2, 2023

As a MongoDB user, it’s crucial to ensure that your data is safe and secure in the event of a disaster or system failure. That’s why it’s essential to implement the best practices and strategies for MongoDB database backups. Why are MongoDB database backups important? mongodump --host=mongodb1.example.net

Best Practices 52

Best Practices Database Storage Servers 52

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? What we should really compare is the MySQL and Aurora database engines provided by Amazon RDS. How do I choose which one to use?

AWS 52

AWS Database Serverless Storage 52

Dynatrace

APRIL 5, 2021

These include website hosting, database management, backup and restore, IoT capabilities, e-commerce solutions, app development tools and more, with new services released regularly. You will likely need to write code to integrate systems and handle complex tasks or incoming network requests. A new record entering a database table.

Lambda 183

Lambda AWS Serverless Hardware 183

Randon ASCII

SEPTEMBER 8, 2019

The fact that this shows up as CPU time suggests that the reads were all hitting in the system cache and the CPU time was the kernel overhead (note ntoskrnl.exe on the first sampled call stack) of grabbing data from the cache. Remember that these are calls to the operating system – kernel calls. Think about that for a moment.

Latency 79

Latency Cache Programming Operating System 79

The Netflix TechBlog

NOVEMBER 2, 2021

For example, when we design a new version of VMAF, we need to effectively roll it out throughout the entire Netflix catalog of movies and TV shows. This article explains how we designed microservices and workflows on top of the Cosmos platform to bolster such video quality innovations. We call this system Cosmos.

Media 171

Media Innovation Metrics Latency 171

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. Integrating Existing Systems When selecting a tool for integrating a hybrid cloud, evaluating several key aspects is important. The tool must be compatible with your current systems.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

O'Reilly Software

JUNE 12, 2018

The benefits of modeling data as events as a mechanism to evolve our software systems. Data is all-important—vital for the continued success of our businesses—but has also been seen as a massive constraint in how we design and evolve our systems. All too often, the software wasn’t designed in a way that made it easy to ship.

Database 72

Database Cache Architecture Latency 72

The Netflix TechBlog

DECEMBER 21, 2020

This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture. Use cases We found several use cases where a system like AutoOptimize can bring tons of value. We can also reorganize the metadata to make file scanning much faster.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

AUGUST 30, 2021

System Setup Architecture The following diagram summarizes the architecture description: Figure 1: Event-sourcing architecture of the Device Management Platform. Fault Tolerance If the underlying KafkaConsumer crashes due to ephemeral system or network events, it should be automatically restarted.

Latency 213

Latency Traffic Transportation Hardware 213

Percona

DECEMBER 11, 2023

Here, we will discuss a notable new feature in Amazon RDS, the Dedicated Log Volume (DLV), that has been introduced to boost database performance. A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables.

AWS 93

AWS Benchmarking Performance Traffic 93

Percona

DECEMBER 26, 2023

This PoC demonstrates how to install and configure pg_stat_monitor in order to extract useful and actionable metrics from a PostgreSQL database and display them on a Grafana dashboard. Grafana database backend: Prometheus version 2.15.2+d About the environment Grafana: version 10.0.0 tps, lat 11.718 ms stddev 3.951 progress: 4440.0

Benchmarking 119

Benchmarking Metrics C++ Database 119

Percona

JULY 20, 2023

It is very common to see many infrastructure layers standing between a PostgreSQL database and the Application server. At Percona Support, we use pg_gather as the method to collect and study the wait events because it is a standalone SQL script and doesn’t need to install anything on the database systems.

Network 61

Network Performance Latency Servers 61

The Morning Paper

FEBRUARY 6, 2020

’ Stateless is fine until you need state, at which point the coarse-grained solutions offered by current platforms limit the kinds of application designs that work well. On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access.

Lambda 98

Lambda Serverless Cache Latency 98

High Scalability

SEPTEMBER 8, 2018

New databases used to be announced seemingly every week. While database neogenesis has slowed down considerably, it has not gone necrotic. The issue is that Anna is now orders of magnitude more efficient than competing systems, in addition to being orders of magnitude faster.

Storage 199

Storage Performance AWS Efficiency 199

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. How Bulldozer leverages Spark, Protobuf and KV DAL for moving the data.

Latency 243

Latency Storage Big Data Tuning 243

Percona

JULY 17, 2023

MongoDB Community Edition software might set the stage for achieving your high-volume database goals, but you quickly learn that its features fall short of your enterprise needs. The popularity of MongoDB MongoDB has emerged as a popular database platform. 5 among all database management systems and No. It ranks No.

Open Source 74

Open Source Database Scalability Software 74

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Now let’s look at how we designed the tracing infrastructure that powers Edgar. Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

MAY 2, 2023

At the same time that I see database engineers relying on the tool, sites such as StackOverflow are banning ChatGPT. ChatGPT: The innodb_redo_log_capacity parameter specifies the maximum size of the InnoDB redo log buffer, which is used to store changes made to the database before they are written to disk. What could it be?

Social Media 119

Social Media Database Servers Cache 119

The Morning Paper

SEPTEMBER 1, 2019

IPA: invariant-preserving applications for weakly consistent replicated databases Balegas et al., For bonus points, revisit your application design so that more parts of it can be coordination free. VLDB’19. IPA for developers, happy days! An example makes all this much clearer. That’s a pretty neat idea!

Database 42

Database Latency Games Design 42