The Netflix TechBlog

AUGUST 30, 2021

System Setup Architecture The following diagram summarizes the architecture description: Figure 1: Event-sourcing architecture of the Device Management Platform. A timeline of the transition from Spring KafkaListener to Alpakka-Kafka is presented here for a better understanding of the motivations for the transition.

Latency 213

Latency Traffic Transportation Hardware 213

Adrian Cockcroft

MARCH 21, 2020

You need an “abtest” database service. In a small system this may be a table in a central relational database, but it’s best setup as a NoSQL data source using something like Amazon DynamoDB or Apache Cassandra, with a caching client library or at large scale a microservice data access layer.

Testing 52

Testing Tuning Database Cache 52

Scalegrid

DECEMBER 19, 2019

It’s a very simple utility that does exactly one thing – it sits between the database and the clients and speaks the PostgreSQL protocol, emulating a PostgreSQL server. Once the authentication succeeds: PgBouncer checks for a cached connection, with the same username+database combination. pgBouncer Architecture | Source.

Cache 246

Cache Architecture Database Servers 246

Brendan Gregg

APRIL 30, 2018

Some topics are still present at LISA, such as network management and uptime (reliability), but many others have been updated over the years. And we believe that everyone has valuable stories to share: Your tales of trial and error, analysis and workarounds, or architecture migrations, can happen at any company. ## Submit Your Work!

DevOps 43

DevOps Network Best Practices Programming 43

SQL Server According to Bob

FEBRUARY 15, 2018

Microsoft engineering is actually sending quite a few folks over the Atlantic to come talk about SQL Server 2017, SQL Server on Linux, GDPR, Performance, Security, Azure Data Lake, Azure SQL Database, Azure SQL Data Warehouse, and Azure CosmosDB. Adaptive query processing in SQL databases – Bob Ward and Conor Cunningham.

Engineering 40

Engineering Azure Best Practices Servers 40

Brendan Gregg

APRIL 29, 2018

Some topics are still present at LISA, such as network management and uptime (reliability), but many others have been updated over the years. And we believe that everyone has valuable stories to share: Your tales of trial and error, analysis and workarounds, or architecture migrations, can happen at any company. ## Submit Your Work!

DevOps 40

DevOps Network Best Practices Programming 40

The Netflix TechBlog

JUNE 13, 2023

Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. They enable us to further fine-tune and configure the system, ensuring the new changes are integrated smoothly and seamlessly.

Traffic 279

Traffic Metrics Systems Strategy 279

Dynatrace

MARCH 11, 2021

During the implementation of the real-time visualization I presented in part three , I had an idea for another visualization; I wanted to visualize the number of detected problems globally, for a longer timeframe. I wanted to understand how I could tune Dynatrace’s problem detection, but to do that I needed to understand the situation first.

Tuning 262

Tuning Architecture Monitoring Big Data 262

The Netflix TechBlog

FEBRUARY 9, 2021

In particular, we’ll define plans and offers, review the legacy architecture and some of its shortcomings, and dig into our new architecture and some of its advantages. When we merge these two concepts together and present them to the customer, we have the plan selection page (shown above).

Engineering 142

Engineering Scalability Architecture Innovation 142

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. Several promising ideas were abandoned owing to the outsized work needed to overcome architectural limitations. The microservice architecture provides strong decoupling between services. 264, AV1, etc.). depending on the use case.

Processing 229

Processing Media Latency Innovation 229

The Netflix TechBlog

APRIL 12, 2022

In a federated graph architecture, how can we answer such a query given that each entity is served by its own service? Architecture Putting this all together, below you can see a simplified view of the architecture. By transacting with a database which is monitored by a CDC connector that creates events, or b.

Engineering 263

Engineering Architecture Java Infrastructure 263

Percona

JULY 30, 2018

Please join Percona’s CEO, Peter Zaitsev as he presents Using MySQL for Distributed Database Architectures on Tuesday, July 31st, 2018 at 7:00 AM PDT (UTC-7) / 10:00 AM EDT (UTC-4). Such architectures bring many benefits (such as scalability and resiliency), but can also bring a lot of pain if incorrectly designed and executed.

Architecture 41

Architecture Database Open Source Internet 41

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 21, 2020

This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture. Other than these principles, there are some other design considerations to support and enable: Multi-tenancy with database and table prioritization.

Storage 203

Storage Latency Efficiency Data Engineering 203

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. The Linux default is usually 60 , which is not ideal for database usage. Spoiler alert: This post focuses on MongoDB 3.6.X

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

OCTOBER 27, 2020

At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. For how our machine learning recommendation systems leverage our key-value stores, please see more details on this presentation. It leverages Netflix Scheduler for scheduling the Bulldozer Jobs.

Latency 243

Latency Storage Big Data Tuning 243

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Dynatrace

DECEMBER 8, 2020

More modern tools can provide runtime insights into certain platforms, like Kubernetes or containers, but are still limited in their ability to detect which libraries are actually used vs. those that are present, but unused. Stay tuned – this is only the start. They also can’t provide deep insights unless you have source code access.

Cloud 287

Cloud Open Source Internet Internet 287

All Things Distributed

DECEMBER 12, 2013

We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability. DynamoDB stores information as database tables, which are collections of individual items.

Games 130

Games Scalability Database Retail 130

O'Reilly Software

APRIL 4, 2018

Considerations for setting the architectural foundations for a fast data platform. Google was among the pioneers that created “web scale” architectures to analyze the massive data sets that resulted from “crawling” the web that gave birth to Apache Hadoop, MapReduce, and NoSQL databases. Back in the days of Web 1.0,

Architecture 80

Architecture Scalability Google Operating System 80

SQL Performance

MARCH 8, 2021

This can make it difficult to draw sound performance-tuning conclusions. The following examples use the public Stack Overflow 2013 database ( download details ), with a single index added: CREATE INDEX PP ON dbo. . ( Achieving complete consistency might be difficult, or even impossible without major architectural changes.

Servers 95

Servers Tuning Architecture Processing 95

HammerDB

OCTOBER 12, 2018

Similarly for this guide MySQL can be swapped for a mySQL based databases such as MariaDB. As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. For this reason sysbench presents a vastly more simplified workload than HammerDB. . 10 rows in set (0.02

Best Practices 40

Best Practices Scalability Performance C++ 40

Percona

NOVEMBER 3, 2023

Rather than listing the concepts, function calls, etc, available in Citus, which frankly is a bit boring, I’m going to explore scaling out a database system starting with a single host. I won’t cover all the features but show just enough that you’ll want to see more of what you can learn to accomplish for yourself.

Database 107

Database Benchmarking Latency C++ 107

SQL Server According to Bob

JANUARY 11, 2020

Microsoft SQL Server I/O Basics Author: ​​ Bob Dorr, Microsoft SQL Server Escalation Published: ​​ December, 2004 SUMMARY: ​​ Learn the I/O requirements for Microsoft SQL Server database file operations. This will help you increase system performance and avoid I/O environment errors.

Servers 40

Servers Cache Media Hardware 40