The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. Therefore, dumps are needed to capture the full state of a source. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. Therefore, dumps are needed to capture the full state of a source. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

Percona

APRIL 17, 2023

We’ll also go over some best practices for MongoDB security as well as MongoDB data modeling. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. Spoiler alert: This post focuses on MongoDB 3.6.X

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

AUGUST 13, 2020

You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? This is a critical application so you drag yourself out of bed, open your laptop, and start poring through dashboards for more info. Telltale combines a variety of data sources to create a holistic view of an application’s health.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Traces collected from various microservices are ingested in a stream processing manner into the data store.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? To be fair, that is also true with PostgreSQL; it hasn’t been tuned either, and it, too, can also perform much better.

Tuning 134

Tuning Cache Servers Benchmarking 134

Percona

NOVEMBER 3, 2023

When it comes to Citus, successfully building out and scaling a PostgreSQL cluster across multiple nodes and even across data centers can feel, at times, to be an art form because there are so many ways of building it out. The following invocation generates almost 4GB of data.

Database 102

Database Benchmarking Latency C++ 102

Scalegrid

MARCH 22, 2024

They can also bolster uptime and limit latency issues or potential downtimes. This covers everything from tech specs and what you expect from the service to security, how you’ll handle data, and how services will be managed. Each private cloud setup has perks and benefits, especially for handling specific tasks.

Strategy 130

Strategy Cloud Government Innovation 130

The Netflix TechBlog

NOVEMBER 2, 2021

It has become a de facto standard for perceptual quality measurements within Netflix and, thanks to its open-source nature , throughout the video industry. This enables us to use our scale to increase throughput and reduce latencies. The VQS API layer will translate the external request into VQS-specific data models.

Media 171

Media Innovation Metrics Latency 171

Percona

MARCH 23, 2023

Nowadays, solid-state drives (SSDs) or non-volatile memory express (NVMe) drives are preferred over traditional hard disk drives (HDDs) for database servers due to their faster read and write speeds, lower latency, and improved reliability. If you see concurrency issues, you can tune this variable.

Performance 123

Performance Hardware Tuning Storage 123

Abhishek Tiwari

JULY 12, 2018

Building data pipelines can offer strategic advantages to the business. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines. Data pipeline initiatives are generally unfinished projects. In this post, we will discuss why you should avoid building data pipelines in first place.

Latency 63

Latency Analytics Scalability Engineering 63

Smashing Magazine

SEPTEMBER 29, 2021

Compression achieved is relevant because the higher the compression, the smaller the file size, and the lower the data required to transfer the image on the network. Ideally, compression should not result in any loss of image data; it should be lossless. It was released in February 2019 by the Alliance for Open Media (AOMedia).

Open Source 144

Open Source Speed Website Google 144

HammerDB

MAY 5, 2019

Note that the main developer of HammerDB is Intel employee (#IAMINTEL) however HammerDB is a personal open source project and HammerDB has no optimization whatsoever for a database running on any particular architecture. x86_64/data language=/home/mysql/mysql-8.0.16/mysql-8.0.16-linux-glibc2.12-x86_64/share/english linux-glibc2.12-x86_64.

Testing 48

Testing Tuning Latency Storage 48

Smashing Magazine

AUGUST 22, 2021

Because we are dealing with network protocols here, we will mainly look at network aspects, of which two are most important: latency and bandwidth. Latency can be roughly defined as the time it takes to send a packet from point A (say, the client) to point B (the server). Two-way latency is often called round-trip time (RTT).

Performance 106

Performance Network Latency Servers 106

O'Reilly

JANUARY 16, 2024

I’ve been diligent about data backup throughout my life, but my files are fragmented amongst different media over the years—burning CDs and DVDs back in the day, several generations of external hard drives (that are in various states of decay), university servers, Dropbox, and other cloud services.

Entertainment 72

Entertainment Internet Internet Design 72

HammerDB

OCTOBER 18, 2019

Nevertheless in this blog sometimes we do publish performance data to highlight best practices or potential configuration pitfalls and although we’ve mentioned this one before it is worth dedicating an entire post to it as this issue seems to appear numerous times running database workloads on Linux. hardware limits: 1000 MHz - 4.00

Benchmarking 63

Benchmarking Performance Hardware C++ 63

The Morning Paper

MARCH 10, 2020

The tracing, replay, and analysis tools developed for this work are released in open source as part of the latest RocksDB release , and the new benchmark is now part of the db_bench benchmarking tool. That matters if your store reads data in (multi-key) blocks. It matters because RocksDB reads from storage in data blocks (e.g.

Benchmarking 59

Benchmarking Storage Cache Open Source 59

KeyCDN

AUGUST 15, 2019

Before you begin tuning your website or application, you must first figure out which metrics matter most to your users and establish some achievable benchmarks. An interruption in data flow due to limited capacity is called a bottleneck. Test and collect data: Let your testing software do all the hard work, then assess the results.

Testing Tools 55

Testing Tools Best Practices Performance Testing Testing 55

Brendan Gregg

OCTOBER 8, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. Back then I could already tell if disks were seeking by interpreting iostat(1) output: seeing high disk latency but small I/O. It's shaping up to be a DTrace version 2.0: eBPF does more.

C++ 110

C++ Virtualization Programming Latency 110

Smashing Magazine

SEPTEMBER 6, 2021

For example, you could reduce compression efficiency , because that works better with more data. Finally, not inlining resources has an added latency cost because the file needs to be requested. Server push allows a server to send response data without first waiting for a request from the client. Server Push and First Flight.

Network 105

Network Servers Cache Traffic 105

Brendan Gregg

OCTOBER 7, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. Back then I could already tell if disks were seeking by interpreting iostat(1) output: seeing high disk latency but small I/O. It's shaping up to be a DTrace version 2.0: eBPF does more.

C++ 40

C++ Virtualization Programming Latency 40