Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? A finely tuned database processes queries more efficiently, leading to swifter results.

Tuning 52

Tuning Database Performance Hardware 52

The Netflix TechBlog

MARCH 1, 2021

It supports both high throughput services that consume hundreds of thousands of CPUs at a time, and latency-sensitive workloads where humans are waiting for the results of a computation. The subsystems all communicate with each other asynchronously via Timestone, a high-scale, low-latency priority queuing system.

Serverless 228

Serverless Media Latency Social Media 228

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. The processed data is typically stored as data warehouse tables in AWS S3. How Bulldozer leverages Spark, Protobuf and KV DAL for moving the data.

Latency 243

Latency Storage Big Data Tuning 243

Dynatrace

JULY 18, 2023

As software development grows more complex, managing components using an automated onboarding process becomes increasingly important. Efficient environment configuration at scale One of software engineers’ most significant challenges is managing the numerous tools and technologies required for the software product lifecycle.

Best Practices 195

Best Practices Code Infrastructure Latency 195

The Netflix TechBlog

SEPTEMBER 24, 2021

By Xiaomei Liu , Rosanna Lee , Cyril Concolato Introduction Behind the scenes of the beloved Netflix streaming service and content, there are many technology innovations in media processing. Packaging has always been an important step in media processing. Uploading and downloading data always come with a penalty, namely latency.

Cloud 237

Cloud Media Storage Cache 237

Dynatrace

JULY 24, 2023

We will also explore the evolution of DevOps automation and the significance of data-driven answers in unlocking streamlined, automated DevOps and SRE processes. Business process automation Business process automation is the foundation for improving operational efficiency. Has the user purchased this product before?

DevOps 225

DevOps Traffic Efficiency Servers 225

The Netflix TechBlog

JUNE 13, 2023

Replay traffic testing gives us the initial foundation of validation, but as our migration process unfolds, we are met with the need for a carefully controlled migration process. A process that doesn’t just minimize risk, but also facilitates a continuous evaluation of the rollout’s impact.

Traffic 279

Traffic Metrics Systems Strategy 279

The Morning Paper

JULY 7, 2019

Software engineering for machine learning: a case study Amershi et al., Previously on The Morning Paper we’ve looked at the spread of machine learning through Facebook and Google and some of the lessons learned together with processes and tools to address the challenges arising. A general process. ICSE’19.

Software Engineering 92

Software Engineering Engineering Software Software 92

The Netflix TechBlog

MAY 14, 2021

Prodicle is one of the many applications that is at the exciting intersection of connecting the world of content productions to Netflix Studio Engineering. We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases.

Traffic 160

Traffic Latency Java Google 160

The Netflix TechBlog

FEBRUARY 18, 2022

Personalized Experience Refresh Netflix Recommendation engine continuously refreshes recommendations for every member. Event Prioritization Considering the use cases were wide ranging both in terms of their sources and their importance, we built segmentation into the event processing.

Systems 334

Systems Traffic Architecture Mobile 334

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which The process started with manual pull of member account information that was part of the session. We needed to increase engineering productivity via distributed request tracing.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

MAY 5, 2021

As more organizations respond to the pressure to release better software faster, there is an increasing need to build quality gates into every stage of BizDevOps processes , from early development to deployment. Automating quality gates creates reliable checks and balances and speeds up the process by avoiding manual intervention.

Benchmarking 188

Benchmarking Latency Speed Software 188

The Netflix TechBlog

AUGUST 22, 2019

Baking Windows with Packer By Justin Phelps and Manuel Correa Customizing Windows images at Netflix was a manual, error-prone, and time consuming process. A key responsibility of Netflix engineers is identifying gaps and pain points in the development and operation of services. Last year, we decided to improve the AMI baking process.

DevOps 207

DevOps AWS Tuning Infrastructure 207

The Netflix TechBlog

DECEMBER 21, 2020

There are several benefits of such optimizations like saving on storage, faster query time, cheaper downstream processing, and an increase in developer productivity by removing additional ETLs written only for query performance improvement. Orient: Gather tuning parameters for a particular table that changed.

Storage 203

Storage Latency Efficiency Data Engineering 203

Percona

NOVEMBER 2, 2023

Upgrading to the newest release of MongoDB is the key to unlocking its full potential, but it’s not as simple as clicking a button; it requires meticulous planning, precise execution, and a deep understanding of the upgrade process. We walk you through the essential steps required.

Best Practices 84

Best Practices Hardware Tuning Scalability 84

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? To be fair, that is also true with PostgreSQL; it hasn’t been tuned either, and it, too, can also perform much better.

Tuning 134

Tuning Cache Servers Benchmarking 134

The Netflix TechBlog

AUGUST 13, 2020

You’re half awake and wondering, “Is there really a problem or is this just an alert that needs tuning? Over the years we’ve learned from on-call engineers about the pain points of application monitoring: too many alerts, too many dashboards to scroll through, and too much configuration and maintenance. By Andrei U.,

Monitoring 210

Monitoring Tuning Traffic Metrics 210

The Netflix TechBlog

AUGUST 29, 2023

For Inter-Process Communication (IPC) between services, we needed the rich feature set that a mid-tier load balancer typically provides. At the time, the service mesh effort was still being bootstrapped, with only a few engineers working on it. This is the first in a series of posts on our journey to service mesh, so stay tuned.

Traffic 220

Traffic Latency Cloud C++ 220

John McCalpin

APRIL 25, 2023

This metric is interesting because we don’t always have the luxury of parallelizing every application we run, and our operating systems almost always process each call (e.g., The NEC Vector Engine processors provide a demonstration of very high single-core bandwidth. Stay tuned! On a VE20B (8 cores, 1.6

Benchmarking 97

Benchmarking Cache Latency Tuning 97

The Netflix TechBlog

JULY 21, 2022

Since memory management is not something one usually associates with classification problems, this blog focuses on formulating the problem as an ML problem and the data engineering that goes along with it. We now explore each of these components individually, while highlighting the nuances of the data pipeline and pre-processing.

Big Data 179

Big Data Cache Engineering Data Engineering 179

Percona

MARCH 23, 2023

As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases. But also larger log files mean that the recovery process will be slower in case of a crash. If you see concurrency issues, you can tune this variable. However, this variable has been deprecated since 8.0.30.

Performance 123

Performance Hardware Tuning Storage 123

Dynatrace

DECEMBER 2, 2021

And why have SLOs and SLIs become so important as teams automate processes to consistently meet SLAs and error budgets? As defined by Gartner , service-level objectives are an agreed-upon target within an SLA that must be achieved for each activity, function, and process to provide the best opportunity for customer success.

Metrics 196

Metrics Best Practices DevOps Infrastructure 196

The Netflix TechBlog

NOVEMBER 2, 2021

This system is responsible for processing incoming media files, such as video, audio and subtitles, and making them playable on the streaming service. This enables us to use our scale to increase throughput and reduce latencies. Here is a step-by-step description of the processes involved: 1. We call this system Cosmos.

Media 171

Media Innovation Metrics Latency 171

The Morning Paper

NOVEMBER 10, 2019

You need a lot of software engineers and the willingness to rewrite a lot of software to entertain that idea. Here are the bombshell paragraphs: Our datacenter applications seek ever more CPU-efficient and lower-latency communication, which Pony Express delivers. The desire for CPU efficiency and lower latencies is easy to understand.

Network 92

Network Transportation Latency Entertainment 92

The Netflix TechBlog

NOVEMBER 2, 2020

As engineers at Netflix, we are constantly reevaluating how to redesign traffic management. In Netflix engineering, we’re driven by ensuring Netflix is there when you need it to be. Those two metrics are approximate indicators of failures and latency.

Traffic 252

Traffic Metrics Infrastructure Architecture 252

Dynatrace

APRIL 26, 2022

STM generates traffic that replicates the typical path or behavior of a user on a network to measure performance for example, response times, availability, packet loss, latency, jitter, and other variables). One use case for STM is to model the behavior of a customer in the form of a flow of transactions along the buyer’s journey.

Monitoring 209

Monitoring Social Media IoT Metrics 209

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. The CFQ works well for many general use cases but lacks latency guarantees. Spoiler alert: This post focuses on MongoDB 3.6.X 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

NOVEMBER 29, 2017

As there are few individuals with this expertise, an easier process presents a significant opportunity for companies who want to accelerate their ML usage. After the dataset is created, you must scale the processing to handle the data, which can often be a blocker. However, many developers find them difficult to build and deploy.

Tuning 94

Tuning AWS Scalability Infrastructure 94

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. What about short-lived processes, like a service restarting in a loop? I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer.

Speed 126

Speed Java AWS Virtualization 126

Abhishek Tiwari

JULY 12, 2018

These data pipelines can process data at petabytes scale and to some extent, their success can be attributed to an army of engineers devoted to build and maintain internal data pipelines. Not everyone is operating at Netflix or Spotify scale data engineering function. Depending on frameworks, data processing units (a.k.a

Latency 63

Latency Analytics Scalability Engineering 63

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Some of DBLog’s features are: Processes captured log events in-order.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Some of DBLog’s features are: Processes captured log events in-order.

Database 212

Database Traffic Transportation Open Source 212

The Morning Paper

MAY 21, 2019

This boils down to a single digit µs latency toleration in the tail for far memory, and in addition to security and privacy concerns, rules out remote memory solutions. Thus we’re fundamentally trading (de)-compression latency at access time for the ability to pack more data in memory. ML-based auto-tuning.

Software 78

Software Software Google Hardware 78

Brendan Gregg

OCTOBER 8, 2018

eBPF (extended Berkeley Packet Filter) is in the Linux kernel and is the new hotness in systems engineering. Screenshot: tracing read latency for PID 181: # bpftrace -e 'kprobe:vfs_read /pid == 30153/ { @start[tid] = nsecs; } kretprobe:vfs_read /@start[tid]/ { @ns = hist(nsecs - @start[tid]); delete(@start[tid]); }'. ^C

C++ 110

C++ Virtualization Programming Latency 110

O'Reilly Software

APRIL 4, 2018

We live in the era of the connected experience, where our daily interactions with the world can be digitized, collected, processed, and analyzed to generate valuable insights. Accumulating all this data to process overnight is not an option anymore. Do we need to process each record individually? Process tolerance.

Architecture 80

Architecture Scalability Google Operating System 80

IO River

NOVEMBER 15, 2023

It's an ongoing, iterative process. Two of them are particularly gnarly: fine-tuning rules to perfection and managing a WAF over a multi-CDN architecture. Similarly, a rule that works perfectly on one CDN may trigger false positives or negatives on another due to slight variations in how each CDN processes web traffic.

Traffic 52

Traffic Network Logistics Architecture 52

All Things Distributed

OCTOBER 2, 2017

Performant – DynamoDB consistently delivers single-digit millisecond latencies even as your traffic volume increases. DynamoDB automatically re-distributes your data to healthy servers to ensure there are always multiple replicas of your data without you needing to intervene.

Internet 128

Internet Internet AWS Performance 128