Latency, Scalability, Storage and Tuning - Technology Performance Pulse

Migrating Critical Traffic At Scale with No Downtime?—?Part 1

The Netflix TechBlog

MAY 4, 2023

Migrating Critical Traffic At Scale with No Downtime — Part 1 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. Logging is selective to cases where the old and new responses do not match.

Traffic

Traffic Latency Tuning Systems

MySQL Performance Tuning 101: Key Tips to Improve MySQL Database Performance

Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? A finely tuned database processes queries more efficiently, leading to swifter results.

Tuning

Tuning Database Performance Hardware

Building Netflix’s Distributed Tracing Infrastructure

The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage.

Infrastructure

Infrastructure Transportation Storage Open Source

Optimizing data warehouse storage

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage

Storage Latency Efficiency Data Engineering

Friends don't let friends build data pipelines

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. If tuned for performance, there is a good change reliability is compromised - and vice versa. In a nutshell, a data pipeline is a distributed system.

Latency

Latency Analytics Scalability Engineering

InnoDB Performance Optimization Basics

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. If you see concurrency issues, you can tune this variable.

Performance

Performance Hardware Tuning Storage

Bulldozer: Batch Data Moving from Data Warehouse to Online Key-Value Stores

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. As most key-value storage engines support efficiently deleting a namespace (e.g.

Latency

Latency Storage Big Data Tuning

Achieving observability in async workflows

The Netflix TechBlog

MAY 14, 2021

We are expected to process 1,000 watermarks for a single distribution in a minute, with non-linear latency growth as the number of watermarks increases. The watermarking functionality, at the start, was a simple offering with various Google Drive integrations for storage and links.

Traffic

Traffic Latency Java Google

Get up to 300 new metrics out of the box with AWS supporting services (GA)

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS

AWS Metrics IoT Storage

Get up to 300 new metrics out of the box with AWS supporting services (GA)

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS

AWS Metrics IoT Storage

Netflix Video Quality at Scale with Cosmos Microservices

The Netflix TechBlog

NOVEMBER 2, 2021

As VMAF evolves and is integrated with more encoding and streaming workflows within Netflix, we need scalable ways of fostering video quality innovations. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. VQS is called using the measureQuality endpoint.

Media

Media Innovation Metrics Latency

An empirical guide to the behavior and use of scalable persistent memory

The Morning Paper

MARCH 17, 2020

An empirical guide to the behavior and use of scalable persistent memory , Yang et al., higher latency and lower bandwidth)… We have found the actual behavior of Optane DIMMs to be more complicated and nuanced than the "slower, persistent DRAM" label would suggest. The read latency for Optane is 2x-3x higher than DRAM.

Scalability

Scalability Latency Cache Media

HammerDB MySQL and MariaDB Best Practice for Performance and Scalability

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . maximum transition latency: Cannot determine or is not supported. . innodb_file_per_table.

Best Practices

Best Practices Scalability Performance C++

Expanding the Cloud: Amazon Machine Learning Service, the Amazon Elastic Filesystem and more

All Things Distributed

APRIL 9, 2015

The Amazon ML console and API provide data and model visualization tools, as well as wizards to guide you through the process of creating machine learning models, measuring their quality and fine-tuning the predictions to match your application requirements. Details on the AWS Blog. The Amazon Elastic File System. for a while already.

Lambda

Lambda Cloud IoT AWS

Accelerate Machine Learning with Amazon SageMaker

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. An elastic, secure, and scalable environment to host your models, with one-click deployment.

Tuning

Tuning AWS Scalability Infrastructure

A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications

All Things Distributed

OCTOBER 2, 2017

We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded. We had an advanced team of database administrators and access to top experts within Oracle. million requests per second.

Internet

Internet Internet AWS Performance