The Netflix TechBlog

MAY 4, 2023

Migrating Critical Traffic At Scale with No Downtime — Part 1 Shyam Gala , Javier Fernandez-Ivern , Anup Rokkam Pratap , Devang Shah Hundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience.

Traffic 339

Traffic Latency Tuning Systems 339

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. Typically a good value is 70%-80% of available memory.

Performance 127

Performance Hardware Tuning Storage 127

Dynatrace

JUNE 27, 2023

Logs complement out-of-the-box metrics and enable automated actions for responding to availability, security, and other service events. Many AWS services and third party solutions use AWS S3 for log storage. A Dynatrace S3 log forwarder has been available for some time to early adopters, with community support only.

AWS 195

AWS Cloud Lambda Analytics 195

Dynatrace

DECEMBER 18, 2023

For busy site reliability engineers, ensuring system reliability, scalability, and overall health is an imperative that’s getting harder to achieve in ever-expanding, cloud-native, container-based environments. But often, we use additional services and solutions within our environment for backups, storage, networking, and more.

Metrics 175

Metrics Engineering Energy Tuning 175

The Netflix TechBlog

MARCH 14, 2023

We must quickly surface the most stand-out highlights from the titles available on our service in the form of images and videos in the member experience. In addition, we were able to perform a handful of A/B tests to validate or negate our hypotheses for tuning the search experience. Artists and video editors must create them.

Media 291

Media Innovation Energy Architecture 291

The Netflix TechBlog

OCTOBER 27, 2020

As the paved path for moving data to key-value stores, Bulldozer provides a scalable and efficient no-code solution. The KV DAL allows applications to use a well-defined and storage engine agnostic HTTP/gRPC key-value data interface that in turn decouples applications from hard to maintain and backwards-incompatible datastore APIs.

Latency 243

Latency Storage Big Data Tuning 243

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS 135

AWS Metrics IoT Storage 135

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS 132

AWS Metrics IoT Storage 132

Percona

OCTOBER 18, 2023

Flexibility and scalability Open source databases provide much greater flexibility regarding customization and configuration. Are you looking to enhance performance, improve scalability, cut expenses, or gain access to specific features you don’t currently have? Start by identifying the reasons driving the migration.

Open Source 94

Open Source Database Hardware Strategy 94

The Netflix TechBlog

NOVEMBER 2, 2021

As VMAF evolves and is integrated with more encoding and streaming workflows within Netflix, we need scalable ways of fostering video quality innovations. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. The workflow is initiated. PSNR and SSIM) is complete.

Media 171

Media Innovation Metrics Latency 171

Percona

AUGUST 7, 2023

InnoDB Undo Log The undo log, also known as the rollback segment, is a crucial part of the InnoDB storage engine. Properly managing flushing and purging operations is crucial for system performance and tuning. Try Percona Distribution for MySQL today!

Database 96

Database Tuning Storage Open Source 96

The Morning Paper

MARCH 17, 2020

An empirical guide to the behavior and use of scalable persistent memory , Yang et al., One thing they all had in common is an evaluation using some kind of simulation of the expected behaviour of NVDIMMs, because the real thing wasn’t yet available. The Optane DIMM is the first scalable, commercially available NVDIMM.

Scalability 59

Scalability Latency Cache Media 59

Percona

AUGUST 17, 2023

PostgreSQL has powerful and advanced features, including asynchronous replication, full-text searches of the database, and native support for JSON-style storage, key-value storage, and XML. How-to documentation is readily available. It has evolved steadily, however, during 25 years as an open source project.

Open Source 80

Open Source Healthcare Social Media Database 80

Percona

DECEMBER 27, 2023

All of the database automation for running a highly available, resilient, and secure database is built into the operator to simplify the operation and management of your clusters. Well, here we are – with storage autoscaling for databases in Kubernetes, slated for release in Q1, 2024 after a year of hard work.

Cloud 82

Cloud Open Source Strategy Best Practices 82

SQL Server According to Bob

SEPTEMBER 9, 2019

Acquiring shared access requires only the local partition be acquired (lightweight scalability). I recall when we were tuning the sp_reset_connection (which releases the database lock and acquires it again) command we tested rates in excess of 250,000/sec to ensure the partitioned database lock scaled: [link].

Servers 103

Servers Database Scalability Tuning 103

Dynatrace

JULY 8, 2020

One of the promises of container orchestration platforms is to make i t easier for the developers to accelerate the deployment of their app lication s without having to worry about scalability and infrastructure dependencies. By default, containers run with unbounded limits. Node and w orkload health .

Infrastructure 196

Infrastructure Monitoring Cloud Metrics 196

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . GHz available cpufreq governors: performance powersave . HammerDB difference from Sysbench.

Best Practices 40

Best Practices Scalability Performance C++ 40

Adrian Cockcroft

JUNE 21, 2022

I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems. At one point we had two conflicting high availability products from different teams, who weren’t cooperating.

Tuning 52

Tuning Benchmarking Engineering C++ 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Percona

AUGUST 5, 2023

Otherwise, the storage engine does a scatter-gather and queries ALL partitions in a UNION that is not concurrent. This method distributes data evenly across partitions to achieve balanced storage and optimal query performance. This helps identify potential issues and fine-tune the partitioning strategy.

Strategy 40

Strategy Storage Database Best Practices 40

Official Blog - World Web Technology

MAY 12, 2023

WordPress has always been the first choice making developers to build highly scalable, robust, and secure web applications. With the help of Headless WordPress, it is possible for developers to combine WordPress and ReactJS to build highly scalable, feature-rich, and dynamic website that serve your business purposes. Stay tuned.

Website 52

Website Development Scalability Architecture 52

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52

Abhishek Tiwari

JULY 12, 2018

Data Pipeline A data pipeline is a software that ingests data from multiple sources, transforms it and finally makes it available to internal or external products. If tuned for performance, there is a good change reliability is compromised - and vice versa. In a nutshell, a data pipeline is a distributed system.

Latency 63

Latency Analytics Scalability Engineering 63

Smashing Magazine

SEPTEMBER 27, 2019

However, no matter the size and complexity of your real application, the approach is well-scalable and is supposed to be suitable for projects of any size. All the code mentioned here is publicly available, and you can get to it whenever you want. There are two repositories available for you to play with: Frankenstein TodoMVC.

Storage 66

Storage Testing Processing Code 66

All Things Distributed

OCTOBER 8, 2014

The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month. We designed DynamoDB to operate with at least 99.999% availability. In 2012, we launched Amazon DynamoDB, the successor to Amazon Dynamo.

Availability 145

Availability Performance Scalability AWS 145

All Things Distributed

MARCH 7, 2013

Werner Vogels weblog on building scalable and robust distributed systems. As I sat down with the DynamoDB team to review our progress over the last year, I realized that DynamoDB had surpassed even my own expectations for how easily applications could achieve massive scale and high availability with DynamoDB. All Things Distributed.

Ecommerce 139

Ecommerce Storage Scalability Hardware 139

All Things Distributed

APRIL 9, 2015

Amazon ML uses powerful algorithms that can help you create machine learning models by finding patterns in existing data, and using these patterns to make predictions from new data as it becomes available. Amazon ML is highly scalable and can generate billions of predictions, and serve those predictions in real-time and at high throughput.

Lambda 122

Lambda Cloud IoT AWS 122

Percona

JANUARY 14, 2019

However, ClickHouse is super efficient for timeseries and provides “sharding” out of the box (scalability beyond one node). With the latest ClickHouse version, all of these features are available, but some of them may not perform fast enough. So can we use it as our main datastore? Deleting messages. Text search.

Database 40

Database Analytics Blockchain Healthcare 40

All Things Distributed

NOVEMBER 28, 2012

Werner Vogels weblog on building scalable and robust distributed systems. They contain large amounts of locally attached storage on multiple spindles and are connected by a minimally oversubscribed 10 Gigabit Ethernet network. Until now, these levels of performance and scalability were prohibitively expensive. Comments ().

Cloud 164

Cloud Storage Architecture Scalability 164

SQL Server According to Bob

JANUARY 11, 2020

​​ The storage ​​ space that is required ​​ for the sparse file is only that of the actual bytes written to the file and not the ​​ maximum ​​ file ​​ size.

Servers 40

Servers Cache Database Media 40

All Things Distributed

OCTOBER 2, 2017

We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded. We had an advanced team of database administrators and access to top experts within Oracle. million requests per second.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. Today, we are announcing the general availability of Amazon SageMaker.

Tuning 94

Tuning AWS Scalability Infrastructure 94

All Things Distributed

MARCH 28, 2011

Werner Vogels weblog on building scalable and robust distributed systems. What used to be only available in physical formats now often has digital equivalents and this digitalization is driving great new innovations. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Comments ().

AWS 138

AWS Cloud Storage Internet 138