The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . This is to be expected and is due to the limitations of the scalability of the storage engine.

Best Practices 40

Best Practices Scalability Performance C++ 40

Adrian Cockcroft

JUNE 21, 2022

I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems. Rich became co-author of the second edition of the Sun Performance Tuning book, to describe how it worked.

Tuning 52

Tuning Benchmarking Engineering C++ 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. If tuned for performance, there is a good change reliability is compromised - and vice versa. In a nutshell, a data pipeline is a distributed system.

Latency 63

Latency Analytics Scalability Engineering 63

Brendan Gregg

APRIL 30, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users. In fact, we’d link to the first LISA conference website for reference, but this conference not only predates the Wayback Machine – it also predates the World Wide Web!

DevOps 43

DevOps Network Best Practices Programming 43

Percona

AUGUST 30, 2018

Because recognizing if the workload is read intensive or write intensive will impact your hardware choices, database configuration as well as what techniques you can apply for performance optimization and scalability. This question looks trivial on the surface, but as you go deeper—complexity emerges.

Benchmarking 47

Benchmarking Database Operating System Architecture 47

Brendan Gregg

APRIL 29, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users. In fact, we’d link to the first LISA conference website for reference, but this conference not only predates the Wayback Machine – it also predates the World Wide Web!

DevOps 40

DevOps Network Best Practices Programming 40

The Netflix TechBlog

MARCH 14, 2023

Specifically, we will dive into the architecture that powers search capabilities for studio applications at Netflix. In addition, we were able to perform a handful of A/B tests to validate or negate our hypotheses for tuning the search experience. This service leverages Cassandra and Elasticsearch for data storage and retrieval.

Media 291

Media Innovation Energy Architecture 291

The Netflix TechBlog

OCTOBER 27, 2020

At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. As the paved path for moving data to key-value stores, Bulldozer provides a scalable and efficient no-code solution. As most key-value storage engines support efficiently deleting a namespace (e.g.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

MAY 14, 2021

Managing and operating asynchronous workflows can be difficult without the proper tools and architecture that puts observability, debugging, and tracing at the forefront. The watermarking functionality, at the start, was a simple offering with various Google Drive integrations for storage and links.

Traffic 160

Traffic Latency Java Google 160

The Netflix TechBlog

NOVEMBER 2, 2021

As VMAF evolves and is integrated with more encoding and streaming workflows within Netflix, we need scalable ways of fostering video quality innovations. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. via bug fixes). We call this system Cosmos.

Media 171

Media Innovation Metrics Latency 171

All Things Distributed

MARCH 7, 2013

Werner Vogels weblog on building scalable and robust distributed systems. s fast and easy scalability can be quickly applied to building high scale applications. Shazam needed to handle an enormous increase in traffic for the duration of the Super Bowl and used DynamoDB as part of their architecture. All Things Distributed.

Ecommerce 139

Ecommerce Storage Scalability Hardware 139

All Things Distributed

NOVEMBER 28, 2012

Werner Vogels weblog on building scalable and robust distributed systems. s architecture and underlying platform are also optimized to deliver high performance for data warehousing workloads. s MPP architecture makes it easy to resize your cluster to keep pace with your storage and performance requirements. Comments ().

Cloud 164

Cloud Storage Architecture Scalability 164

Smashing Magazine

SEPTEMBER 27, 2019

However, no matter the size and complexity of your real application, the approach is well-scalable and is supposed to be suitable for projects of any size. To make things simple, we just copied over Host’s storage file into Alien and hooked up our components to it : import todoStorage from "./storage"; In this case, it’s localStorage.

Storage 66

Storage Testing Processing Code 66

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. An elastic, secure, and scalable environment to host your models, with one-click deployment.

Tuning 94

Tuning AWS Scalability Infrastructure 94

All Things Distributed

MARCH 28, 2011

Werner Vogels weblog on building scalable and robust distributed systems. We see that with our Amazon customers; when they hear a great tune on a radio they may identify it using the Shazam or Soundhound apps on their mobile phone and buy that song instantly from the Amazon MP3 store. Driving Storage Costs Down for AWS Customers.

AWS 138

AWS Cloud Storage Internet 138

All Things Distributed

OCTOBER 8, 2014

The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month. More than a decade ago, Amazon embarked on a mission to build a distributed system that challenged conventional methods of data storage and querying.

Availability 145

Availability Performance Scalability AWS 145

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52