The Netflix TechBlog

NOVEMBER 22, 2019

December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Morning Paper

AUGUST 29, 2019

We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. For cost calculations, the costs are a combination of compute costs, storage costs, data scan costs, and software license costs. Key findings. Query restrictions. Serverless o?erings

Architecture 76

Architecture Cloud Storage Serverless 76

Percona

SEPTEMBER 1, 2023

This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. This reduction in latency ensures that applications and websites provide a more rapid and responsive user experience. This can significantly elevate user satisfaction and engagement.

Tuning 52

Tuning Database Performance Hardware 52

ScaleOut Software

JULY 21, 2020

Designed to help scalable applications deliver high performance, it stores live, fast-changing data in memory (DRAM) for fast updates and retrieval. Instead of querying objects from the IMDG and analyzing them in the client, applications can now simply run this analysis within the IMDG itself using APIs available in ScaleOut StateServer Pro.

Analytics 52

Analytics Airlines Cache Scalability 52

ScaleOut Software

JULY 21, 2020

Designed to help scalable applications deliver high performance, it stores live, fast-changing data in memory (DRAM) for fast updates and retrieval. Instead of querying objects from the IMDG and analyzing them in the client, applications can now simply run this analysis within the IMDG itself using APIs available in ScaleOut StateServer Pro.

Analytics 52

Analytics Airlines Cache Scalability 52

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. can enhance Redis by handling management tasks, backups, and scalability, facilitating global reach and easy cloud integration for global businesses.

Cache 130

Cache Storage Scalability Architecture 130

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. Driving Storage Costs Down for AWS Customers. All Things Distributed. Comments ().

AWS 112

AWS Cloud Games Latency 112

Scalegrid

DECEMBER 21, 2023

Identifying key Redis® metrics such as latency, CPU usage, and memory metrics is crucial for effective Redis monitoring. To monitor Redis® instances effectively, collect Redis metrics focusing on cache hit ratio, memory allocated, and latency threshold.

Strategy 130

Strategy Monitoring Latency DevOps 130

All Things Distributed

SEPTEMBER 25, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Amazon RDS Provisioned IOPS is ideal for mission-critical online transaction processing (OLTP) workloads that require a high performance storage option with consistent IOPS, within a narrow band of tolerance. Provisioned IOPS storage in RDS. Comments ().

Cloud 117

Cloud AWS Storage Database 117

Dotcom-Montior

OCTOBER 6, 2021

Site reliability engineers are focused with creating scalable and reliable software systems, so this also includes ensuring that development work is efficient and reliable, so when the finished product is ready for production, there are no surprises. Availability. Performance. Monitoring. Incident Response. On-call Support.

Engineering 67

Engineering DevOps Monitoring Google 67

Abhishek Tiwari

DECEMBER 27, 2017

faster access to external storage and data locality (I/O, bandwidth). Under the hood, the BlueData used several enhancements to boosts the I/O performance and scalability of container-based clusters. Kubernetes has all necessary primitives available to make it happen. Storage provisioning. Native frameworks.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Abhishek Tiwari

JULY 12, 2018

Data Pipeline A data pipeline is a software that ingests data from multiple sources, transforms it and finally makes it available to internal or external products. Pipeline is infinitely scalable Given the elasticity offered by public clouds, one would think that data pipelines are infinitely scalable which is entirely misleading.

Latency 63

Latency Analytics Scalability Engineering 63

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). AWS is committed to making its services available at low cost. Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. With this change, we will improve the granularity of pricing information you receive by introducing a Spot Instance price per Availability Zone rather than a Spot Instance price per Region. Driving Storage Costs Down for AWS Customers. Comments ().

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Route 53 provides Authoritative DNS functionality implemented using a world-wide network of highly-available DNS servers.

Cloud 117

Cloud Internet Internet AWS 117

HammerDB

DECEMBER 5, 2020

In the days before highly performant SSDs and persistent memory, database benchmarks had a significant challenge in comparing performance due to the available I/O performance. For TPC-C this meant enough available spindles to reduce I/O latency and for TPC-H enough bandwidth for data throughput.

C++ 40

C++ Benchmarking Virtualization Open Source 40

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Today, I am very proud to be a part of the Amazon Web Services team as we truly make HPC available as an on-demand commodity for every developer to use. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

Cloud 118

Cloud AWS Automotive Latency 118

O'Reilly Software

JUNE 12, 2018

How do you ensure this is done in a way that is sympathetic to your application’s latency and load conditions? I have to say that, when I first looked at it, Kafka seemed pretty similar to the tooling I’d been using for decades, but with a few fancy scalability properties thrown in. How do you do that effectively?

Database 72

Database Cache Architecture Latency 72

The Netflix TechBlog

NOVEMBER 22, 2019

December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Scalegrid

MARCH 14, 2024

This approach allows companies to combine the security and control of private clouds with public clouds’ scalability and innovation potential. Mastering Hybrid Cloud Strategy Are you looking to leverage the best private and public cloud worlds to propel your business forward? A hybrid cloud strategy could be your answer.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. There are four main reasons to do so: Performance - For many applications and services, data access latency to end users is important. The new Singapore Region offers customers in APAC lower-latency access to AWS services. All Things Distributed.

AWS 61

AWS Cloud Latency Storage 61

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. Driving Storage Costs Down for AWS Customers. All Things Distributed.

AWS 111

AWS Government Big Data Cloud 111

Scalegrid

JANUARY 12, 2024

All rely heavily on utilizing allocated portions from existing pools made available through specific providers as part of their service offerings. There are numerous choices available for deploying these workloads on various cloud provider platforms that offer unique capabilities.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. Typically a good value is 70%-80% of available memory.

Performance 127

Performance Hardware Tuning Storage 127

The Netflix TechBlog

NOVEMBER 3, 2022

When a new leader is elected it loads all data from external storage. In PACELC terms we choose PC/EC and have the same level of availability for writes of our previous system while improving our theoretical availability for reads. cell): Titus Job Coordinator is a leader elected process managing the active state of the system.

Cache 224

Cache Latency Traffic Systems 224

The Netflix TechBlog

OCTOBER 27, 2020

The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. As most key-value storage engines support efficiently deleting a namespace (e.g.

Latency 243

Latency Storage Big Data Tuning 243

All Things Distributed

JANUARY 18, 2012

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. The original Dynamo design was based on a core set of strong distributed systems principles resulting in an ultra-scalable and highly reliable database system.

Scalability 167

Scalability Database Ecommerce Latency 167

Percona

JULY 17, 2023

DBAs and developers appreciate its combination of flexibility, scalability, and performance. MongoDB replica sets enable data redundancy and automatic failover, setting the stage (there’s that term again) for high availability. For starters, MongoDB Community lacks the advanced security features available in MongoDB Enterprise.

Open Source 77

Open Source Database Scalability Software 77

The Netflix TechBlog

APRIL 26, 2022

These facts are managed and made available by services like viewing history or video metadata services outside of Axion. The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. Time is a critical component of Axion?—?When

Storage 187

Storage Design Scalability Latency 187

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS 136

AWS Metrics IoT Storage 136

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3). Amazon Redshift.

AWS 133

AWS Metrics IoT Storage 133

The Netflix TechBlog

NOVEMBER 2, 2021

As VMAF evolves and is integrated with more encoding and streaming workflows within Netflix, we need scalable ways of fostering video quality innovations. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. The workflow is initiated. PSNR and SSIM) is complete.

Media 171

Media Innovation Metrics Latency 171

Percona

JULY 1, 2023

These may be performance, high availability, operational cost, management, capacity planning, scalability, security, monitoring, etc. It efficiently manages read and write operations, optimizes data access, and minimizes contention, resulting in high throughput and low latency to ensure that applications perform at their best.

AWS 52

AWS Database Serverless Storage 52

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. All these capabilities are available to customers at no additional charge, and maintain open-source Redis compatibility. Building upon Redis.

Scalability 130

Scalability Analytics Cache AWS 130

The Morning Paper

MARCH 17, 2020

An empirical guide to the behavior and use of scalable persistent memory , Yang et al., One thing they all had in common is an evaluation using some kind of simulation of the expected behaviour of NVDIMMs, because the real thing wasn’t yet available. The Optane DIMM is the first scalable, commercially available NVDIMM.

Scalability 59

Scalability Latency Cache Media 59