The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

All Things Distributed

NOVEMBER 13, 2014

Capital-intensive storage solutions became as simple as PUTting and GETting objects in Amazon S3. At AWS we innovate by listening to and learning from our customers, and one of the things we hear from them is that they want it to be even simpler to run code in the cloud and to connect services together easily.

Lambda 136

Lambda AWS Cloud Scalability 136

Scalegrid

MARCH 14, 2024

Mastering Hybrid Cloud Strategy Are you looking to leverage the best private and public cloud worlds to propel your business forward? A hybrid cloud strategy could be your answer. This approach allows companies to combine the security and control of private clouds with public clouds’ scalability and innovation potential.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. Expanding the Cloud â??

AWS 112

AWS Cloud Games Latency 112

All Things Distributed

SEPTEMBER 25, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud â?? Amazon RDS Provisioned IOPS is ideal for mission-critical online transaction processing (OLTP) workloads that require a high performance storage option with consistent IOPS, within a narrow band of tolerance. All Things Distributed.

Cloud 117

Cloud AWS Storage Database 117

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Cluster Compute Instances for Amazon EC2. Today, Amazon Web Services took very an important step in unlocking the advantages of cloud computing for a very important application area. Driving Storage Costs Down for AWS Customers.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud with DNS - Introducing Amazon Route 53. Often these namespaces are hierarchical in nature such that it becomes easier to manage them and to decentralize control, which makes the system more scalable. Expanding the Cloud â??

Cloud 117

Cloud Internet Internet AWS 117

ScaleOut Software

JULY 19, 2021

Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. Let’s take a closer look at today’s conventional streaming analytics architectures, which can be hosted in the cloud or on-premises.

IoT 78

IoT Analytics Big Data Architecture 78

Scalegrid

JANUARY 12, 2024

What is workload in cloud computing? Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. The environments, which were previously isolated, are now working seamlessly under central control.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. can enhance Redis by handling management tasks, backups, and scalability, facilitating global reach and easy cloud integration for global businesses. 3d render.

Cache 130

Cache Storage Scalability Architecture 130

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - New AWS Region: US-West (Northern California). This new Region consists of multiple Availability Zones and provides low-latency access to the AWS services from for example the Bay Area. Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud â?? This new Region has been highly requested by companies worldwide, and it provides low-latency access to AWS services for those who target customers in South America. Driving Storage Costs Down for AWS Customers. Comments ().

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Quite often "The Cloud" is portrayed as something magically transparent that lives somewhere in the internet. Driving Storage Costs Down for AWS Customers. Comments ().

AWS 61

AWS Cloud Latency Storage 61

Dynatrace

JANUARY 17, 2024

Growing AI adoption brings rising cloud costs There are three key reasons that AI costs can spiral out of control: AI consumes additional resources. Running artificial intelligence models and querying data requires massive amounts of computational resources in the cloud, which results in higher cloud costs. Use containerization.

Strategy 224

Strategy Artificial Intelligence Storage Cloud 224

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. As mentioned before, most of data pipelines are not stateless which means scalability is not given a thing.

Latency 63

Latency Analytics Scalability Engineering 63

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS GovCloud (US) Region. There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. Cloud First.

AWS 111

AWS Government Big Data Cloud 111

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 202 A day in the life of a Netflix Engineer Dave Hahn , SRE Engineering Manager Abstract : Netflix is a large, ever-changing ecosystem serving millions of customers across the globe through cloud-based systems and a globally distributed CDN. We explore all the systems necessary to make and stream content from Netflix.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. As a part of that process, we also realized that there were a number of latency sensitive or location specific use cases like Hadoop, HPC, and testing that would be ideal for Spot. Driving Storage Costs Down for AWS Customers. Expanding the Cloud â??

AWS 85

AWS Storage Cloud Big Data 85

Abhishek Tiwari

DECEMBER 27, 2017

portability between cloud and on-premises. faster access to external storage and data locality (I/O, bandwidth). Under the hood, the BlueData used several enhancements to boosts the I/O performance and scalability of container-based clusters. Storage provisioning. But Kubernetes storage is evolving quite quickly.

Big Data 63

Big Data Storage Benchmarking Hardware 63

O'Reilly Software

JUNE 12, 2018

The advent of the cloud was a huge boon to the work we were doing, as the improved automation made us even more productive. How do you ensure this is done in a way that is sympathetic to your application’s latency and load conditions? But I kept hitting up against some fundamental issues. How do you do that effectively?

Database 72

Database Cache Architecture Latency 72

All Things Distributed

NOVEMBER 19, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Here are some the links I shared this week on twitter and facebook : Cloud Computing. Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. Comments ().

AWS 67

AWS Cloud Benchmarking Storage 67

HammerDB

DECEMBER 5, 2020

For TPC-C this meant enough available spindles to reduce I/O latency and for TPC-H enough bandwidth for data throughput. Even with superior I/O performance today the I/O configuration required on-premise or in the cloud remains costly. This was both expensive and time consuming to configure.

C++ 40

C++ Benchmarking Virtualization Open Source 40

High Scalability

DECEMBER 21, 2018

It's HighScalability time: Have a very scalable Xmas everyone! What could be better than a book on the cloud? Explain the Cloud Like I'm 10. swardley: X : What's going to happen in cloud in 2019? 2) Cloud will decentralise in terms of provision not power i.e. Me : Nothing special. See you in the New Year.

Internet 137

Internet Internet Scalability Serverless 137

Percona

MARCH 23, 2023

Storage The type of storage and disk used for database servers can have a significant impact on performance and reliability. Cloud Different cloud providers offer a range of instance types and sizes, each with varying amounts of CPU, memory, and storage. Setting oom_score_adj to -800. References How MySQL 8.0.21

Performance 128

Performance Hardware Tuning Storage 128

High Scalability

JULY 20, 2018

And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. It would mean a great deal to me. They'll love you even more. There are a few more quotes. Not so many this week.

Internet 121

Internet Internet Scalability Automotive 121

All Things Distributed

JANUARY 18, 2012

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. DynamoDB is the result of 15 years of learning in the areas of large scale non-relational databases and cloud services. All Things Distributed. Amazon DynamoDB â??

Scalability 167

Scalability Database Ecommerce Latency 167

The Netflix TechBlog

MAY 5, 2021

A file and folder interface for Netflix Cloud Services Written by Vikram Krishnamurthy , Kishore Kasi , Abhishek Kapatkar , and Tejas Chopra In this post, we are introducing Netflix Drive, a Cloud drive for media assets and providing a high level overview of some of its features and interfaces. The major pieces, as shown in Fig.

Media 174

Media Storage Architecture Cloud 174

The Netflix TechBlog

NOVEMBER 3, 2022

by Tomasz Bak and Fabio Kung Introduction Titus is the Netflix cloud container runtime that runs and manages containers at scale. When a new leader is elected it loads all data from external storage. We started seeing increased response latencies and leader servers running at dangerously high utilization.

Cache 224

Cache Latency Traffic Systems 224

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3).

AWS 136

AWS Metrics IoT Storage 136

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Amazon Simple Storage Service (S3).

AWS 133

AWS Metrics IoT Storage 133

Percona

JULY 17, 2023

DBAs and developers appreciate its combination of flexibility, scalability, and performance. Those problems (some with shared elements) include: High cost and complicated pricing structure : Many companies say MongoDB has an expensive and complicated pricing structure ( Cloud Zero, January 2023 ).

Open Source 78

Open Source Database Scalability Software 78

All Things Distributed

APRIL 9, 2015

Amazon ML is highly scalable and can generate billions of predictions, and serve those predictions in real-time and at high throughput. AWS has been offering a range of storage solutions: objects, block storage, databases, archiving, etc. Details on the AWS Blog. The Amazon Elastic File System. for a while already.

Lambda 122

Lambda Cloud IoT AWS 122

Percona

JULY 1, 2023

Understanding DBaaS DBaaS cloud services allow users to use databases without configuring physical hardware and infrastructure or installing software. These may be performance, high availability, operational cost, management, capacity planning, scalability, security, monitoring, etc. How do I choose which one to use?

AWS 52

AWS Database Serverless Storage 52

ScaleOut Software

JULY 14, 2021

The Microsoft Azure IoT ecosystem offers a rich set of capabilities for processing IoT telemetry, from its arrival in the cloud through its storage in databases and data lakes. It provides support for a range of message protocols, buffering, and scalable message distribution to downstream services.

IoT 52

IoT Azure Analytics Serverless 52

Dynatrace

DECEMBER 22, 2021

Metrics are measures of critical system values, such as CPU utilization or average write latency to persistent storage. Observability platforms are becoming essential as the complexity of cloud-native architectures increases. A database could start executing a storage management process that consumes database server resources.

Artificial Intelligence 180

Artificial Intelligence Metrics Architecture DevOps 180

The Netflix TechBlog

APRIL 26, 2022

The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. Since we were optimizing at the logging level for storage and performance, we had less data and metadata to play with to optimize the query performance.

Storage 187

Storage Design Scalability Latency 187

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. This allows for faster failover times while minimizing latency. Amazon’s enhancements address many day-to-day challenges with running Redis.

Scalability 130

Scalability Analytics Cache AWS 130