The Netflix TechBlog

JUNE 7, 2021

Network Availability: The expected continued growth of our ecosystem makes it difficult to understand our network bottlenecks and potential limits we may be reaching. Cloud Network Insight is a suite of solutions that provides both operational and analytical insight into the cloud network infrastructure to address the identified problems.

Network 325

Network Transportation AWS Cloud 325

ScaleOut Software

JUNE 15, 2020

When analyzing telemetry from a large population of data sources, such as a fleet of rental cars or IoT devices in “smart cities” deployments, it’s difficult if not impossible for conventional streaming analytics platforms to track the behavior of each individual data source and derive actionable information in real time.

Analytics 52

Analytics IoT Lambda Big Data 52

ScaleOut Software

JUNE 15, 2020

When analyzing telemetry from a large population of data sources, such as a fleet of rental cars or IoT devices in “smart cities” deployments, it’s difficult if not impossible for conventional streaming analytics platforms to track the behavior of each individual data source and derive actionable information in real time.

Analytics 52

Analytics IoT Lambda Big Data 52

Dynatrace

MARCH 24, 2023

This orchestration includes provisioning, scheduling, networking, ensuring availability, and monitoring container lifecycles. Part of its popularity owes to its availability as a managed service through the major cloud providers, such as Amazon Elastic Kubernetes Service , Google Kubernetes Engine , and Microsoft Azure Kubernetes Service.

Infrastructure 182

Infrastructure Open Source Operating System Cloud 182

All Things Distributed

DECEMBER 13, 2016

Today, I'm happy to announce that the AWS Europe (London) Region, our 16th technology infrastructure region globally, is now generally available for use by customers worldwide. GoSquared provides various analytics services that web and mobile companies can use to understand their customers' behaviors.

AWS 166

AWS Cloud Artificial Intelligence IoT 166

ScaleOut Software

NOVEMBER 24, 2020

Real-Time Digital Twins Can Add Important New Capabilities to Telematics Systems and Eliminate Scalability Bottlenecks. At the same time, telemetry snapshots are stored in a data lake, such as HDFS , for offline batch analysis and visualization using big data tools like Spark.

Analytics 52

Analytics Architecture Scalability Software Architecture 52

All Things Distributed

OCTOBER 7, 2015

Today, I am excited to share with you a brand new service called Amazon QuickSight that aims to simplify the process of deriving insights from a wide variety of data sources in a fast and affordable manner. QuickSight is a fast, cloud native, scalable, business intelligence service for the 1/10th the cost of old-guard BI solutions.

Cloud 137

Cloud Big Data AWS Analytics 137

ScaleOut Software

APRIL 3, 2020

An innovative new software approach called “real-time digital twins” running on a cloud-hosted, highly scalable, in-memory computing platform can help address this challenge. With up-to-date information for all ventilators immediately at hand, analysts can ask questions like: “Where are all the available ventilators at this moment?”.

Logistics 52

Logistics Analytics Scalability Cloud 52

ScaleOut Software

APRIL 3, 2020

An innovative new software approach called “real-time digital twins” running on a cloud-hosted, highly scalable, in-memory computing platform can help address this challenge. With up-to-date information for all ventilators immediately at hand, analysts can ask questions like: “Where are all the available ventilators at this moment?”.

Logistics 52

Logistics Analytics Scalability Cloud 52

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Caching has become a standard component in many applications to achieve a fast and predictable performance, but maintaining a collection of cache servers in a reliable and scalable manner is not a simple task. Driving down the cost of Big-Data analytics.

Cloud 117

Cloud Cache AWS Storage 117

The Morning Paper

SEPTEMBER 19, 2019

Hyper Dimension Shuffle describes how Microsoft improved the cost of data shuffling, one of the most costly operations, in their petabyte-scale internal big data analytics platform, SCOPE. What if the network was no longer the bottleneck? Maybe we should be switching to active-memory replication ?

Blockchain 46

Blockchain Hardware Google Analytics 46

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. The storage systems weve pioneered demonstrate extreme scalability while maintaining tight control over performance, availability, and cost. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Comments ().

AWS 112

AWS Cloud Games Latency 112

All Things Distributed

FEBRUARY 25, 2011

Werner Vogels weblog on building scalable and robust distributed systems. They had taken the approach that they would not only be offering their software as a scalable multi-tenant product but also as a single tenant environment for customers that want to have their own isolated environment. Driving down the cost of Big-Data analytics.

AWS 121

AWS Cloud Scalability Storage 121

All Things Distributed

FEBRUARY 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. been running at a traditional hosting site for many years until this preferred simple solution became available: today marks that day and I couldnt be happier about it. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

AWS 134

AWS Website Storage Servers 134

All Things Distributed

DECEMBER 9, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Big Just Got Bigger - 5 Terabyte Object Support in Amazon S3. Amazon S3 has always been a scalable, durable and available data repository for almost any customer workload. Driving down the cost of Big-Data analytics.

AWS 80

AWS Big Data Scalability Storage 80

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. With this change, we will improve the granularity of pricing information you receive by introducing a Spot Instance price per Availability Zone rather than a Spot Instance price per Region. Driving down the cost of Big-Data analytics.

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

MAY 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. All Things Distributed. New Route 53 and ELB features: IPv6, Zone Apex, WRR and more. Comments ().

Internet 114

Internet Internet AWS Scalability 114

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. A whole range of innovative new services, ranging from media conversion to geo-location-context services have been developed by our customers using this flexibility and are available in the AWS ecosystem. All Things Distributed. Comments ().

AWS 115

AWS Cloud Java Operating System 115

All Things Distributed

MARCH 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. To make it easy we have free AWS usage credits available onsite for who ever needs them. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. Comments ().

AWS 93

AWS Internet Internet Storage 93

All Things Distributed

JULY 3, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There is a tremendous interest in using the AWS multi-Availability Zone features for disaster prevention and recovery in the wake of the March 11 earthquake. I will update this posting as more details about public events become available. APAC Summer Tour.

AWS 67

AWS Storage Big Data Cloud 67

All Things Distributed

MARCH 28, 2011

Werner Vogels weblog on building scalable and robust distributed systems. As a big music fan with well over 100Gb in digital music I am particularly excited that I now have access to all my digital music anywhere I go. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed.

AWS 138

AWS Cloud Storage Internet 138

Percona

JANUARY 14, 2019

What if we use ClickHouse (which is a columnar analytical database) as our main datastore? Well, typically, an analytical database is not a replacement for a transactional or key/value datastore. However, ClickHouse is super efficient for timeseries and provides “sharding” out of the box (scalability beyond one node).

Database 40

Database Analytics Blockchain Healthcare 40

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). AWS is committed to making its services available at low cost. Driving down the cost of Big-Data analytics.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Route 53 provides Authoritative DNS functionality implemented using a world-wide network of highly-available DNS servers.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. Today a powerful new feature is available for our Amazon EC2 customers: the ability to boot their instances from Amazon EBS (Elastic Block Store). A wide variety of operating systems and software configurations is available for use. Comments ().

AWS 60

AWS Storage Operating System Cloud 60

All Things Distributed

MAY 18, 2010

Werner Vogels weblog on building scalable and robust distributed systems. This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. We use several techniques to ensure the durability of the data our customers trust us with, and some of those (e.g.

Storage 71

Storage Cloud AWS Scalability 71

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Today, I am very proud to be a part of the Amazon Web Services team as we truly make HPC available as an on-demand commodity for every developer to use. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Comments ().

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. This incredible power is available for anyone to use in the usual pay-as-you-go model, removing the investment barrier that has kept many organizations from adopting GPUs for their workloads even though they knew there would be significant performance benefit.

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Availability - The cloud makes it possible to build resilient applications to make sure they can survive different failure scenarios. The Asia Pacific (Singapore) region launches with two Availability Zones. Driving down the cost of Big-Data analytics.

AWS 61

AWS Cloud Latency Storage 61

All Things Distributed

FEBRUARY 24, 2010

Werner Vogels weblog on building scalable and robust distributed systems. There are many factors that come into play when you need to meet stringent availability and performance requirements under ultra-scalable conditions. Data Consistency Models in the Amazon Services. All Things Distributed. Choosing Consistency.

AWS 69

AWS Latency Database Scalability 69

KeyCDN

OCTOBER 2, 2018

It’s awesome for discovering how grid systems, CSS animation, Big Data, etc all play roles in real-world web design. CSSWizardy CSSWizardy is a good place to learn about scalable CSS practices and robust coding in general. Frontend Focus Frontend Focus’s issues are available from their site or as subscription emails.

Development 75

Development Website Design Code 75

Highly Scalable

MARCH 10, 2015

Although there are many books on data mining in general and its applications to marketing and customer relationship management in particular [BE11, AS14, PR13 etc.], The rest of the article is organized as follows: We first introduce a simple framework that ties together a retailer’s actions, profits and data. The model (2.1)

Retail 152

Retail C++ Analytics Metrics 152

All Things Distributed

DECEMBER 13, 2009

Werner Vogels weblog on building scalable and robust distributed systems. You are assured that your Reserved Instance will always be available in the Availability Zone in which you purchased it. This snapshot-restart technique is a well known methodology already available to many batch oriented applications. Comments ().

Cloud 63

Cloud AWS Storage Innovation 63

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. Figure 1 shows how we use Bulldozer to move data at Netflix. Moving data with Bulldozer at Netflix.

Latency 243

Latency Storage Big Data Tuning 243