All Things Distributed

AUGUST 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. This week it is an opening for senior leaders with AWS Database Services. Countdown to What is Next in AWS.

AWS 75

AWS Database Storage Scalability 75

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. To reach those goals, Shell in 2010 began using AWS. These sensors create massive amounts of geological data.

Cloud 129

Cloud Energy AWS Healthcare 129

The Morning Paper

JUNE 23, 2019

After all, we’ve been doing that forever with the 2nd-level cache of ORMs , and it is highly encouraged in e.g. the AWS Lambda programming model — which was born on the cloud— to help mitigate function start-up times. We’ve seen similar high marshalling overheads in big data systems too.) From RInK to LInK.

Cache 79

Cache Latency Google Lambda 79

All Things Distributed

SEPTEMBER 30, 2010

Werner Vogels weblog on building scalable and robust distributed systems. With existing customers I get a change to dive deep on their AWS usage and understand what works well and where we can do better. With existing customers I get a change to dive deep on their AWS usage and understand what works well and where we can do better.

AWS 75

AWS Storage Cloud Best Practices 75

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Additionally, many high-end HPC applications take advantage of knowing their in-house hardware platforms to achieve major speedup by exploiting the specific processor architecture. Driving Storage Costs Down for AWS Customers. All Things Distributed.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

NOVEMBER 19, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. Comments ().

AWS 67

AWS Cloud Benchmarking Storage 67

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. All Things Distributed. Comments (). Some fundamentals on Naming.

Cloud 117

Cloud Internet Internet AWS 117

The Netflix TechBlog

MARCH 25, 2019

We adopted the following mission statement to guide our investments: “Provide a complete and accurate data lineage system enabling decision-makers to win moments of truth.” Nonetheless, Netflix data landscape (see below) is complex and many teams collaborate effectively for sharing the responsibility of our data system management.

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Meson was based on a single leader architecture with high availability.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

OCTOBER 27, 2020

Usually Data scientists and engineers write Extract-Transform-Load (ETL) jobs and pipelines using big data compute technologies, like Spark or Presto , to process this data and periodically compute key information for a member or a video. The processed data is typically stored as data warehouse tables in AWS S3.

Latency 243

Latency Storage Big Data Tuning 243

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks. Driving Storage Costs Down for AWS Customers.

Technology 151

Technology Technology AWS Storage 151

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. Then deep dive into the merging use case of AutoOptimize and share some results and benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

All Things Distributed

MARCH 28, 2011

Werner Vogels weblog on building scalable and robust distributed systems. The scalability, reliability and durability requirements for Cloud Drive are very high which is why they decided to make use of the Amazon Simple Storage Service (S3) as the core component of their service. Driving Storage Costs Down for AWS Customers.

AWS 138

AWS Cloud Storage Internet 138

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. These trade-offs have even impacted the way the lowest level building blocks in our computer architectures have been designed. All Things Distributed. Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. Comments ().

AWS 136

AWS Latency Programming Architecture 136