All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. Government and Big Data. All Things Distributed. Comments ().

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. In particular this has been true for applications based on algorithms - often MPI-based - that depend on frequent low-latency communication and/or require significant cross sectional bandwidth. Driving down the cost of Big-Data analytics.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Often these namespaces are hierarchical in nature such that it becomes easier to manage them and to decentralize control, which makes the system more scalable. DNS is an absolutely critical piece of the internet infrastructure. All Things Distributed.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. However, when speaking about cloud services in terms of Infrastructure-as-a-Service, it is very important to make the geographic locations of services more explicit. The new Singapore Region offers customers in APAC lower-latency access to AWS services.

AWS 61

AWS Cloud Latency Storage 61

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. The processed data is typically stored as data warehouse tables in AWS S3. Moving data with Bulldozer at Netflix.

Latency 243

Latency Storage Big Data Tuning 243

All Things Distributed

APRIL 4, 2017

The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. After migrating, database queries that took six seconds now take three seconds in their AWS infrastructure.

AWS 116

AWS Airlines Latency Games 116

The Netflix TechBlog

DECEMBER 21, 2020

These principles reduce resource usage by being more efficient and effective while lowering the end-to-end latency in data processing. AutoOptimize relies on some of the Iceberg specific features such as snapshot and atomic operations to perform the optimizations in an accurate and scalable manner. More processing resources.

Storage 203

Storage Latency Efficiency Data Engineering 203

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. For example, the most fundamental abstraction trade-off has always been latency versus throughput. The throughput of this pipeline is more important than the latency of the individual operations. Driving down the cost of Big-Data analytics.

AWS 136

AWS Latency Programming Architecture 136

The Netflix TechBlog

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale. This makes the query service lightweight, scalable, and execution agnostic.

Processing 265

Processing Engineering Infrastructure Latency 265