All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data 111

Big Data Analytics AWS Cloud 111

The Morning Paper

OCTOBER 26, 2020

As a production system within Microsoft capturing around a quadrillion events and indexing 16 trillion search keys per day it would be interesting in its own right, but there’s a lot more to it than that. These two narratives of reference architecture and ingestion/indexing system are interwoven throughout the paper.

Cloud 104

Cloud Big Data Latency Architecture 104

ScaleOut Software

JULY 19, 2021

How are we managing the torrent of telemetry that flows into analytics systems from these devices? Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The list goes on.

IoT 78

IoT Analytics Big Data Architecture 78

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. No Server Required - Jekyll & Amazon S3. As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. No Server Required. Comments ().

Servers 126

Servers Social Media AWS Website 126

All Things Distributed

MAY 18, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Amazon S3 Reduced Redundancy Storage. Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). Under the covers Amazon S3 is a marvel of distributed systems technologies.

Storage 71

Storage Cloud AWS Scalability 71

SQL Server According to Bob

JANUARY 15, 2020

SQL Server BDC Hints and Tips: The node’s Journal can be your best friend If you are new to Kubernetes (K8’s), like I am, you quickly learn that the servicing node’s journal holds a wealth of information. ​​ For example, my master-0, SQL Server, pod was getting evicted and restarted.

Servers 40

Servers Metrics Big Data Operating System 40

All Things Distributed

DECEMBER 13, 2016

With the launch of the AWS Europe (London) Region, AWS can enable many more UK enterprise, public sector and startup customers to reduce IT costs, address data locality needs, and embark on rapid transformations in critical new areas, such as big data analysis and Internet of Things. Fraud.net is a good example of this.

AWS 166

AWS Cloud Artificial Intelligence IoT 166

All Things Distributed

FEBRUARY 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. I had held out implementing an alternative to my simple blog server that had.

AWS 134

AWS Website Storage Servers 134

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. Why are developers using RInK systems as part of their design? Fetching too much data in a single query (i.e.,

Cache 79

Cache Latency Google Lambda 79

All Things Distributed

JULY 7, 2011

Werner Vogels weblog on building scalable and robust distributed systems. AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet. Amazon Import/Export is an important tool for customers to accelerate moving large amounts of data into the AWS storage systems.

AWS 96

AWS Cloud Storage Internet 96

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Systems that make extensive use of caching almost all report a significant reduction in the cost of their database tier. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. All Things Distributed.

Cloud 117

Cloud Cache AWS Storage 117

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Naming is one of the fundamental concepts in Distributed Systems. Authoritative servers hold the definitive mappings.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

FEBRUARY 23, 2011

Werner Vogels weblog on building scalable and robust distributed systems. In a follow up to the last blog post I have removed the last two dependencies this blog had on running a server somewhere: comments are now served by Disqus and search is now handled by Bing. Driving Storage Costs Down for AWS Customers. Comments ().

AWS 78

AWS Storage Big Data Servers 78

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. And while many of our systems are based on the latest in computer science research, this often hasnt been sufficient: our architects and engineers have had to advance research in directions that no academic had yet taken. All Things Distributed. Comments ().

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Expanding the Cloud â??

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements.

AWS 115

AWS Cloud Java Scalability 115

All Things Distributed

AUGUST 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. The ideal candidate will be someone who has built and ran large scale distributed systems and/or databases.

AWS 75

AWS Database Storage Scalability 75

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

MARCH 28, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Our smart phones and tablets are obvious examples, but many other devices are quickly gaining these capabilities; TV Sets and Hifi systems are internet enabled, and soon our treadmills and automobiles will be equally plugged into the digital world. Comments ().

AWS 138

AWS Cloud Storage Internet 138

All Things Distributed

FEBRUARY 25, 2011

Werner Vogels weblog on building scalable and robust distributed systems. When a new customer is onboarded, the ISV has to spin up a collection of AWS resources to run their web-servers, app-servers and databases in a multi-AZ (availability zone) setting to achieve high-availability. Driving Storage Costs Down for AWS Customers.

AWS 121

AWS Cloud Scalability Storage 121

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 112

AWS Cloud Games Latency 112

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 104

AWS Retail Innovation Strategy 104

All Things Distributed

MARCH 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. It is likely that the Amazon Web Services will be used by many of the participants for their compute, storage, database and other cloud resource needs. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

AWS 93

AWS Internet Internet Storage 93

All Things Distributed

FEBRUARY 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 60

AWS Storage Big Data Scalability 60

All Things Distributed

JANUARY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. DROAM - Dreaming about Cheap Data Roaming. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. Driving Storage Costs Down for AWS Customers. Driving down the cost of Big-Data analytics.

Wireless 107

Wireless AWS Internet Internet 107

All Things Distributed

DECEMBER 9, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Big Just Got Bigger - 5 Terabyte Object Support in Amazon S3. By supporting such large object sizes, Amazon S3 better enables a variety of interesting big data use cases. Driving Storage Costs Down for AWS Customers. All Things Distributed.

AWS 80

AWS Big Data Scalability Storage 80

All Things Distributed

SEPTEMBER 29, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 69

AWS Storage Big Data Scalability 69

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Not just for HPC but for mission critical enterprise systems such as OLTP. Dedicated High Performance Compute clusters require significant capital investments and their procurement often has longer lead times than many enterprise class server systems.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

MAY 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

Internet 114

Internet Internet AWS Scalability 114

All Things Distributed

JULY 3, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. APAC Summer Tour. Comments ().

AWS 67

AWS Storage Big Data Cloud 67

Highly Scalable

MARCH 1, 2012

This aspect of NoSQL is well-studied both in practice and theory because specific non-functional properties are often the main justification for NoSQL usage and fundamental results on distributed systems like the CAP theorem apply well to NoSQL systems. Processing complexity VS total data volume. 2) Aggregates.

Database 279

Database Ecommerce Efficiency Engineering 279

All Things Distributed

APRIL 23, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 60

AWS Storage Cloud Big Data 60

All Things Distributed

SEPTEMBER 30, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Around the World in 28 Days.

AWS 75

AWS Storage Cloud Best Practices 75

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Comments (). At werner.ly

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. A wide variety of operating systems and software configurations is available for use. with more memory or CPU), a different operating system (e.g., with new security patches installed), or add new user data. No Server Required - Jekyll & Amazon S3.

AWS 60

AWS Storage Operating System Cloud 60

All Things Distributed

NOVEMBER 19, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics.

AWS 67

AWS Cloud Benchmarking Storage 67