AWS re:Invent 2013

All Things Distributed

Today we are kicking off AWS re:Invent 2013. Over the course of the next three days, we will host more than 200 sessions, training bootcamps, and hands on labs taught by expert AWS staff as well as dozens of our customers. This year’s conference kicks off with a keynote address by AWS Senior Vice President Andy Jassy, followed by my keynote on Thursday morning. Tune in to hear the latest from AWS and our customers.

AWS 69

AWS re:Invent 2013 - All Things Distributed

All Things Distributed

AWS re:Invent 2013. By Werner Vogels on 17 July 2013 05:00 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Permalink. Comments (). The AWS re:Invent user conference last year in Las Vegas was by many described as the best technology conference they had been to in a long time. We had worked hard to give you great keynote sessions as well as deep technical content by AWS engineers, partners and customers.

AWS 67
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

2013 Favorites

Tim Kadlec

Here are 2013’s favorites, in order of popularity. Time for the long-running (since way back in 2012 !) annual tradition of listing the top five most popular posts on this site of the past year. Why We Need Responsive Images. The responsive image discussion is the Never Ending Story of web technology.

Media 40

In-Stream Big Data Processing

Highly Scalable

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It became clear that real-time query processing and in-stream processing is the immediate need in many practical applications. In recent years, this idea got a lot of traction and a whole bunch of solutions like Twitter’s Storm, Yahoo’s S4, Cloudera’s Impala, Apache Spark, and Apache Tez appeared and joined the army of Big Data and NoSQL systems.

Expanding the Cloud - Introducing AWS OpsWorks, a Powerful.

All Things Distributed

By Werner Vogels on 18 February 2013 11:30 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Introducing AWS OpsWorks, a Powerful Application Management Solution. Permalink. Comments (). Today Amazon Web Services launched AWS OpsWorks , a flexible application management solution with automation tools that enable you to model and control your applications and their supporting infrastructure.

AWS 125

Heading to Mobilism 2013

Tim Kadlec

A week from now I’ll be on a plane heading to Amsterdam for the excellent Mobilism conference. This is year three for them, and you only need to take one look at this year’s program to see just how incredible the event is going to be. The lineup is full of great speakers with fantastic topics to discuss! Just as impressive is the list of attendees —there’s sure to be some great discussions with a group like this all in one place.

What I Read in 2013

Tim Kadlec

This is a tough year for me to list my top three non-fiction and fiction books because there were a lot of really good ones—particularly in fiction. The fact that there were so many great books probably contributes to why I was able to reverse the downward trend in my book count the past few years. That being said, for fiction I’d have probably have to rank Lexicon , Dust and Ocean at the End of the Lane as my top three.

Expanding the Cloud: Enabling Globally Distributed Applications and Disaster Recovery

All Things Distributed

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6 and enable you to maintain a nearly up-to-date copy of your master database in a different AWS Region.

AWS 102

Taking DynamoDB beyond Key-Value: Now with Faster, More Flexible, More Powerful Query Capabilities

All Things Distributed

You saw the first iteration in April 2013 with the launch of Local Secondary Indexes (LSI). We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability.

Games 95

AWS Activate – Supporting Startups on AWS

All Things Distributed

I am very excited to announce AWS Activate , a program designed to provide startups with the resources they need to build applications on AWS. Startups will forever be a very important customer segment of AWS. They were among our first customers and along the way some amazing businesses have been built by these startups, many of which running for 100% on AWS.

AWS 93

Updated Lampson's Hints for Computer Systems Design

All Things Distributed

This year I have not been able to publish many back-to-basics readings, so I will not close the year with a recap of those. Instead I have a video of a wonderful presentation by Butler Lampson where he talks about the learnings of the past decades that helped him to update his excellent 1983 " Hints for computer system design ". The presentation was part of the Heidelberg Laureate Forum helt in September of this year. At the Forum many of the Abel, Fields and Turing Laureates held presentations.

Dutch Enterprises and The Cloud

All Things Distributed

This spring I travelled through Europe for the AWS Global Summit series. In my many conversations with customers, and with the media, I encountered surprise and excitement about the extent that European enterprises have already been using the Amazon Web Services for some time.

Cloud 92

Expanding the Cloud: More memory, more caching and more performance for your data

All Things Distributed

Today, we added two important choices for customers running high performance apps in the cloud: support for Redis in Amazon ElastiCache and a new high memory database instance (db.cr1.8xlarge) for Amazon RDS.

Cache 92

Exerting Fine Grain Control Over Your Cloud Resources - All Things.

All Things Distributed

By Werner Vogels on 07 July 2013 06:30 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Exerting Fine Grain Control Over Your Cloud Resources. Permalink. Comments (). I am thrilled that now both Amazon EC2 and Amazon RDS support resource-level permissions. As customers move increasing amounts of compute and database workloads over to AWS, they have expressed an increased desire for finer grain control over their underlying resources.

Cloud 80

The Netflix OSS Cloud Prize - All Things Distributed

All Things Distributed

By Werner Vogels on 20 March 2013 05:00 PM. The contest will run through September 15 2013 after which a Judging Panel, which I am excited to be part of, will pick the winners. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. The Netflix OSS Cloud Prize. Permalink. Comments (). Netflix has over the years become one of the absolute best engineering powerhouses for building cloud-native applications.

Cloud 77

DynamoDB Keeps Getting Better (and cheaper!) - All Things.

All Things Distributed

By Werner Vogels on 15 May 2013 06:30 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. DynamoDB Keeps Getting Better (and cheaper!). Permalink. Comments (). We love getting feedback so we can deliver the improvements and new features that really matter to our customers. You can see from the pace at which we roll out new functionality that teams across AWS take this very seriously. One of the teams thatâ??s

Feeling the Customer Love for AWS - All Things Distributed

All Things Distributed

By Werner Vogels on 19 July 2013 11:00 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Feeling the Customer Love for AWS. Permalink. Comments (). We work hard to meet our customers expectations and to continue to innovate on their behalf. This week at the Singapore AWS Summit we were fortunate that our customers Astro Radio from Kuala Lumpur were willing to join us on stage.

AWS 74

Expanding the Cloud: Faster, More Flexible Queries with DynamoDB

All Things Distributed

By Werner Vogels on 17 April 2013 10:30 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud: Faster, More Flexible Queries with DynamoDB. Permalink. Comments (). Today, Iâ??m m thrilled to announce that we have expanded the query capabilities of DynamoDB. We call the newest capability Local Secondary Indexes (LSI). While DynamoDB already allows you to perform low-latency queries based on your tableâ??s

Games 80

Simplifying Mobile App Data Management with DynamoDB's Fine-Grained Access Control

All Things Distributed

Speed of development, scalability, and simplicity of management are among the critical needs of mobile developers. With the proliferation of mobile devices and users, and small agile teams that are tasked with building successful mobile apps that can grow from 100 users to 1 million users in a few days, scalability of the underlying infrastructure and simplicity of management are more important than ever.

Amazon Redshift and Designing for Security - All Things Distributed

All Things Distributed

By Werner Vogels on 22 May 2013 08:30 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Amazon Redshift and Designing for Security. Permalink. Comments (). s been a few months since I last wrote about Amazon Redshift and I thought Iâ??d d update you on some of the things we are hearing from customers. Since we launched, weâ??ve ve been adding over a hundred customers a week and are well over a thousand today. Thatâ??s s pretty stunning.

Amazon Redshift and Designing for Resilience - All Things Distributed

All Things Distributed

By Werner Vogels on 15 February 2013 12:01 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Amazon Redshift and Designing for Resilience. Permalink. Comments (). As you may remember from our announcement at re: Invent in November 2012, Amazon Redshift is a fast and powerful, fully managed, petabyte-scale data warehouse service that delivers fast query performance at less than one tenth the cost of most traditional data warehouse systems.

Feeling the Customer Love for AWS

All Things Distributed

We work hard to meet our customer’s expectations and to continue to innovate on their behalf. This week at the Singapore AWS Summit we were fortunate that our customers Astro Radio from Kuala Lumpur were willing to join us on stage.

AWS 78

Making Mobile App Development Easier with Cross Platform Mobile Push

All Things Distributed

This year as I hosted AWS Summits in 12 different cities around the world, I met thousands of developers who are building powerful new applications for smartphones, tablets and other connected devices, all running mobile cloud backends on AWS. These developers want to engage their users with timely, dynamic content even when the users haven’t opened their mobile apps.

The Amazon Elastic Transcoder - All Things Distributed

All Things Distributed

By Werner Vogels on 11 February 2013 04:00 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud: The Amazon Elastic Transcoder. Permalink. Comments (). While I was returning from an exciting time in New Orleans watching the Super Bowl, AWS launched a very cool, brand new service: Amazon Elastic Transcoder.

Media 72

Back-to-Basics Weekend Reading - U-Net: A User-Level Network Interface

All Things Distributed

Many of you know Thorsten von Eicken as the founder of Rightscale , the company that has helped numerous organizations find their way onto AWS. In what seems almost a previous life by now Thorsten was one of the top young professors in Distributed Systems and I had the great pleasure of working with him at Cornell in the early 90''s. What set Thorsten aside from so many other system research academics was his desire to build practical, working systems, a path that I followed as well.

Back-to-Basics Weekend Reading - Join Processing in Relational.

All Things Distributed

By Werner Vogels on 12 April 2013 04:00 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Join Processing in Relational Databases. Permalink. Comments (). Joins are one of the fundamental relational database query operations. It is very hard to implement the join operation efficiently as there any many unknowns in the execution of the operation.

Elastic Beanstalk a la Node - All Things Distributed

All Things Distributed

By Werner Vogels on 11 March 2013 04:00 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Elastic Beanstalk a la Node. Permalink. Comments (). I spent a lot of time talking to AWS developers, many working in the gaming and mobile space, and most of them have been finding Node.js well suited for their web applications. With its asynchronous, event-driven programming model, Node.js

Games 71

Back-to-Basics Weekend Reading - Auctions and bidding: A guide.

All Things Distributed

By Werner Vogels on 08 June 2013 06:00 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Auctions and bidding: A guide for computer scientists. Permalink. Comments (). I have just returned from the AWS Summits in New Zealand and Japan, which were both very well attended and, according to the feedback, very successful.

DynamoDB for Location Data: Geospatial querying on DynamoDB datasets

All Things Distributed

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Back-to-Basics Weekend Reading - An Introduction to Spatial Database Systems

All Things Distributed

Storing and querying datasets that contain objects in a geometric space have always required special treatment. The choice of data structures and query algorithms can easily make the different between a query that runs in seconds or in days. Much of the fundamental work has been done in the late eighties and early nineties, for examples around topological relations (disjoint, meet, equal, overlap, contains, etc.), direction relations (north, north-east, etc.)

Back-to-Basics Weekend Reading - A Decomposition Storage Model

All Things Distributed

Traditionally records in a database were stored as such: the data in a row was stored together for easy and fast retrieval. Not everybody agreed that the "N-ary Storage Model" (NSM) was the best approach for all workloads but it stayed dominant until hardware constraints, especially on caches, forced the community to revisit some of the alternatives.

Back-to-Basics Weekend Reading - Tor: The Second-Generation Onion Router

All Things Distributed

The anonymity routing network Tor is frequently in the news these days, which makes it a good case to read up on the fascinating technologies behind it. Tor stands for The Onion Router as its technology is based on the onion routing principles. These principles were first described by Goldschlag, et al., from the Naval Research Lab, in their 1996 paper on Hiding Routing Information.

Back-to-Basics Weekend Reading - Principles of Transaction.

All Things Distributed

By Werner Vogels on 29 March 2013 03:30 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Principles of Transaction-Oriented Database Recovery. Permalink. Comments (). I have been reading mainly newer papers in the beginning of this year, but it is time to get back to the basics and start reading some more historical papers again.

Back-to-the-Future Weekend Reading - Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud

All Things Distributed

The intense travels around the world in the spring have kept me from keeping up on the historical reading that I would like to do, as such there have not been that many suggesting for the back-to-basics reading list. The fall is going be not that much different but I will make an effort to get back into a reading habit.

Back-to-Basics Weekend Reading - Practical Applications of.

All Things Distributed

By Werner Vogels on 05 April 2013 08:30 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Practical Applications of Triggers and Constraints: Successes and Lingering Issues. Permalink. Comments (). At the end of the 80s Ceri and Widom were researching the fundamentals of integrity constraints in databases.

Back-to-Basics Weekend Reading - Using continuations to.

All Things Distributed

By Werner Vogels on 10 May 2013 09:30 AM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Using continuations to implement thread management and communication in operating systems. Permalink. Comments (). I have returned from a great series of AWS Summits in NYC and in Europe so it is time to get back to some weekend reading.

Back-to-Basics Weekend Reading - Epidemics - All Things Distributed

All Things Distributed

By Werner Vogels on 25 January 2013 06:00 PM. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Epidemics. Permalink. Comments (). My paper to read this weekend was the Alan Demers seminal paper on epidemic techniques for database replication.

Recommended reading: Why mobile web apps are slow (Drew Crawford)

Sutter's Mill

I don’t often link to other articles, but this one is worth reading. Why mobile web apps are slow. by Drew Crawford. … So if you are trying to figure out exactly what brand of crazy all your native developer friends are on for continuing to write the evil native applications on the cusp of the open web revolution, or whatever, then bookmark this page, make yourself a cup of coffee, clear an afternoon, find a comfy chair, and then we’ll both be ready. He offers data (imagine!)

Fear, Uncertainty and “Desperation”

All Things Distributed

''

60

“256 cores by 2013”?

Sutter's Mill

So: Was I was right about 2013 estimates? First, let’s look at the lower baseline, ‘most general mainstream users to have [4-16 way parallelism] machines in 2013’? Second, what about the higher potential line for 2013? So not only did we already reach the 2013 upper line early, in 2012, but we already exceeded it for applications that can harness the GPU for computation. I just saw a tweet that’s worth commenting on: Almost right, and we have already reached that.

Why we need responsive images

Tim Kadlec

The topic of responsive images has been one of the most hotly debated topics amongst web developers for what feels like forever. I think Jason Grigsby was perhaps the first to publicly point out that simply setting a percentage width on images was not enough, you needed to resize these images as well. He showed that if you served appropriately sized images on the original responsive demo site, you could shave 78% off the weight of those images (about 162kB) on small screens.

Performance tips for building responsive sites

Speed Curve

The following article was originally published in the 2013 Performance Calendar. There's 31 great articles to explore in the calendar including Steve Souders's browser wishlist and Tim Kadlec's take on what it takes to create a performance culture. -. Responsive Web Design (RWD) is now a well established technique yet it’s adoption is still surprisingly low. Guy Podjarny’s recent research shows that only one in eight websites is responsive.

Conflict, Part I

The Agile Manager

Product management was never a formal responsibility; it just sort of happened. Early on, it was driven by what the technical wizards came up with. But the magic left the development team years ago: it had been gutted by several rounds of staff cuts that took the garrulous personalities and innovative thinkers. It took the wind from development's sails: those who were still on the payroll were just happy to have kept their jobs.