All Things Distributed

AUGUST 9, 2007

Amazon Hackday 2007 in progress. Midnight in Seattle. World-wide

78

78

All Things Distributed

NOVEMBER 19, 2007

The emotions about reading books in digital form and the Amazon Kindle are running high, already before the device was released. For me there are two features that sold me on the device: the networked content push and the content interaction.

Network 78

Network 78

All Things Distributed

NOVEMBER 15, 2007

On May 31 2008 a tribute will be held at UC Berkeley to honor Jim Gray, who went missing during a solo sailing trip in January of this year. Although Jim is listed as missing, and will be until 2011, a Tribute be held to honor him before too much time has passed

78

78

All Things Distributed

DECEMBER 19, 2007

I wrote a first version of this posting on consistency models in December 2007, but I was never happy with it as it was written in haste and the topic is important enough to receive a more thorough treatment. ACM Queue asked me to revise it for use in their magazine and I took the opportunity to improve the article

60

60

All Things Distributed

SEPTEMBER 17, 2007

A popular request has been to make more of the Amazon product data available as easy widgets for reuse on blogging, social network and associates websites.

Website 78

Website Network Availability 78

All Things Distributed

JULY 21, 2007

2.223 million pre-orders on our sites world-wide. million on Amazom.com alone. These orders trickled in over the period of 5.5 months, but from a distributed systems perspective today is the day as these orders go en-masse from pre-orders to orders, being charged and delivered.

Transportation 78

Transportation Systems 78

All Things Distributed

OCTOBER 9, 2007

The 2007 Nobel prize in Physics has been awarded to Albert Fert and Peter Grünberg for independently discovering Giant Magnetoresistance in 1988. Their work had a tremendous impact on the computer industry as it revolutionized the way we could store and retrieve information on hard disks. It was the first major application of nanotechnology and allowed hard disks to shrink from the size of a large washing machine to the device that fits in an mp3 player

60

60

All Things Distributed

APRIL 18, 2007

Today at noon Swami made the transition to Dr. Swaminathan Sivasubramanian at the Vrije Universiteit in Amsterdam. In most European countries the doctoral defense (promotie in Dutch) goes with a lot of protocol and pomp & circumstance.

Code 78

Code 78

All Things Distributed

APRIL 16, 2007

Congrats to Scott Berkun for sending his "Myths of Innovation"book to the printer. Scott has collected an excellent set of recommendations for his book; among others JSB, Guy and me. It is well deserved; I read a beta version of the book and I was impressed.

Innovation 78

Innovation 78

All Things Distributed

FEBRUARY 11, 2007

As Mike Olsen just wrote on the Tenacious Search weblog, today was the first day that action could be taken on the boats found in the satellite and ER-2 streams. Bad weather has kept any aerial search parties on the ground until now.

78

78

All Things Distributed

FEBRUARY 6, 2007

Over 530,000 Mechanical Turk assignments have been completed by more than 12,000 volunteers in the search for Jim Gray. We need a little more of a push and then all the images will have been processed.

Processing 78

Processing 78

All Things Distributed

FEBRUARY 4, 2007

It is now 3 PM on Sunday afternoon and the group of volunteers in thethe search for Jim Gray has worked their way through almost 100,000 assignments since Friday 5 PM. Since then we have seen over 6000 individual workers completing anywhere from 1 to almost a 1000 assignments.

78

78

All Things Distributed

FEBRUARY 3, 2007

We have now added the data captured by the NASA ER-2 plane yesterday over the ocean area outside of San Francisco. We were very fortunate that this flight was scheduled for yesterday and that the NASA folks were interested in having it capture these images.

78

78

All Things Distributed

FEBRUARY 2, 2007

Computer science icon Jim Gray mysteriously disappeared after a solo trip with his sail boat outside San Francisco Bay. The coast guard has been searching for 4 days but has not been able to locate anything, not even debris.

78

78

All Things Distributed

JANUARY 29, 2007

My long time friend and mentor Jim Gray took his sail boat out for a trip to the Farallon Island yesterday and has not been heard from since. The coast guard has been searching since last night. Jim is an extremely experienced sailor with more than 40 years experience. This is very worrisome

78

78

All Things Distributed

OCTOBER 2, 2007

In two weeks we’ll present a paper on the Dynamo technology at SOSP, the prestigious biannual Operating Systems conference. Dynamo is internal technology developed at Amazon to address the need for an incrementally scalable, highly-available key-value storage system. The technology is designed to give its users the ability to trade-off cost, consistency, durability and performance, while maintaining high-availability

Storage 62

Storage Operating System Technology Technology 62

All Things Distributed

DECEMBER 20, 2007

North Shore. 10 Days. See you next year

60

60

All Things Distributed

DECEMBER 9, 2007

Sadness. Marc Orchant, the ulimate optimist, is no longer with us

60

60

All Things Distributed

DECEMBER 6, 2007

Today we had a wonderful day with the finalists of the 900 contestants in the AWS Startup Challenge, all seven proved themselves worthy of winning the ultimate price. The finalists were: Brainscape Commerce360 Justin.tv Milemeter Ooyala WeoGeo Go to the startup challenge vote page to see videos of each of these companies and to see the voting results

AWS 60

AWS 60

All Things Distributed

NOVEMBER 5, 2007

I am about to go on stage at Web 2.0 Expo in Berlin where we will announce the ability for Amazon S3 customers to store their objects in Amazon’s European storage cloud. This has been a frequently requested feature by our European customers for various reasons, better latency control being the most important one

Latency 60

Latency Storage Cloud 60

All Things Distributed

OCTOBER 23, 2007

If we would just forget about discriminatory traffic management (e.g. based on deep packet inspection) life on the network would be pretty simple, even under overload conditions. A lot has been written about this already but Wes really nails it in his summary. Maximized profit, minimized frustration, everybody’s happy

Traffic 60

Traffic Network 60

All Things Distributed

OCTOBER 6, 2007

The Institutional Yes is a Harvard Business Review interview with Jeff Bezos about the way strategies are developed at Amazon. I have written before about how the relentless customer focus translates into driving architecture and design using the “working backwards” approach. The interview with Jeff gives you more insight on the impact of customer focus on overall strategy and how it drives a culture of experimentation

Strategy 60

Strategy Architecture Design Development 60

All Things Distributed

OCTOBER 3, 2007

There is a lot of positive feedback about the Dynamo paper but I noticed that something I wrote in introducing the paper is being misunderstood. This was my fault, I wrote it too strongly. What I meant by internal-only is that Dynamo is not directly exposed externally. However, Dynamo and similar Amazon technologies are used to power parts of our Amazon Web Services, such as S3

Technology 60

Technology Technology 60

All Things Distributed

OCTOBER 1, 2007

My old friend Steve Vinoski is back online. Steve was with Iona for many, many years, working as the main architect on many of their Middleware technologies. Steve left Iona this spring to work for a start-up in the Boston area. But Steve became really famous for this drawing, which he used to express his opinion about my keynote at Middleware 2004

Technology 60

Technology Technology 60

All Things Distributed

SEPTEMBER 30, 2007

As a follow-up to the keynote I gave at the Next Generation Datacenter Conference the folks at Network World organized an online chat, to happen for an hour tomorrow (October 1) from 11-12 PST. I have never done something like this before so it will be fun to see how good my typing skills are, but at least I get to sneak out of a very long meeting for an hour

Network 60

Network 60

All Things Distributed

SEPTEMBER 29, 2007

I just finished reading the Wired interview with Ridley Scott about Blade Runner, the ultimate film noir. The movie is certainly one of my favorites and I own several editions of it. Of course I now also want this “final cut version”, hoping that it would be available in HD. It turns out it being produced in 7 different packagings, all to be released on December 18 (which is pretty late for online holiday shoppers in my eyes

Availability 60

Availability 60

All Things Distributed

SEPTEMBER 28, 2007

In the recent years Mike Stonebraker has been advocating that the current commercial databases have become one-size-fits-all tools that are so general and heavy-weight that they do not excel in any particular area. Mike has written some papers on this topic of comparing general databases with specialized storage engines in areas of data-warehousing, text search, stream processing and scientific processing

Storage 60

Storage Database Processing Engineering 60

All Things Distributed

SEPTEMBER 24, 2007

The beta of the long awaited DRM free Amazon music store is ready for use. Go download

60

60

All Things Distributed

SEPTEMBER 10, 2007

Since Saturday morning you can help search for Steve Fossett through Amazon's Mechanical Turk. Click here to get to the start page and then press the "Work on HITs" button to start reviewing satellite images. As with the search for Jim Gray, this is only possible because of the collaboration of several organizations

60

60

All Things Distributed

JULY 11, 2007

I was putting together a short panel presentation on the role of a Chief Technology Officer in corporate innovation and I once again realized that there is quite a bit of confusion around the role of the CTO. The first thing that always comes up when you want to discuss the role of a CTO is that there is no well established definition of what a CTO actually does

Innovation 60

Innovation Technology Technology 60

All Things Distributed

AUGUST 2, 2007

Today Amazon AWS launched a limited public beta of Amazon FPS, the Amazon Flexible Payment Service. Amazon FPS is a payment service that is 100% focused on the needs of developers. Traditionally, developers have been limited in how they can manage payments.

AWS 60

AWS Infrastructure Development 60

All Things Distributed

JULY 19, 2007

When I was building one of my first teams at Amazon, one that had to work on some really advanced distributed systems technology I put up a job description on this weblog. I was certainly pleased with the responses. Last year at a conference I heard from some of my former academic colleagues that they were using this description to educate their students abput where they were lacking in knowledge or experience

Education 60

Education Engineering Technology Technology 60

All Things Distributed

JULY 23, 2007

The august edition of Wired has a long article on the disappearance of Jim Gray, earlier this year. Steve Silberman worked diligently on this article and interviewed many people including Donna Carnes, Jim’s wife. I like the article because it focuses more on Jim as the technologist, friend and mentor than on the technicalities of the search after his disappearance

60

60

All Things Distributed

JULY 6, 2007

I recently gave a few talks in which I gave some reading advice to the audience and I promised to follow-up with posting the links here. The first article is the interview of Michael Stonebraker by Margo Seltzer in the May/June edition of ACM Queue (unfortunately the article is online yet, but this link seems to work thanks to Peter O’Kelly

60

60

All Things Distributed

APRIL 29, 2007

Tori Amos - American Doll Posse – Each of the 5 persona on the album have their personal blogs, more details at the wikipedia page for the album (May 1). Groove Armada - Soundboy Rock – Unpredictable (May 10 US release

60

60

All Things Distributed

FEBRUARY 14, 2007

Today in the opening session of FAST there are two papers on the studies of hard disk reliability. Both these papers present very interesting results that blow away some of the common assumptions in failure modelling of systems. Bianca Schroeder and Garth Gibson from CMU in their paper Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You

Systems 60

Systems 60

All Things Distributed

FEBRUARY 12, 2007

I have had the luxury of almost 4 months with any real conferences. Don’t get me wrong it is not that I do not enjoy the public speaking side of my job; it is just that I experience it always as rather disruptive. It is great to focus for a while and get things done

60

60

All Things Distributed

JANUARY 4, 2007

Each year I try to immerse myself in a particular technology, either as a user or as a developer. This technology has preferably nothing to do with my day-job, or at least not directly. For the past year this was Second life. Today I sold all the SL real-estate that I acquired in the past year and parked my avatar permanently in The Blarney Stone Irish Bar in Dublin

Technology 60

Technology Technology Development 60

The Agile Manager

OCTOBER 28, 2007

An Unhealthy America: The Economic Burden of Chronic Disease -- Charting a New Course to Save Lives and Increase Productivity and Economic Growth October 2007 2 McLaughlin, Katy. The Price of a Four Star Rating The Wall Street Journal, 6-7 October 2007. How Top Watchmakers Intervene in Auctions The Wall Street Journal, 8 October 2007. Apocalypse No The Wall Street Journal, 27-28 October 2007.

Government 40

Government Retail Energy Games 40

The Agile Manager

DECEMBER 28, 2007

Management Leaders Turn Attention to Followers ” The Wall Street Journal, 24 December 2007. With the cost of capital on the rise, the need to focus on returns is much more acute. Unfortunately, IT has not traditionally excelled at maximising returns. Industry surveys consistently show that a third to a half of all IT projects fail outright or significantly exceed their cost estimate.

Retail 40

Retail Best Practices Technology Technology 40

The Agile Manager

AUGUST 23, 2007

How a Company Made Everyone a Team Player ” The Wall Street Journal, Monday 13 August 2007. August 2007. Why Deere is Weeding Out Dealers Even as Farms Boom ” The Wall Street Journal, Tuesday, 14 August 2007. Pharmaceutical companies do not successfully deliver drugs just because they hire a lot of highly skilled researchers in lab coats.

Technology 40

Technology Technology Innovation Latency 40

The Agile Manager

SEPTEMBER 26, 2007

1 Comparing August 2001 to August 2007 monthly averages, the USD declined 28% to the GBP, 34% to the EUR, 28% to the CHF, 37% to the AUD, 31% to the CAD, 13% to the INR, 8% to the CNY. US based IT departments are facing turbulent times.

Airlines 40

Airlines Strategy Metrics Java 40

The Agile Manager

JULY 29, 2007

IBM Institute for Business Value, July 2007. The Wall Street Journal, 23 July 2007. Demand for IT in business continues to rise. Looking backward, over the last 10 years the IT market has absorbed the new capacity in Asia and South America, yet still we find global and national/regional IT employment is up since 2000. 1 Looking forward, all indications are that demand will continue to rise.

Open Source 40

Open Source Latency Innovation Development 40