No QA. No Backups. No Sleep

All Things Distributed

Amazon Hackday 2007 in progress. Midnight in Seattle. World-wide

78

Breaking through Physical Boundaries

All Things Distributed

The emotions about reading books in digital form and the Amazon Kindle are running high, already before the device was released. For me there are two features that sold me on the device: the networked content push and the content interaction.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Tribute to Honor Jim Gray

All Things Distributed

On May 31 2008 a tribute will be held at UC Berkeley to honor Jim Gray, who went missing during a solo sailing trip in January of this year. Although Jim is listed as missing, and will be until 2011, a Tribute be held to honor him before too much time has passed

78

Eventually Consistent

All Things Distributed

I wrote a first version of this posting on consistency models in December 2007, but I was never happy with it as it was written in haste and the topic is important enough to receive a more thorough treatment. ACM Queue asked me to revise it for use in their magazine and I took the opportunity to improve the article

60

Amazon Widgets

All Things Distributed

A popular request has been to make more of the Amazon product data available as easy widgets for reuse on blogging, social network and associates websites.

Potter Delivered

All Things Distributed

2.223 million pre-orders on our sites world-wide. million on Amazom.com alone. These orders trickled in over the period of 5.5 months, but from a distributed systems perspective today is the day as these orders go en-masse from pre-orders to orders, being charged and delivered.

GMR

All Things Distributed

The 2007 Nobel prize in Physics has been awarded to Albert Fert and Peter Grünberg for independently discovering Giant Magnetoresistance in 1988. Their work had a tremendous impact on the computer industry as it revolutionized the way we could store and retrieve information on hard disks. It was the first major application of nanotechnology and allowed hard disks to shrink from the size of a large washing machine to the device that fits in an mp3 player

60

Dr. Swami

All Things Distributed

Today at noon Swami made the transition to Dr. Swaminathan Sivasubramanian at the Vrije Universiteit in Amsterdam. In most European countries the doctoral defense (promotie in Dutch) goes with a lot of protocol and pomp & circumstance.

Code 78

Myths of Innovation

All Things Distributed

Congrats to Scott Berkun for sending his "Myths of Innovation"book to the printer. Scott has collected an excellent set of recommendations for his book; among others JSB, Guy and me. It is well deserved; I read a beta version of the book and I was impressed.

The Search for Jim Gray Continues

All Things Distributed

As Mike Olsen just wrote on the Tenacious Search weblog, today was the first day that action could be taken on the boats found in the satellite and ER-2 streams. Bad weather has kept any aerial search parties on the ground until now.

78

Half a Million Assignments Completed.

All Things Distributed

Over 530,000 Mechanical Turk assignments have been completed by more than 12,000 volunteers in the search for Jim Gray. We need a little more of a push and then all the images will have been processed.

Turkers Working Hard on the Search for Jim Gray

All Things Distributed

It is now 3 PM on Sunday afternoon and the group of volunteers in thethe search for Jim Gray has worked their way through almost 100,000 assignments since Friday 5 PM. Since then we have seen over 6000 individual workers completing anywhere from 1 to almost a 1000 assignments.

78

High Altitude Search for Jim Gray

All Things Distributed

We have now added the data captured by the NASA ER-2 plane yesterday over the ocean area outside of San Francisco. We were very fortunate that this flight was scheduled for yesterday and that the NASA folks were interested in having it capture these images.

78

Help Find Jim Gray

All Things Distributed

Computer science icon Jim Gray mysteriously disappeared after a solo trip with his sail boat outside San Francisco Bay. The coast guard has been searching for 4 days but has not been able to locate anything, not even debris.

78

Jim Gray Missing at Sea

All Things Distributed

My long time friend and mentor Jim Gray took his sail boat out for a trip to the Farallon Island yesterday and has not been heard from since. The coast guard has been searching since last night. Jim is an extremely experienced sailor with more than 40 years experience. This is very worrisome

78

Amazon's Dynamo

All Things Distributed

In two weeks we’ll present a paper on the Dynamo technology at SOSP, the prestigious biannual Operating Systems conference. Dynamo is internal technology developed at Amazon to address the need for an incrementally scalable, highly-available key-value storage system. The technology is designed to give its users the ability to trade-off cost, consistency, durability and performance, while maintaining high-availability

Heâ??e nalu

All Things Distributed

North Shore. 10 Days. See you next year

60

Marc

All Things Distributed

Sadness. Marc Orchant, the ulimate optimist, is no longer with us

60

And the Winner is.

All Things Distributed

Today we had a wonderful day with the finalists of the 900 contestants in the AWS Startup Challenge, all seven proved themselves worthy of winning the ultimate price. The finalists were: Brainscape Commerce360 Justin.tv Milemeter Ooyala WeoGeo Go to the startup challenge vote page to see videos of each of these companies and to see the voting results

AWS 60

Amazon S3 in Europe

All Things Distributed

I am about to go on stage at Web 2.0 Expo in Berlin where we will announce the ability for Amazon S3 customers to store their objects in Amazon’s European storage cloud. This has been a frequently requested feature by our European customers for various reasons, better latency control being the most important one

Max, Min & Fair

All Things Distributed

If we would just forget about discriminatory traffic management (e.g. based on deep packet inspection) life on the network would be pretty simple, even under overload conditions. A lot has been written about this already but Wes really nails it in his summary. Maximized profit, minimized frustration, everybody’s happy

HBR - The Institutional Yes

All Things Distributed

The Institutional Yes is a Harvard Business Review interview with Jeff Bezos about the way strategies are developed at Amazon. I have written before about how the relentless customer focus translates into driving architecture and design using the “working backwards” approach. The interview with Jeff gives you more insight on the impact of customer focus on overall strategy and how it drives a culture of experimentation

Clarifying Internal-only

All Things Distributed

There is a lot of positive feedback about the Dynamo paper but I noticed that something I wrote in introducing the paper is being misunderstood. This was my fault, I wrote it too strongly. What I meant by internal-only is that Dynamo is not directly exposed externally. However, Dynamo and similar Amazon technologies are used to power parts of our Amazon Web Services, such as S3

Steve's Back

All Things Distributed

My old friend Steve Vinoski is back online. Steve was with Iona for many, many years, working as the main architect on many of their Middleware technologies. Steve left Iona this spring to work for a start-up in the Boston area. But Steve became really famous for this drawing, which he used to express his opinion about my keynote at Middleware 2004

Chat

All Things Distributed

As a follow-up to the keynote I gave at the Next Generation Datacenter Conference the folks at Network World organized an online chat, to happen for an hour tomorrow (October 1) from 11-12 PST. I have never done something like this before so it will be fun to see how good my typing skills are, but at least I get to sneak out of a very long meeting for an hour

The Final Blade Runner

All Things Distributed

I just finished reading the Wired interview with Ridley Scott about Blade Runner, the ultimate film noir. The movie is certainly one of my favorites and I own several editions of it. Of course I now also want this “final cut version”, hoping that it would be available in HD. It turns out it being produced in 7 different packagings, all to be released on December 18 (which is pretty late for online holiday shoppers in my eyes

50 X

All Things Distributed

In the recent years Mike Stonebraker has been advocating that the current commercial databases have become one-size-fits-all tools that are so general and heavy-weight that they do not excel in any particular area. Mike has written some papers on this topic of comparing general databases with specialized storage engines in areas of data-warehousing, text search, stream processing and scientific processing

Amazon MP3

All Things Distributed

The beta of the long awaited DRM free Amazon music store is ready for use. Go download

60

Mechanical Turk and the Search for Steve Fossett

All Things Distributed

Since Saturday morning you can help search for Steve Fossett through Amazon's Mechanical Turk. Click here to get to the start page and then press the "Work on HITs" button to start reviewing satellite images. As with the search for Jim Gray, this is only possible because of the collaboration of several organizations

60

The Different CTO Roles

All Things Distributed

I was putting together a short panel presentation on the role of a Chief Technology Officer in corporate innovation and I once again realized that there is quite a bit of confusion around the role of the CTO. The first thing that always comes up when you want to discuss the role of a CTO is that there is no well established definition of what a CTO actually does

The Amazon Flexible Payments Service (Amazon FPS)

All Things Distributed

Today Amazon AWS launched a limited public beta of Amazon FPS, the Amazon Flexible Payment Service. Amazon FPS is a payment service that is 100% focused on the needs of developers. Traditionally, developers have been limited in how they can manage payments.

AWS 60

Job Opening for a Senior Research Engineer

All Things Distributed

When I was building one of my first teams at Amazon, one that had to work on some really advanced distributed systems technology I put up a job description on this weblog. I was certainly pleased with the responses. Last year at a conference I heard from some of my former academic colleagues that they were using this description to educate their students abput where they were lacking in knowledge or experience

Wired on Jim Gray

All Things Distributed

The august edition of Wired has a long article on the disappearance of Jim Gray, earlier this year. Steve Silberman worked diligently on this article and interviewed many people including Donna Carnes, Jim’s wife. I like the article because it focuses more on Jim as the technologist, friend and mentor than on the technicalities of the search after his disappearance

60

Reading References

All Things Distributed

I recently gave a few talks in which I gave some reading advice to the audience and I promised to follow-up with posting the links here. The first article is the interview of Michael Stonebraker by Margo Seltzer in the May/June edition of ACM Queue (unfortunately the article is online yet, but this link seems to work thanks to Peter O’Kelly

60

New in May

All Things Distributed

Tori Amos - American Doll Posse – Each of the 5 persona on the album have their personal blogs, more details at the wikipedia page for the album (May 1). Groove Armada - Soundboy Rock – Unpredictable (May 10 US release

60

On the Reliability of Hard Disks

All Things Distributed

Today in the opening session of FAST there are two papers on the studies of hard disk reliability. Both these papers present very interesting results that blow away some of the common assumptions in failure modelling of systems. Bianca Schroeder and Garth Gibson from CMU in their paper Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You

The Conference Season is Opening Up Again

All Things Distributed

I have had the luxury of almost 4 months with any real conferences. Don’t get me wrong it is not that I do not enjoy the public speaking side of my job; it is just that I experience it always as rather disruptive. It is great to focus for a while and get things done

60

SL -> WoW

All Things Distributed

Each year I try to immerse myself in a particular technology, either as a user or as a developer. This technology has preferably nothing to do with my day-job, or at least not directly. For the past year this was Second life. Today I sold all the SL real-estate that I acquired in the past year and parked my avatar permanently in The Blarney Stone Irish Bar in Dublin

IT Governance Maximises IT Returns

The Agile Manager

An Unhealthy America: The Economic Burden of Chronic Disease -- Charting a New Course to Save Lives and Increase Productivity and Economic Growth October 2007 2 McLaughlin, Katy. The Price of a Four Star Rating The Wall Street Journal, 6-7 October 2007. How Top Watchmakers Intervene in Auctions The Wall Street Journal, 8 October 2007. Apocalypse No The Wall Street Journal, 27-28 October 2007.

Mitigating Capability Risk

The Agile Manager

Management Leaders Turn Attention to Followers ” The Wall Street Journal, 24 December 2007. With the cost of capital on the rise, the need to focus on returns is much more acute. Unfortunately, IT has not traditionally excelled at maximising returns. Industry surveys consistently show that a third to a half of all IT projects fail outright or significantly exceed their cost estimate.

Good Management Can Work Miracles

The Agile Manager

How a Company Made Everyone a Team Player ” The Wall Street Journal, Monday 13 August 2007. August 2007. Why Deere is Weeding Out Dealers Even as Farms Boom ” The Wall Street Journal, Tuesday, 14 August 2007. Pharmaceutical companies do not successfully deliver drugs just because they hire a lot of highly skilled researchers in lab coats.

Investing in Strategic Capability versus Buying Tactical Capacity

The Agile Manager

1 Comparing August 2001 to August 2007 monthly averages, the USD declined 28% to the GBP, 34% to the EUR, 28% to the CHF, 37% to the AUD, 31% to the CAD, 13% to the INR, 8% to the CNY. US based IT departments are facing turbulent times.

Alpha Returns Require an Alpha IT Capability

The Agile Manager

IBM Institute for Business Value, July 2007. The Wall Street Journal, 23 July 2007. Demand for IT in business continues to rise. Looking backward, over the last 10 years the IT market has absorbed the new capacity in Asia and South America, yet still we find global and national/regional IT employment is up since 2000. 1 Looking forward, all indications are that demand will continue to rise.