Recounting EC2 One Year Later

It’s been over a year since my original Anatomy of an EC2 Resource ID post. In what became my little claim to fame in the industry, I uncovered the pattern behind those cryptic IDs AWS assigns to every object allocated (such as an instance, EBS volume, etc.). The discovery revealed that underlying the IDs is a regular serial number that increases with each resource allocated. While this may sound technical and insignificant, it turned out to be very valuable: it enabled, for the first time, a glimpse into the magnitude of Amazon’s cloud.

The numbers gathered in that post back in September 2009 showed that approximately 50,000 instances were being spun up every day on EC2 (in the us-east-1 region). So what’s happened since? I joined forces with CloudKick, providers of a cloud management and monitoring platform, to dig up more data. Here’s what we found: (click to expand to an interactive chart)

The chart above plots the number of instances launched per day, from mid-2007 till present day. Growth is, well, unmistakable. A couple of peaks dominate the landscape around February and October 2010, peaks which somewhat correlate to availability of new AWS services (see interactive map). The evidence is highly circumstantial though as I find it hard to draw a direct conclusion on how these specific events pushed the daily launch count as high up as 150,000.

Let’s zoom out now and look at EC2′s growth over the years:

For this chart, we averaged out the instance launch counts over each year (data for 2007 and 2010 may be partial). Based on the results, activity on EC2 has been multiplying several times over every year. The biggest step was 2008-2009, exhibiting 375% growth (that’s almost 5X). 2009 was the year AWS really exploded, but what strikes me as odd is that growth actually slowed down the following year (to 121%). One theory would be that the cloud has begun to saturate the early adopters and it is truly time to cross the chasm. Crossing, however, is turning out to be a difficult feat.

Responding to my previous research, a top Amazon official commented that a count of instance launches doesn’t really reflect anything meaningful (like the actual customer base, server count or revenues – all of which we’d all love to figure out). I respond that it’s examining the numbers one year later that provides the real value: it’s like looking at a mysterious dial on your car’s dashboard: even without understanding the exact parameter measured, if it shoots up then there’s a decent chance you’re driving faster.