Sunday, October 11, 2009

We Don't Need No Steenking Backup!

Let's all welcome Microsoft and T-Mobile's Sidekick service as the latest Database Deadpool winners!

Here's the reported time line for the T-Mobile disaster:

Friday, October 2, 2009

Microsoft, whose Danger subsidiary powers the [T-Mobile] Sidekick service, said it is "working urgently" to deal with the problems, which interrupted data service for most Sidekick users starting at 1:30 a.m. PDT on Friday.
Major outage hits T-Mobile Sidekick users October 6, 2009 11:57 AM PDT by Ina Fried
We are working to resolve the issue and hope to have things restored to normal soon.
Sidekick Service Issue (Oct 2nd) 10-02-2009, 08:40 PM

Sunday, October 4, 2009

...it seems that Danger's data center has been on the fritz for a solid two days now with symptoms ranging from text message weirdness to dead address books and everything in between, up to and including a completely unusable experience. Frantic calls to T-Mobile are resulting in comped bills for some subscribers, so if you're affected, you might want to give 'em a ring.
Danger's server woes leave Sidekicks in the lurch by Chris Ziegler posted Oct 4th 2009 at 4:48PM

Monday, October 5, 2009

Users of T-Mobile's Sidekick smartphones spent much of the weekend without data service, and many are still waiting for service to be restored.
T-Mobile: Danger Working to Resolve Sidekick Data Outage Monday October 5, 2009

Thursday October 8, 2009

We sincerely apologize for the inconvenience, and appreciate your patience as Danger works hard, around-the-clock to resolve this issue. T-Mobile will continue to keep you updated as we learn more. Thank you!
Sidekick Service Issue 10-08-2009, 04:22 PM

Saturday, October 10, 2009

The massive data failure at Microsoft's Danger subsidiary threatens to put a dark cloud over the company's broader "software plus services" strategy.
Sidekick outage casts cloud over Microsoft October 10, 2009 3:26 PM PDT by Ina Fried
...this is shaping up to be one of the biggest disasters in the history of cloud computing.
T-Mobile: we probably lost all your Sidekick data by Chris Ziegler posted Oct 10th 2009 at 3:45PM
Perez Hilton loses 2000 contacts in his Sidekick. 2000 people to meet in Griffith Park for biggest group hug ever.
John Mayer
Regrettably, based on Microsoft/Danger's latest recovery assessment of their systems, we must now inform you that personal information stored on your device - such as contacts, calendar entries, to-do lists or photos - that is no longer on your Sidekick almost certainly has been lost as a result of a server failure at Microsoft/Danger.
T-MOBILE AND MICROSOFT/DANGER STATUS UPDATE ON SIDEKICK DATA DISRUPTIONUpdated: 10/10/2009 12:35 PM PDT
This is a code red cloud disaster.
Cloud Goes Boom, T-Mo Sidekick Users Lose All Data Posted by Eric Zeman, Oct 10, 2009 08:22 PM
This latest large-scale publicized data loss will surely lead to managers everywhere forwarding a link to the story to their IT departments asking "what are we doing so that this doesn't happen to us." ...
If you read almost any technology website or newspaper, you could be forgiven for thinking that "The Cloud" solves everything.
...
In the Danger case, it appears from initial speculation that the data was lost because they attempted to upgrade a storage array without backing it up first. Here is a case of smart and rational people who do this for a living at one of the best companies in the world, and they didn't even bother making a backup..
Letting Data Die A Natural Death Nik Cubrilovic TechCrunch.com Saturday, October 10, 2009; 8:20 PM

Sunday, October 11, 2009

Microsoft/Danger has stated that they cannot recover the data but are still trying.
Server Failure Destroys Sidekick Users' Backup Data Posted by timothy on Sunday October 11, @05:29AM

4 comments:

tom s. said...

I was just wondering whether to send you an email about this yesterday, but obviously there was no need. I hadn't see the timeline presented so compactly.

A few more of these and you could collect them as horror stories to keep IT people awake at night.

Breck Carter said...

Tom: Thanks... you *should* have sent me an email, I was blissfully unaware of this story until today (Sunday October 11). This is way bigger than ma.gnolia, for several reasons: T-Mobile and Microsoft make money from Sidekick, Sidekick has been around for years and years, Microsoft is in the business of selling database and backup solutions, Microsoft is awash in technical folk who know better, and so on... let the slagging begin!

edwin sanchez said...

I never liked the Sidekick anyway, I was always a Blackberry owner...very faithful...like the rest of the world.

tom s. said...

Is this another one?
http://www.theregister.co.uk/2009/10/19/swissdisk_failure/