Nobody was injured by the explosion, which occurred at around 1.50pm local time and reportedly caused six-foot-high flames to shoot out of a manhole cover on Mission Street’s 500 block.
But the ensuing power outages affected large areas of San Francisco’s east side, including the collocation data center at 365 Main, which hosts a number of well-known web services.
Apparent problems with 365 Main’s backup power supplies meant that sites hosted there went down, while sites hosted elsewhere in the affected neighborhoods stayed up
The virtual world Second Life, which allows users to buy and sell pretend goods, experienced problems with login, payments and other in-world issues, according to game maker Linden Research. It took four hours to get the service working properly.
Thousands of bloggers using Six Apart’s LiveJournal, Vox and Typepad services were also among those finding they were unable to access their sites. The knock-on effect of this was that the national daily newspaper USA Today, which hosts its blogs with Typepad, was hit by outages too.
Craigslist, the popular classified ads site that acts as the main artery of person-to-person commerce in the city, the blog tracker Technorati, the reviews site Yelp! and the text link advertising firm AdBrite, were all also affected by periods of downtime that lasted well into the evening.
ZDNet, a news and blogging site owned by CNet, also said it had suffered some downtime after the a problem with the backup power corrupted some databases when the PG&E outage hit.
Press spokespeople at the local utility, Pacific Gas & Electricity, did not answer our calls yesterday afternoon, but it was reported by the AP and Reuters that up to 51,000 customers in the city were without electricity early in the afternoon.
365 Main released this statement at 4pm local time: At 1:45 pm today, there was a major power event in San Francisco that impacted business operations for many San Francisco based companies, including 365 Main’s San Francisco data center.
Some customers within the 365 Main facility were temporarily effected by the utility failure, the statement said. The building is currently 100% operational and running on back-up power (generators) until the company can confirm that utility power is stable.
At that time, some of the affected sites appeared to be having erratic availability, though whether that was due to the power outage could not be ascertained.
Requests for further information from 365 Main were not responded to by press time.
Ironically, 365 Main had published a press release just hours earlier boasting that it had supplied one customer, RedEnvelope, with two years of 100-percent uptime.
The web site at RedEnvelope.com could not accessed by this reporter at 4pm Pacific yesterday. A spokesperson for the gift retailer did not return a call for comment.
A customer of 360 Spear, a rival collocation facility situated on the same block as 365 Main, near the city’s famous Embarcadero bay-front boulevard, told us his systems there had been functional all afternoon.
365 Main’s web site says the data center uses two 2.1 megawatt Hitec Continuous Power System generators, which run 24 hours a day.
According to a joint press release with PG&E, 365 Main qualified for environmental discounts by using a more efficient testing procedure on those generators, reducing power consumption, earlier this year.
Our View
It’s obviously good news that nobody was hurt in the explosion (unlike a similar PG&E explosion a few blocks away on Kearny Street two years ago, which severely injured an unfortunate passer-by).
The outlook for 365 Main’s image is less pleasant, however. Like most collocation facilities, 365 Main prides itself on its theoretically uninterruptable power supplies, which are supposed to prevent this kind of downtime.
So the outage, even it was for just a few hours, is highly embarrassing for the company. That’s not to mention the potential financial impact of failing to meet service level agreements.
Competitors are likely to quickly capitalize on these events, if not by stealing customers outright, then at least by offering them second-site backup services.
We believe companies affected by outages yesterday, particularly those where downtime directly equates to lost revenue, would be sensible to investigate the cost case for backup data centers too, if they have not done so already.
That is particularly true if their only data center is in earthquake-prone San Francisco, a city that, let’s face it, could sink into the Pacific with nary a moment’s notice.
