RackSpace Data Center Goes Dark After Accident

The internet may seem omnipresent and infallible, but all it took was a single truck in Texas to take a chunk of it down. Much like the incident at 365 Main that brought down our site, one of RackSpace's data centers in San Antonio went dark last night after a truck careened into a transformer. When the company's backup power system failed to deploy properly, it was official -- sites like 37Signals, GigaOm, and Laughing Squid were down for the count for almost three hours. Here's the explanation from RackSpace:

In the second incident at approximately 6:30 PM CST Monday, a vehicle struck and brought down the transformer feeding power to the DFW data center. It immediately disrupted power to the entire data center and our emergency generators kicked in and operated as intended. When we transferred power to our secondary utility power system, the data center's chilling units were cycled back up. At this time, however, the utility provider shut down power in order to allow emergency rescue teams safe access to the accident victim. This repeated cycling of the chillers resulted in increasing temperatures within the data center. As a precautionary measure we decided to take some customers' servers offline. These servers are now back up, as are the chillers.

The issue has since been resolved, last night's incident was apparently the third power-related hiccup the data center experienced since the weekend. The onset of the accident was definitely unforeseen (like the situation last July at 365 Main), but one has to wonder if these data centers are really performing equipment readiness tests to ensure that these mishaps aren't repeated.

[Via Laughing Squid]

Photo: deanj