—————————–
Update 27/11/2009
Please find official RFO document at the following URL: http://noc.c4l.co.uk/OutageReport-TATA-Thursday26-11-2009.pdf
—————————–
We have lost contact to both our network devices in Stratford. At the moment, we are investigating the cause of this with the datacentre.
We will update here when we have more information
Update 17:45
We have learned from various sources that all three floors in the TATA datacentre have suffered similar problems, not just C4L. We have no RFO at this time.
Update 18:30
This appears to be a Stratford wide power outage. We are unsure as to the status of the UPS and the generators, they are clearly not operational. We will post further news as we get it.
Update 18:55
Although we are yet to receive any official confirmation, we have learnt that power restoration may take up to 30 – 60 minutes. We will update if we get a more accurate time to restoration or any offiicial outage report.
Update 19:25
We have heard form various sources that power is being restored gradually to the building. Our devices are yet to come back online, we will keep you informed.
Update 19:28
Our devices and connectivity have now been restored.
Update 19:40
We can confirm now that power and connectivity has been restored. We will now be contacting all customers at this facility to confirm all is OK with their kit. We do not have an official RFO at this time but we will be emailing all customers tomorrow am. If you need any equipment looking at please drop us a ticket at https://support.c4l.co.uk and we will get round to it as soon as we can. We have engineers on site to assist.
Update 20:15
We have received a preliminary outage report from our suite owner:
The TATA Stratford facility suffered a power failure today. The generator systems on this occasion did not start correctly.
We apologise for any inconvenience you have been caused and will contact all clients with more detailed information within the next 24 hours.
Update 21:50
We have been informed that in order to get the power back up earlier a temporary fix was put in place. A power reset will have to be informed to get the site back onto its N+1 configuration. We have been told there will be no need to power down equipment during this, however we must inform customers thatfor the next 8 hours all customers should consider themselves to be *AT RISK* should there be another grid failure.