We have a UPS Engineer on site running a calibration test on our UPS. This should not affect any power – however there is a very small element of risk with any work on the power to the Data Centre.
We will update this once the work is complete
Following a planned maintenance our Generator at Avro Court requires a replacement water heater. The work to replace this will take place tomorrow morning.
This work will not cause a downtime, however during the work the generator will be unavailable, so in the event of an extended mains failure there is a risk to services here.
We will have full UPS cover throughout the work.
We will update this post as the work is completed
We are seeing a low level of packet loss to (and within) our Avro Court Data Cantre.
We believe this issue is between two of our core switches. We have attempted to resolve this issue by disabling various network ports but the issue remains. The next step is to attempt a reboot on hte affected switches.
We plan to perform this just after midnight tonight (Wednesday morning). This will affect access to some servers hosted in Huntingdon for approx 5 minutes while the switches reboot.
There may be a couple of small issues while traffic re-routes round the switch.
We will update this ticket as the work progresses
There is a Power cut in the area around our Avro Court (Huntingdon) Data Centre.
Power to the data centre is currently being maintained by our UPS and generator on site
UK Power Networks are working to restore the mains power to the area. In the mean time power should be considered at risk, However our UPS and Generator are working currently so there should be no issues.
Mains Power was restored just after 23:30 we are monitoring but have switched back from Generator to mains power for now.
Further to the replacement of our Primary UPS a few weeks ago some follup up work is required to install a management card and new batteries for the UPS. This is taking place tomorrow (10/12/2013).
During this maintenance there will be periods when the building will be supplied via raw mains and not through the UPS system. While this should cause no outage there is a risk as there is less protection than normal during this time.
We will update this post once the work is complete.
This work is now completed
At approx 4:15pm Friday the UPS in our Huntingdon data centre totally failed. Rather than switching to bypass power (raw mains) as it should have done it turned off all power fed from it.
Our core network in Huntingdon has a separate UPS which stayed on but many servers did power down.
We switched over to raw (unprotected) mains within 10-15 minutes and a majority of servers had their services restored. We did have one rack with a failed network switch which was replaced approx 9pm and a number of servers which needed manually bringing back to service.
Thanks to our UPS service company a replacement 80KVA UPS was installed during the day yesterday (Saturday) and all load was switched back to this restoring service ans resilience to all servers there.
There are a few final jobs to do but these should not be customer affecting. IF you are still seeing any issues please do let us know
Note this outage should NOT have affected ADSL customers as long as you are using the recommended DNS servers as these are not located in Huntingdon – the only services affected *should* have been those that are there.
If you have any questions at all please contact support
We are sorry for anyone affected by this issue
There is a routine maintenance on our UPS systems in Huntingdon this afternoon starting approx 3pm. During this time the buyilding will be running on Generator power.
We do not expect any issues but as with all maintenance there is an element of risk involved.
We are performing this during office hours due to the availability of engineers and spares should they be needed.
We will update you as this progresses
We are aware of an issue affecting C&W / Vodafone based ADSL Connections. Many of these are currently down.
We have reported this and they are aware of an issue in Telehouse North that should be resolved overnight.
While this issue is in place any C&W connections will see sync BUT no Internet connectivity.
This issue was caused by a fibre break and was cleared early Friday morning.
BT have informed us of a known issue on their network affecting some of our ADSL services.
This is affecting many ISPs not just ourselves.
We are awaiting an update with more details.
BT reported this issue closed approx 10pm 31/05/2013 – We can’t see any real impact to our users thankfully.
If you do have any issues please contact support via the normal routes
We are aware of a network issue that occurred this morning. This appears to have been caused by one of our routers crashing.
We do have multiple routers so normally this would not cause a problem – however in this case it would appear that while the router stopped routing some traffic properly it was still advertising the route to our other routers.
We have now reset the failed router which so far seems stable – we are monitoring the router for any signs of issues
We are sorry for any problems caused by this issue.