Following an issue located during testing today there will be some work carried out to resolve a water leak on our generator.
An inspection will take place approx 8:30am Friday and once the leak has been localised a plan of action will be made to resolve the issue.
This work will not affect power to the data centre however there may be short periods where the generator in unable to take the load.
We have been advised that one of our suppliers will be performing some work on one of the links between London and Avro court leading to a 30 minutes outage on that link. As there are three diverse links to Avro Court this should not cause any outage here, but may lead to a small disturbance as the network falls over to the resilient paths.
The orks are planned between 00:01 and 06:00 on the 12th June
This work is now completed
One of our transit Providers will be making a small change to their connection in the early hours of 3rd June. We have multiple Transit providers so this *should* have no impact on our service as traffic will be moved gracefully away from this provider before the work happens and then switched back as soon as the work is complete.
We will update this post as the work completes
This work is completed
Our UPS at the Huntingdon Data Centre will be having it’s planned maintenance on 29/05/2014. This is to test and ensure it is performing as it should.
This should have no impact on customers BUT it will involve some test periods on UPS loads and there may be some short periods where the building is running on raw mains rather than on the UPS feed.
With any work on on near the UPS there is a small risk of a problem to services in Huntingdon – However this is very very low.
This work is completed. Unfortunately the engineer tripped one phase of the power during the work. All power is back – we are just checking all servers have returned to service
We will be making two very small network changes today. Neither *should* cause any issues BUT may cause some very short blips
1) We are moving the switch that one of our London to Manchester links is connected to. There are redundant paths BUT you may see a blip for ~1 minute while this link moves as traffic moves to alternative paths
2) Later this evening we will be re-enabling a redundant link in London. This should not have any impact BUT again there is a small risk of a blip as the network re-converges. IF we see any issues we will simply disable the link.
The first change (as it is low risk) will be performed by engineers at the data centre some point today. The second change will happen this evening.
This work is completed – all resilient links are now working in London and you should see a improvement on service and speed from this work. We *may* be some network balancing across the links but we will review this and announce this at a later date
We will be performing some planned works in our Harbour Exchange Data Centre during the evening of Tuesday 13th May.
The work will involve replacing one of our core routers with two new routers and the upgrade on one of the switches in the Rack.
We have already moved traffic away from the router in question so the router change should have no impact. As the switch is replaced there may be some brief interruptions to some services in Avro Court as one of the links is via the switch being upgraded. We hope that any outage will be a matter of minutes as cables are patched to the new switch.
We plan to bring the new router into service overnight if possible – if not this will happen at some point on Wednesday
This work was completed in the early hours of 14/05
We have a UPS Engineer on site running a calibration test on our UPS. This should not affect any power – however there is a very small element of risk with any work on the power to the Data Centre.
We will update this once the work is complete
Following a planned maintenance our Generator at Avro Court requires a replacement water heater. The work to replace this will take place tomorrow morning.
This work will not cause a downtime, however during the work the generator will be unavailable, so in the event of an extended mains failure there is a risk to services here.
We will have full UPS cover throughout the work.
We will update this post as the work is completed
We are seeing a low level of packet loss to (and within) our Avro Court Data Cantre.
We believe this issue is between two of our core switches. We have attempted to resolve this issue by disabling various network ports but the issue remains. The next step is to attempt a reboot on hte affected switches.
We plan to perform this just after midnight tonight (Wednesday morning). This will affect access to some servers hosted in Huntingdon for approx 5 minutes while the switches reboot.
There may be a couple of small issues while traffic re-routes round the switch.
We will update this ticket as the work progresses
There is a Power cut in the area around our Avro Court (Huntingdon) Data Centre.
Power to the data centre is currently being maintained by our UPS and generator on site
UK Power Networks are working to restore the mains power to the area. In the mean time power should be considered at risk, However our UPS and Generator are working currently so there should be no issues.
Mains Power was restored just after 23:30 we are monitoring but have switched back from Generator to mains power for now.