Mar 13, 2022 | Planned Work
During Monday 14th March our specialist contractors will be replacing the batteries on our UPS in our Huntingdon Data Centre.
While this should have no impact on services – it does pose a slight risk and reduces resilience to services in our Huntingdon Data Centre
We will update as this work progresses
[completed 14:00]
The battery replacement is completed and the UPS and resilience are now back to normal in our Huntingdon Data Centre
Feb 14, 2022 | Planned Work
During the 15th February 2022 engineers will be replacing the fuel on our generator at our Huntingdon Data Centre.
This will not cause any direct issues and we still have our UPS system to maintain power to the Data Centre. It does however mean that in the event of a power failure on the Utility supply there is a risk as for a short period the Generator will not be available to supply load.
Our suppliers will be working hard to ensue that the at risk period is kept to an absolute minimum
NOTE the new fuel will be the more environmentally friendly HVO fuel as part of our commitment to reduce the environmental impact of our company
We will update this post as the work progresses
[Update]
This work completed this morning and the generator is back available as needed
Feb 1, 2022 | Planned Work
Following on from the fibre break on our Huntingdon to Manchester link yesterday our supplier is carrying out further work overnight to replace approx1km of fibre following the damage they found. This work is currently scheduled to run between 11pm this evening until approx 7am on Wednesday.
We are told the link should not be down for the entire duration but will be down for some point of this.
As previously this will not cause a outage on our network but it does reduce the resilience – and therefore create risk for connectivity at our Huntingdon Data Centre and Manchester.
We will update this post as the work starts and as it progresses / completes
Jan 3, 2022 | Planned Work
We have been informed that Virgin Media will be performing a re-route on one of the backhaul links between London and our Huntingdon Data Centre between the 12/1/2022 and 13/1/2022
We have resilient links to provide service here however this will reduce the resilience for services in Huntingdon and also reduces resilience to Manchester
The planned work has a 10 hour window hpwever we don’t anticipate the link being down that long.
We will update this post if we see any impact and once the line is restored.
Dec 23, 2021 | Information, Planned Work
We are seeing some VLANs on our core network flap when they pass through a core switch in Telehouse West.
We may need to reset or reload this switch to clear this issue. This MAY cause some instability to the network for a short period and services hosted in Telehouse West may drop tor 15-20 minutes
We will update this as investigations continue
[update 1 – 23:30]
We have reloaded one of our core Juniper switches and are monitoring for stability.
[Update 24/12 9:44]
We have monitored the network after the switch reload at 11pm last night – and this has stayed stable with no further flaps. While the switch had no error logs during the incident it was randomly blocking traffic on some VLANs including a backup vlan between two core routers in Telehouse East and North. This was causing some packet loss / drops as the traffic was switching between links. We apologise for the issues seen however while we are aware this issue started during the day we needed to perform the reboot out of hours since it caused a 10-15 minute outage to services directly connected.
Oct 27, 2021 | Planned Work
We have an issue with the above router that is causing some issues on our monitoring – and with a couple of internal BGP sessions being down.
We are planning to upgrade and resolve this during a maintenance this evening.
Given the redundancy we now have we do not anticipate this work being noticeable – however it does reduce slightly the resilience within our network while the work is performed. We intend to start this work approx 9pm and it should be completed within approx 1 hour – thought we have a window until midnight just in case.
We will update this post once the work completes
