Due to a warning on one of our Broadband Routers in Telehouse West we may been to upgrade this and reboot it later this evening.
If your session is affected your session is affected you should fine you re-connect within a few minutes
If for any reason the service continues to be down – please then reboot your router
We will update this after the work completes – Sorry for any issues
We are seeing a minor error on one of our switches in HEX for the past few weeks. We have tried a number of non service affecting fixes – which have unfortunately not cleared the issue.
The warnings at this point are not service affecting – however we want to clear this before it becomes a issue.
We are in the data centre for some other work tomorrow – and if time allows we will run some checks while we have an engineer there.
There is a risk we may need to power cycle the switch to try and clear the issue. Depending on what we find we may perform this tomorrow or we may schedule this another time.
There is no core routing via this switch however there are links to some leased line and broadband connections on the switch – IF the switch is rebooted most of these will see a brief drop while the sessions move to alternative links
We will update this as we know more
[update]
We performed some initial work here but will re-plan the re-boot / replacement for another date.
One of our wholesale partners are upgrading the switch that one of our feeds is connected to over night on the 21st November
We will gracefully move sessions to the second NNI before the work starts and re-enable the affected link after their work.
Customers with sessions on the affected link may see their connection drop briefly when this work takes place
Our provider for one of our core links between London and Manchester has a fibre break meaning that our core link between London and Manchester is currently down.
This is not currently causing any issue but it does mean that we have a lack of resilience to both the Manchester and Huntingdon sites.
Our supplier have identified the fibre break location caused by a digger and are working to repair the damage currently.
We will update this as we know more
[resolved[
The fibre damage was resolved yesterday and the link has remained stable since
During the day of 20th July we will have specialist engineers working on our Generator system at our Huntingdon Data Centre.
This should not impact services at all as as even in the case of a power failure we have a UPS system. However there is always a slight risk and we will have slightly lower redundancy during the work
We will update this as the work progresses and completes
During the day of 20th July we will have specialist engineers working on our UPS system at our Huntingdon Data Centre.
This should not impact services at all as we have a N+1 redundant system. However there is always a slight risk and we will have slightly lower redundancy during the work
We will update this as the work progresses and completes
Our router vendor used for customer leased line routers has released a number of firmware updates. Over these three evenings we will upgrade the firmare on most customer routers. Each router will see aprrox 5 minutes downtime while the router reboots after the upgrade.
We will confirm as the work progresses and completes
We are planning a small upgrade to our links between Telehouse North and Telehouse West and Telehouse Noth and harbour Exchange quare on the evening of Monday 17th July
This will leave the sites at risk and some potential very brief blips in connectivity as the fibres on these buildings are moved over to our new DWDM platform.
We will update this post once the work completes
[completed]
This work completed as planned with minimal impact – the DWDM fibre links are now working as expeted
We saw loss of connectivity to some of the servers in Huntingdon between 3pm and 4pm this afternoon. This was due to an issue with one power feed to a core rack.
Most of our new network is now dual fed- however some of the legacy network on the older Cisco switches is single fed. We saw one switch down during the issue.
Our engineers attended site and resolved the issue – we are performing further checks and will continue the replace the older switches as quickly as we can.
We apologise for any issues
Our engineers along with Telehouse will be working to upgrade the Power Supply to our rack in Telehouse West this evening to allow expansion and additional equipment to be installed. The rack had two resiliant feeds and all core equipment is connected to both feeds.
During the work one Power bar will be replaced at a time – reducing meaning there will for a short period be no resilience on the power in the rack. We do not anticipate any impact but the service should be considered at risk
We also have a fibre link between Telehouse West and Telehouse North which is showing some low level errors. Last week Telehouse tested the fibre which passed – while on site we will replace the optics on this link to see if this clears the issue – if not we will ask Telehouse to run further checks on this fibre. While this is being done again there will be a brief loss of resilience on our fibre ring but we do not expect any customer impact
We will update this if anything changes – and on completion of the work
[completed]
This work completed on Friday evening – we did see two issues through this work. The router that is usd for some OFNL connections for Air customers had an issue for about an hour due to an unplanned powercycle and not re-starting correctly. A few fibre NNIs also bounced. This also affected one of our Name servers. We are sorry for any service disruption from this.