Router Reboot / Upgrades 27 and 28th April 2024 [completed]

We have seen one of our core IBR routers crash this afternoon which affected reachability for a subset of customers

Service is now back however we would like to upgrade and reboot the affected devices later this evening to bring them to the latest code and reduce the risk of any further issues.

There are currently four routers left from the upgrade process mentioned in our previous post – we will aim to complete the upgrade and reboot them later this evening and if necessary tomorrow evening – though we hope to complete the work in one evening.

This should have minimal impact to customers as traffic should just route via other core routers but it is obviously ‘at risk’

[update 28/04 9AM]
All work on this completed overnight and we saw very minimal if any impact to customers. All core routers are upgraded and working as expected. If you see any issues related to this please do contact support via the normal routes

 

Planned Work – Router Upgrade Project [completed]

We are planning to start a rolling project of upgrading our core routers over the next few weeks – this will involve the LNS routers (those that Broadband customers connect to) and our Internet facing border routers.

In the case of the LNS / Broadband routers this will lead to a brief drop and re-connect of your service when the router is rebooted after the update.

In the case of the core routers this should not be visible as in most cases traffic will simply re-route round them and we will remove traffic from the router we are working on before upgrading it

We will update this posy shortly with a schedule and update on the project

[completed]

All upgrades have been completed for this work.

 

 

Planned Work – At Risk – Huntingdon and Manchester 19/3/2024

We have been advised by Virgin Media they will be performing work on a backup link between our Huntingdon and Manchester Data Centre.

The work is scheduled for the early hours of the 19th March.

We do not anticipate any downtime from this but the network does have reduced resilience while this work is being undertake,

 

Planned Word – Router Upgrade / Reboot – 17/1/24

Due to a warning on one of our Broadband Routers in Telehouse West we may been to upgrade this and reboot it later this evening.

If your session is affected your session is affected you should fine you re-connect within a few minutes

If for any reason the service continues to be down – please then reboot your router

We will update this after the work completes – Sorry for any issues

 

At Risk – Switch Maintenance HEX 7/1/24

We are seeing a minor error on one of our switches in HEX for the past few weeks. We have tried a number of non service affecting fixes – which have unfortunately not cleared the issue.

The warnings at this point are not service affecting – however we want to clear this before it becomes a issue.

We are in the data centre for some other work tomorrow – and if time allows we will run some checks while we have an engineer there.

There is a risk we may need to power cycle the switch to try and clear the issue. Depending on what we find we may perform this tomorrow or we may schedule this another time.

There is no core routing via this switch however there are links to some leased line and broadband connections on the switch – IF the switch is rebooted most of these will see a brief drop while the sessions move to alternative links

We will update this as we know more

[update]

We performed some initial work here but will re-plan the re-boot / replacement for another date.

At Risk – Some Broadband Sessions – 21/11/2023

One of our wholesale partners are upgrading the switch that one of our feeds is connected to over night on the 21st November

We will gracefully move sessions to the second NNI before the work starts and re-enable the affected link after their work.

Customers with sessions on the affected link may see their connection drop briefly when this work takes place