Manchester Upgrade 16/06 – 17/06 – completed

Continuing the recent network upgrades on our network the core routers and switches in Manchester are due to be upgraded over the weekend to our new Juniper core.

During the upgrade process there will be breaks in access to some of our backup transit connections in Manchester and a number of leased lines connected in Manchester. All leased line connections in Manchester are backup links so we do not expect and connections to customers to go down – However there will be a lack if resilience while this upgrade is in place. We will upgrade this notification as the work progresses

[Update] This work is completed and all services at Manchester have been updated. There was an unplanned issue routing to a small number of locations during this work but this was identified and resolved. If you are seeing any issues please contact support via the normal routes

Network Migration – Telehouse North 30/03/2018

Further to the recent posts – we are planning the final Network upgrades in our core network later today. This will decommission the last of our old network in London. This involves the move of our final leased line bearers and the final Broadband Interconnects.

For Broadband Interconnects you may see your session drop – However we have alternative Interconnects to all wholesale providers in our other PoPs so these should re-connect almost instantly on an alternative link.

Leased Lines terminating in Telehouse will have an outage of at maximum 30 minutes while they are moved between the old and new network. In many cases the outages will be much shorter than this

We will update this post as the work progresses

 

 

Network Migration – Ongoing

During the next few weeks we will be migrating our core network onto new, faster Juniper router and switches. The new routers and switches are already in place in our data centres and configured. The current work involves moving links in a controlled manner between the old and new network . The majority of this i moving peering and transit connections one by one. This will not cause any outages – just subtle routing changes as the traffic moves away from a transit – the link is moved and the transit (or peering re-appears via the new network.

Once this is completed there will be some brief blips on leased lines and broadband connections as they are moved to the new network – these will be communicated as these are planed – most outages will be less than a couple of minutes in these cases as the cables are moved.

The end result will be a more modern network with far greater capacity for growth

We will update this site as the plans progress

 

 

Planned Work – HEX Data Centre 11/03/2018 – Completed

As part of the upgrades and improvements to our network we are moving our connectivity in this data centre to a new rack with dual power feeds and additional cooling.

We already have a new router and switch in this rack – However subject to resources we will be moving some of our hardware and links between the two racks this coming Sunday. The work will start in the late afternoon and into the evening.

During this time various parts of our network and services connected directly to this network PoP may experience a period of downtime. Traffic may also take alternative routes to some locations and there’s a chance of some congestion for some traffic.

This PoP currently hosts

– some Transit and peering links which will simply take alternative routes

– One of our DNS servers (the other servers will work fine)

– Some TalkTalk based ADSL traffic – in these cases PPP sessions will drop BUT should auto re-connect on the Telehouse link

– Approx 21 leased line circuits that will be contacted directly

We will confirm the date once we have confirmation of resources from the data centre provider to help move some fibre cross conencts

We will keep the downtime to a minimum and we will update this post as much as we can

Any questions please email support in the normal way

 

[update]

This work is completed. All services in HEX are now in our new rack and core network

RESOLVED: Power Issue – Harbour Exchange Square

We have become aware of another power issue in Harbour Exchange Square. This occurred this afternoon while no work was being performed by ourselves.

We have raised this to the Data Centre (Equinix) requesting an urgent update. We also have an engineer en route to the site to assist as needed.

Most broadband and leased lines should be working although there will be issues for any service ONLY connected at HEX.

More as soon as we have an update

update: 6:30pm The core issue is now resolved and we are seeing all services in HEX back up and running. We are continuing to check for any remaining issues. There was a issue earlier affecting new FTTC/ADSL logins that was resolved. If you are still seeing an issue please call or email support via the normal routes.

We are continuing to work with Equinix to understand the root cause of the issue

Routing instability

We have seen some routing issues on our Telehouse North core today affecting circuits terminated in this location or traffic passing through thn-gw1 or thn-gw2.

The primary issue has now been resolved and all traffic should be routing correctly. We anticipate some follow up work and we will communicate this as soon as it is planned

We apologise for the issues that some customers may have seen

[update] As a side effect of this with the router reload – it seems that one of the core routers decided to load a OLD version of the config which cased an issue for a number of directly connected customers. The config has now been restored to the latest version and we believe routing is now stable and correct