OUTAGE: Some Virgin Media Based Leased Lines 20/3/2024 Update 2

We are seeing one of our Virgin Media NNIs down currently affecting services that are provided on the affected NNI

This is being raised with Virgin Media currently and we will update this post as we have mode details.

NOTE this only affects leased lines on the one affected link – other services are working.

[Update 13:10]

We have the following update from Virgin Media

We’re aware of an issue impacting your servicesNetwork Teams have identified a number of 10gb High Capacity Services have lost connectivity.An Engineer had already been requested and arrived onsite in Northampton at 12:13 and investigations are underway. All respective support teams have been engaged to progress investigations and create a fix plan.

We will update as we know more

[Update 14:00]

We have seen the affected NNI return to service just before 2pm. We do not have an update from Virgin to confirm that this is fully resolved – but for now we see services  back to normal. We do treak them as ‘at risk’ until the issue is closed down.

Brief issue – Huntingdon 4/6/2023 – RESOLVED

We saw loss of connectivity to some of the servers in Huntingdon between 3pm and 4pm this afternoon. This was due to an issue with one power feed to a core rack.

Most of our new network is now dual fed- however some of the legacy network on the older Cisco switches is single fed. We saw one switch down during the issue.

Our engineers attended site and resolved the issue – we are performing further checks and will continue the replace the older switches as quickly as we can.

We apologise for any issues

Switch Issue – Telehouse West [resolved]

We are aware of an issue affecting one of our core switches in Telehouse West. this is also causing some disturbance in connections on other traffic intermittently.

Our engineers are working on this and we hope to have an update in the next 30 minutes

[resolved 01:30]
On-site engineers have rebooted one of our core switches in Telehouse West ans service is now back. Our network engineers are investigating the cause of this and we will work to ensure there is no further disruption here

UPDATE: Outage on some broadband lines [13/10/2021]

14:46 UPDATE
The supplier advises their issues should now have been resolved, however it may take some time for all circuits to reconnect due to the increased load of subscribers attempting to login to their Radius. Our apologies again for this supplier outage.
13.20 UPDATE
Good afternoon.
The supplier had a power outage at one of their data centres that’s affected their network transit and backhaul and therefore some

lines. We are seeing some lines return but not all and will update here as we get more news.

————-
Good afternoon.
We are aware of an issue within one of our suppliers we use for these lines. At this point we don’t have an announcement or full details but we are aware of this. The issue is also affecting their portal which we access and their status page isn’t working.
We are trying to get an update on this which we will pass on as soon as we can.
Note: this issue is not on the Merula network but within the supplier network. Our apologies to this circuits affected.

Routing Issue – 24/9/2021

We are aware of a routing issue affecting traffic to some destinations currently. The NOC team are aware of this and are working to resolve this – we expect this to be resolved very shortly and will update as we know more

[Update 8:30]
We believe the reachability issue is now resolved – there is some further work taking place to follow up – however this will not have any impact at this time. We will work to bring the failed link back to restore redundancy shortly however this will not have any impact on services at this time.

UPDATE: 14th April 20:45

Once again, please accept our apologies for the problems you’ve seen over the previous couple of days. We realise that this has caused you all serious issues and for that, we’re very sorry.

Various internal changes have been implemented over the last 48 hours and currently, we believe that the network and associated services are now stable and will remain that way. We continue to monitor the situation closely to ensure that our network remains stable and there’s no further impact to your services.

Please email us in the normal way if you have any question or concerns. Thanks again for your support through this incident.