Jan 31, 2022 | Information
We are aware of one of the two 10G Link lines to Huntingdon down (this is not the link worked on over the weekend)
We have a case raised to our supplier and are waiting for updates. This is not service affecting but removes resilience for our Huntingdon Data Centre and our Manchester PoP.
We will update this as we know more
[Update 15:30]
Virgin Media have confirmed this is the result of a fibre break in Leicester. This is being worked on and we will update you as we hear. As before this is not currently affecting service but does leave our service at risk.
[Update 19:30[
The link came back approx 17:15 and Virgin have now confirmed they located damaged fibre which has now been re-spliced. Virgin have now closed the fault. We will continue to monitor however we do not anticipate any further impact
Jan 26, 2022 | Uncategorized
We have been informed that Virgin Media will be performing a re-route on one of the backhaul links between London and our Huntingdon Data Centre between the 28/1/2022 and 29/1/2022
We have resilient links to provide service here however this will reduce the resilience for services in Huntingdon and also reduces resilience to Manchester
The planned work has a 10 hour window however we don’t anticipate the link being down that long.
We will update this post if we see any impact and once the line is restored.
Note this is a re-schedule of the earlier announced work
[Update 29/1/2022 9:30]
We saw the link drop approx 12:30am and returned approx 5:30. Failover went as planned and there was no noticeable affect on traffic
Jan 11, 2022 | Uncategorized
We saw a drop in most traffic to transit providers between 19:15 and 19:40 this evening.
This showed as a drop in traffic over that period for any traffic via Transit (connectivity reached over our own peering was fine.The effect would have been the inability to reach certain sites dueing this time.
The issue resolved itself while we started investigating. We believe this issue was caused by an upstream network as opposed to something inside our own network
We are closely monitoring and investigating this issue and working with our upstream providers. As and when we hear back we will update this post
Jan 3, 2022 | Planned Work
We have been informed that Virgin Media will be performing a re-route on one of the backhaul links between London and our Huntingdon Data Centre between the 12/1/2022 and 13/1/2022
We have resilient links to provide service here however this will reduce the resilience for services in Huntingdon and also reduces resilience to Manchester
The planned work has a 10 hour window hpwever we don’t anticipate the link being down that long.
We will update this post if we see any impact and once the line is restored.
Dec 23, 2021 | Information, Planned Work
We are seeing some VLANs on our core network flap when they pass through a core switch in Telehouse West.
We may need to reset or reload this switch to clear this issue. This MAY cause some instability to the network for a short period and services hosted in Telehouse West may drop tor 15-20 minutes
We will update this as investigations continue
[update 1 – 23:30]
We have reloaded one of our core Juniper switches and are monitoring for stability.
[Update 24/12 9:44]
We have monitored the network after the switch reload at 11pm last night – and this has stayed stable with no further flaps. While the switch had no error logs during the incident it was randomly blocking traffic on some VLANs including a backup vlan between two core routers in Telehouse East and North. This was causing some packet loss / drops as the traffic was switching between links. We apologise for the issues seen however while we are aware this issue started during the day we needed to perform the reboot out of hours since it caused a 10-15 minute outage to services directly connected.
Oct 27, 2021 | Planned Work
We have an issue with the above router that is causing some issues on our monitoring – and with a couple of internal BGP sessions being down.
We are planning to upgrade and resolve this during a maintenance this evening.
Given the redundancy we now have we do not anticipate this work being noticeable – however it does reduce slightly the resilience within our network while the work is performed. We intend to start this work approx 9pm and it should be completed within approx 1 hour – thought we have a window until midnight just in case.
We will update this post once the work completes
Oct 19, 2021 | Planned Work
Following on from the packet loss seen earlier today our vendor has suggested an upgrade to be actioned to help resolve the issue we saw.
Given the fault we have decided to start the upgrade process later this evening. This will be approx 9pm – 10pm. As this will be in a controlled manner this should have minimal – if any impact to customer services but there may be brief periods where traffic takes alternative routes to some destinations. While we would prefer to provide more notiece however given the issues seen we would rather ensure this is completed so any follow up work can progress quickly.
We will update this ticket later this evening with an update on the progress
Oct 19, 2021 | Uncategorized
We are aware of an issue that caused a period of packet loss and potentially slow connecticity for services located or Routed via Telehouse East earlier today.
The network is now stable and not showing any issues. We are working with the Data Centre and the NOC team to fully understand the trigger for this period of instability. IF any work is needed following on from this we will update this ticket.
Oct 13, 2021 | Information, Outages, Unplanned downtime
14:46 UPDATE
The supplier advises their issues should now have been resolved, however it may take some time for all circuits to reconnect due to the increased load of subscribers attempting to login to their Radius. Our apologies again for this supplier outage.
13.20 UPDATE
Good afternoon.
The supplier had a power outage at one of their data centres that’s affected their network transit and backhaul and therefore some
lines. We are seeing some lines return but not all and will update here as we get more news.
————-
Good afternoon.
We are aware of an issue within one of our suppliers we use for these lines. At this point we don’t have an announcement or full details but we are aware of this. The issue is also affecting their portal which we access and their status page isn’t working.
We are trying to get an update on this which we will pass on as soon as we can.
Note: this issue is not on the Merula network but within the supplier network. Our apologies to this circuits affected.
Oct 8, 2021 | Uncategorized
Due to an on-going incident with one of our SIP providers we are aware that calls to some of our own numbers may have call quality issues, The supplier is working to resolve this. In the mean time if you wish to call us please feel free to call us on 0330 341 6666 for support or 0330 341 0000 for our main number which uses an alternative carrier.
This will still reach the same teams here to help with any query
