Dec 31, 2012 | Outages
At approx 3:30pm today there was a power outage from the raw mains to our Huntingdon Data Centre which resulted in the loss of a single phase of the mains feed.
While we have a UPS and Generator backup to the data centre the actual failure coincided with a surge from the data we have here – this lead to our UPS protecting itself and our servers and consequently taking down the power feed to all servers in the data centre.
As soon as we were sure of the cause we manually transferred the power feed to our generator and restored power to all affected routers and servers – most were restored within 20-30 minutes. We are currently going through all remaining servers and checking their status.
We now believe that the main issue has been resolved and all public facing services and co-location servers are back and accessible.
IF you are still seeing an issue please call or email support the normal way and we will investigate and resolve the problem.
We are sorry for any issues that have been caused by this outage
Dec 29, 2012 | Outages
app approx 5am this morning our main outbound email server crashed do to a memory issue.
This has been resolved and all services should have been running again just after 8:30am
We are sorry of this issue affected you in any way
Dec 27, 2012 | Outages
Some of our users may have noticed some periods of packet loss and inability to reach certain locations today.
We have been working on this issue since the early hours of this morning – and after a lot of debugging e have traced this issue to a change made by a supplier to two of our inter-site fibre links.
We are liaising with the supplier to get this fixed and have made some temporary changes to stop the issue re-occurring however this had left the network with less automatic resilience than normal.
There is a small risk of a network break on our other services causing some areas of our network to become unreachable – however if this happens we will manually switch over to the backup links. In addition once or supplier completes work here there will be an at risk period when we swap back to all current links.
Virgin Media have now replaced a faulty NTE in our Huntingdon Data Centre which appears to have now resolved this issue. All of our links and resilient paths are now fully enabled. We will be monitoring this issue t ensure it does not re-occur.
We are sorry for any issues caused by this problem
Oct 1, 2012 | Planned Work
Due to an issue with our router in Sovereign house we are performing a update earlier than planned.
This will limit access to servers on the 217.146.105 and 85.158.153 IP ranges.
We anticipate the downtime will not be longer than 90 minutes and will start approx 8pm this evening.
Note this will NOT affect access to ADSL and connectivity services only a small number of servers (including the merula website and portal) that are on the IPs above.
This work was completed though took a little longer than planned. All services were restored just after midnight
Sep 23, 2012 | Planned Work
Our suppliers will be carrying out planned maintenance on one of our UPS systems at our Huntingdon Data Centre during the afternoon of the 1st October 2012.
While there should be no impact from this there is always a small risk with any work.
We will update this post on completion of the work
Jul 30, 2012 | Update
We are aware that some u8sers on BT based Broadband packages may be seeing slower than normal speeds at times. This is due to a combination of higher than normal usage (we are assuming this is related to the Olympics) and BT capacity issues in London. We are chasing BT for a resolution to this and working our hardest to ensure that we are able to provide the best speed possible.
Please note that our leased lines and o2 and cable and wireless connections are not affected in any way.
We apologise for any issues caused and are working with BT to resolve this as soon as possible.
May 31, 2012 | Outages
We are aware of some issues on the LINX peering network. While this is not affecting our network at present some sites and locations may be slow as a side effect of this. We are awaiting more details from LINX and will update this as more is known
Feb 28, 2012 | Uncategorized
We have identified an issue with the reporting from our primary UPS in the Huntingdon Data Centre. We have specialist engineers arriving on site with the necessary spares to fully test and resolve this issue.
During the work there will be times when the data centre will be powered via mains power rather than via the UPS system. This means that in the unlikely event of a problem with mains power to the building there may be issues with some services here.
We do not anticipate any issues but wanted to advise our customers of this work due to the slight risk associated with it.
This work has now been completed with the replacement of the main board in our UPS which appears to have resolved all issues
Nov 7, 2011 | Outages
We are aware of reports of a bug affecting Juniper core routers affecting some ISPs and large interconnects that are causing these routers to crash. Symptoms include increased latency and slow speeds when attempting to load websites. Please be aware of this as it may affect your own connections if any of the intermediate links depend on these ISPs or routers.
Nov 4, 2011 | Outages, Unplanned downtime
We apologise for this brief window of down-time caused by a misconfiguration on a third party suppliers link. This meant that a network loop was introduced into our network today causing a brief outage for some users.
The core switch here recovered automatically after about 10 minutes and took the link out of action and we have left it down while we investigate the cause in more detail.
We will post more as soon as we know.
