Need help or advice?
Call us FREE 0800 298 2375

Update

[CLEAR] Telehouse North router maintenance 13th Nov, from 10pm onwards

[UDATE] All work completed to plan.

We plan on making some further changes to a couple of our core network routers to isolate and correct a service setting that is causing a few BGP issues.

This may mean a few blips in routing tables as the changes are disseminated and an occasional up-tick in latency as routes converge and stabilise but we expect no downtime or any significant service issues.

If this planning changes, we will update this ticket

RESOLVED: Outage affecting some Merula services inc. ADSL

15:15  The problem has been traced to one of our core routers which ‘hung’ without (as it should have) notifying the automatic monitoring system of problems. This in turn affected routing for some customers connected to our Telehouse data centre. Some other important switches and routers were also impacted as they were unable to see that this router was down and also failed silently.

We have now restored service to the router and are monitoring for any further issues.

This should not have happened but despite our planning it did and we can only apologise for this; we are working on reconfiguring the core layout very shortly to make sure that this can’t cause such cascading problems for our customers again.

13.30 — we believe that we have resolved these service issues. That said, we are monitoring still and looking for the root cause. Again our apologies for this loss of service and updates will continue to be posted here once we’ve had a chance to check logs etc.

UPDATE: we’re working with our link team as this is mainly affecting services out of our London data centres.  Apologies for this extended down-time, we’re all working on this problem and will update here as we know more detail.

We’re aware of and are investigating the cause of outages affecting a number of services inc. some leased lines, ethernet circuits and broadband lines. As soon as we know the root cause & likely time to fix, we’ll update there.

WARNING: fake domain registration emails

Attention: Phishing emails are circulating.

Currently, waves of forged emails are being sent to customers of several registrars, including Merula.

The subject of these emails is ‘Domain xy.com Suspension Notice’; the faked sender is shown as ‘abuse@xyzdomain’.

These emails are fakes – please ignore and delete them and do not click on any included link.

In case you have any doubts regarding emails regarding your domains, please don’t hesitate to get in touch with support in the usual ways.

[UPDATED]: Planned maintenance works on one of our transit links, London 25th/26th Nov

UPDATE: the supplier has confirmed the new date, with timings unchanged for this work.

This should not affect any services or connectivity. There may be a brief transient period when routes flap until the routing tables converge again.

Start of maintenance: 25/11/2015 – 00:00 GMT
End of maintenance: 26/11/2015 – 04:00 GMT
Duration of work: Up to 30 minutes during the window

Classification of works: Standard
Description of works: Essential Third Party maintenance

[CLOSED] ADSL issues in Suffolk & Essex

We’ve been advised by BT Openreach that since around 9:46Am today, they’ve been experiencing connectivity problems with a number of their ADSL lines especially in these two counties caused by what they believe is a broken/failed back-haul cable. There’s no time to fix given yet.

Service restored: FINAL: The onsite cable repair team completed the activities by 03:55. All Broadband services were prioritised and restored at either 00:15 or 00:30.

[UPDATE] Cable incident has now been raised as the fault has been proved out from both ends (Colchester & Ipswich). Precision testing officers (PTO) are working on localizing the fault which is 4.72km away from Ipswich telephone exchange. A PTO is in the field driving to the location of Bourne Hill which is in the Wherstead area of Ipswich. The fault has yet to be confirmed but on initial findings it looking like a 96 fibre cable that has been affected.

[COMPLETED] Planned maintenance work on transit link 20th August

We’ve been advised by Cogent Network, one of our transit suppliers, of work to be undertaken on some of their core routers. This should not mean any loss of service to us or our end-users as we have multiple diverse routes into and out of our data centres — there may be a few moments of route instability until they re-converge and stabilise.

Date and Time of Maintenance: 20/08/2015 00:00 to 04:00 GMT (01:00 to 05:00 BST)

Impact: Loss of service for up to 15 minutes during the outage window

[Update] URGENT: Leased line outages

 

[Update] 15:45 The engineers have repaired the faults identified and all circuits are up & reporting as being heathy. If anyone is still seeing issues please mail into support@merula.net. We are conducting a postmortem with the supplier to ascertain the root cause, the fix made and to find out why their agreed backup routing didn’t kick in as part of the DR process. We will update the ticket as we know more.

15:10 Multiple back-hauls to us and other customers are affected by this fault; this has been escalated internally by the supplier to their 3rd-level team. No time to fix as yet.

14:26 Engineers have arrived on-site and are starting their investigations.

Engineers are en-route to both ends of this connection.

We are aware of a problem affecting one of our core bearer lines into HEX in London which in turn is affecting a number of customer  leased lines. These are currently hard down.

This is a high-priority outage for us and we are working to get this resolved as quickly as possible and apologise for the down-time being seen. We do not currently have a time to fix but expect a status report within the next 30-45 minutes.

We will post updates here.

[Update] Windows 10 release 29th July onwards

28th July: the Windows 10 code is currently being downloaded “in the background” to machines that have pre-ordered the update which means that some back-haul providers are already seeing some slow-downs. Be aware that this is likely to continue for a while.

——————-

Microsoft are making available from this date their new version of Windows. It will roll out over a  period of time, starting on this date.

Whilst we will have brought on-stream additional back-haul being provided by our suppliers, we believe that there will still be some instances of higher latency and slower speeds caused by this, that could affect many of our users especially as this may also affect our supplier core networks.

We will make every effort to mitigate these effects but you should be aware this may be an issue before raising a fault ticket.

[UPDATED] Planned maintenance work on transit link 8th August 2015

Change of date:

We’ve been advised by Cogent Network, one of our transit suppliers of work to be undertaken on some of their core routers. This should not mean any loss of service to us or our end-users as we have multiple diverse routes into and out of our data centres — there may be a few moments of route instability until they re-converge and stabilise.

Maintenance window:

Date of work: 8.8.2015
Start time: 00:00 BST
End time: 02:30 BST
Place of work: London Telehouse North
Work description: Hardware upgrade on Core router controller cards

[CLOSED]: Routing issues esp. affecting VOIP circuits

UPDATE 28th May: the link was re-enabled overnight and was monitored. It is now stable & we are confident that the problems it caused have now been fixed by LINX.
UPDATE: we have temporarily disabled this link and routes are now quickly recovering. We will update when this issue is fixed at LINX and we re-enable this connection.
We (and other ISPs) are seeing some instability at one of the LINX peering points today, which may cause slowness to some locations etc and strange routing paths. This is particularly affecting many VOIP circuits but other transits are also showing problems.
We are monitoring and waiting for an update from LINX – if the instability becomes too great we will disable our peering; however as other ISPs also connect there, this may not resolve all issues and we’re actively investigating alternatives.