Need help or advice?
Call us FREE 0800 298 2375

Update

COMPLETE: Supplier interconnect work planned 9th Feb 2016

No issues.

Planned Duration:

Start: 09/02/2016 00:01 GMT
Finish: 09/02/2016 05:00 GMT

Task:

One of our interconnect suppliers will be upgrading their capacity with BTW for DSL services in Telehouse North. This will involve migrating traffic on a pair of interconnects to new, higher capacity ones.

Customer Impact:

We do not anticipate any impact to Merula customer traffic, however all BT DSL services delivered from our Telehouse North node should be considered as AT RISK during this maintenance window.
If you have any queries about this work please raise a support ticket with our helpdesk.

We will issue an ALL CLEAR once this work has been completed.

[CLEARED] Speed & traffic issues across the Merula broadband network

Yesterday was Patch Tuesday from Microsoft. In addition, last night Apple released large updates to both iOS and OS X. As a consequence, all supplier back-haul networks (BT, TalkTalk, Vodafone etc) are seeing a dramatic spike in traffic across their links which is impacting our own services to our customers – those of you on ADSL/FTTC type circuits.

We apologise for this but this traffic should slowly die away over the course of the day so download speeds will start to see improvements.

Please bear with us. If you’re still seeing speed problems tonight/tomorrow, then raise a support ticket in the usual manner.

At risk: Electrical supply Avro Court 26th Nov [Completed]

COMPLETED: All Work was completed during the evening on 26th November and the Data Centre now has full access to both Mains Power, Generator Power and the UPS. There was no outage to hosted servers during the work.

UPDATE: we will have engineers on site on Thursday 26th, to replace parts in our automatic transfer panel. Most of the work will not be service affecting, however there will be a short period where the data centre will be supplied by UPS alone. While we don’t anticipate any disruption, the power to our racks should be considered at risk during this period. All “at risk” work will be carried out outside core business hours and we will have staff and external electricians on site as needed to monitor/resolve any issues found.

UPDATE: the engineers are about to isolate the mains supply and generator and move us over to UPS power. This has been tested to support the data centre for at least 30 minutes but for the period of down-time, we are at risk.

We are aware of an issue with our changeover panel here and an engineer is en-route to work on this.

If the panel needs to go off-line then resilience will be reduced for a short period but the generator and UPS are still available and have just been fully tested as part of our normal weekly maintenance regime and will cut in automatically. As a reminder, the generator has fuel for at least 24-hours of continuous running.

Once the panel issue has been resolved, we’ll update this post.

[CLEAR] Telehouse North router maintenance 13th Nov, from 10pm onwards

[UDATE] All work completed to plan.

We plan on making some further changes to a couple of our core network routers to isolate and correct a service setting that is causing a few BGP issues.

This may mean a few blips in routing tables as the changes are disseminated and an occasional up-tick in latency as routes converge and stabilise but we expect no downtime or any significant service issues.

If this planning changes, we will update this ticket

RESOLVED: Outage affecting some Merula services inc. ADSL

15:15  The problem has been traced to one of our core routers which ‘hung’ without (as it should have) notifying the automatic monitoring system of problems. This in turn affected routing for some customers connected to our Telehouse data centre. Some other important switches and routers were also impacted as they were unable to see that this router was down and also failed silently.

We have now restored service to the router and are monitoring for any further issues.

This should not have happened but despite our planning it did and we can only apologise for this; we are working on reconfiguring the core layout very shortly to make sure that this can’t cause such cascading problems for our customers again.

13.30 — we believe that we have resolved these service issues. That said, we are monitoring still and looking for the root cause. Again our apologies for this loss of service and updates will continue to be posted here once we’ve had a chance to check logs etc.

UPDATE: we’re working with our link team as this is mainly affecting services out of our London data centres.  Apologies for this extended down-time, we’re all working on this problem and will update here as we know more detail.

We’re aware of and are investigating the cause of outages affecting a number of services inc. some leased lines, ethernet circuits and broadband lines. As soon as we know the root cause & likely time to fix, we’ll update there.

WARNING: fake domain registration emails

Attention: Phishing emails are circulating.

Currently, waves of forged emails are being sent to customers of several registrars, including Merula.

The subject of these emails is ‘Domain xy.com Suspension Notice’; the faked sender is shown as ‘abuse@xyzdomain’.

These emails are fakes – please ignore and delete them and do not click on any included link.

In case you have any doubts regarding emails regarding your domains, please don’t hesitate to get in touch with support in the usual ways.

[UPDATED]: Planned maintenance works on one of our transit links, London 25th/26th Nov

UPDATE: the supplier has confirmed the new date, with timings unchanged for this work.

This should not affect any services or connectivity. There may be a brief transient period when routes flap until the routing tables converge again.

Start of maintenance: 25/11/2015 – 00:00 GMT
End of maintenance: 26/11/2015 – 04:00 GMT
Duration of work: Up to 30 minutes during the window

Classification of works: Standard
Description of works: Essential Third Party maintenance

[CLOSED] ADSL issues in Suffolk & Essex

We’ve been advised by BT Openreach that since around 9:46Am today, they’ve been experiencing connectivity problems with a number of their ADSL lines especially in these two counties caused by what they believe is a broken/failed back-haul cable. There’s no time to fix given yet.

Service restored: FINAL: The onsite cable repair team completed the activities by 03:55. All Broadband services were prioritised and restored at either 00:15 or 00:30.

[UPDATE] Cable incident has now been raised as the fault has been proved out from both ends (Colchester & Ipswich). Precision testing officers (PTO) are working on localizing the fault which is 4.72km away from Ipswich telephone exchange. A PTO is in the field driving to the location of Bourne Hill which is in the Wherstead area of Ipswich. The fault has yet to be confirmed but on initial findings it looking like a 96 fibre cable that has been affected.

[COMPLETED] Planned maintenance work on transit link 20th August

We’ve been advised by Cogent Network, one of our transit suppliers, of work to be undertaken on some of their core routers. This should not mean any loss of service to us or our end-users as we have multiple diverse routes into and out of our data centres — there may be a few moments of route instability until they re-converge and stabilise.

Date and Time of Maintenance: 20/08/2015 00:00 to 04:00 GMT (01:00 to 05:00 BST)

Impact: Loss of service for up to 15 minutes during the outage window

[Update] URGENT: Leased line outages

 

[Update] 15:45 The engineers have repaired the faults identified and all circuits are up & reporting as being heathy. If anyone is still seeing issues please mail into support@merula.net. We are conducting a postmortem with the supplier to ascertain the root cause, the fix made and to find out why their agreed backup routing didn’t kick in as part of the DR process. We will update the ticket as we know more.

15:10 Multiple back-hauls to us and other customers are affected by this fault; this has been escalated internally by the supplier to their 3rd-level team. No time to fix as yet.

14:26 Engineers have arrived on-site and are starting their investigations.

Engineers are en-route to both ends of this connection.

We are aware of a problem affecting one of our core bearer lines into HEX in London which in turn is affecting a number of customer  leased lines. These are currently hard down.

This is a high-priority outage for us and we are working to get this resolved as quickly as possible and apologise for the down-time being seen. We do not currently have a time to fix but expect a status report within the next 30-45 minutes.

We will post updates here.

EMERGENCY CONTACTS

The main support number is 0845 330 0666 (geographical 01480 355566)

There’s a second, fallback (geographical) number: 01480 411616. All numbers ring directly at our support centre, manned 24hrs x 365 days a year.

We'd also suggest that all customers subscribe to our mailing list (link above); status messages and updates will be delivered by email.

Subscribe