CLOSED: Portal diagnostics
UPDATE: the supplier code issues have been worked around and all tests are now available again.
—————————–
We are aware of issues working with the APIs from the suppliers that’s affecting the ability to run diagnostics via the Merula portal (https://adslreports.merula.net); we are working on this now and will advise here as soon as this is resolved. Apologies for this loss of service.
OUTAGE: overnight works
RESOLVED: the fibre break was fixed at 8:53pm last night and all services are back to normal. This ticket is now closed.
UPDATE 15:50 — our supplier advises “in relation to the issue identified in Cambridge area regarding loss of service. We are still working hard to resolve these issues as a priority.
Due to heavy traffic in the area, this is impacting our ability to get into the pit location necessary to move services onto spare fibres. Traffic management is required to safely carry out the work and this cannot be implemented until 20:00 this evening due to local authority restrictions.
From 20:00 onwards we can commence repairs so we should see services begin to come back online overnight this evening.
Please accept our apologies at this time as we are treating this matter with the utmost urgency.”
UPDATE: 14:45pm — because of the amount of splicing work needed and the traffic management problems at this location, the ERT has been pushed out to 20.45 today.
UPDATE: 13:00pm — Traffic management is only permitted until 1530Hrs but traffic lights and barriers are on site ready to go. We’ve not been advised of a new ERT as yet.
UPDATE 12:30pm — work continues on-site at the node. Raised for traffic management. Has ERT for 15 minutes time but engineer still working and waiting for response from traffic management team.
Next update due at 1300Hrs.
UPDATE 10:31am — at 1008Hrs an engineer arrived on site. The overnight change was moving to new cable but there appears to be a fibre break. No ERT at present.
UPDATE 10:11am — this is now being treated as a high priority fault as completion has overrun and classified as a MSO due to the affect on Merula (and other) customers through this node.
UPDATE: we are escalating this for a substantive update as the works have again over-run their new completion time.
Emergency unscheduled works by one of our main backhaul suppliers have overrun and mean that one of our 1Gb links to London from the data centre here in Huntingdon is hard down. Traffic is being routed via one of our backhaul links but as this is slower you may see some slowness on some traffic until this work is completed.
We have been told that the work was due to finish at 6am but this has now been pushed out to 8am. We will continue to update here. Our apologies if this affects your connection(s).
Emergency work 15/02/2018
Our supplier has identified a potential issue with a fibre bearer which holds some backup links to our Huntingdon Data Centre. They are making a config change to correct this overnight.
As these are backup links it should not have a significant impact but there is some chance of packet loss as the link drops / returns as the network re-converges.
We will update this post as the work progresses
ADVISORY: security issues with some Netgear routers
Good morning.
Whilst we don’t supply Netgear routers, we know that a number of our customers use them, so wanted to ensure you were aware of potential vulnerabilities that need to be addressed via a firmware update.
The list of (so far) affected models is below. You or your IT specialist should ensure that you update as soon as possible to mitigate any risks.
List of NETGEAR Routers At Risk
DGN2200v4
R6100
R6220
R6250
R6300v2
R6400
R6400v2
R6700
R6900
R6900P
R7000
R7000P
R7100LG
R7300DST
R7500
R7500v2
R7800
D7800
R7900
R8000
R8300
R8500
D8500
WNDR3400v3
WNDR4500v2
EX6200v2
The Five Vulnerabilities
TWSL2018-002: Password Recovery and File Access
Some routers allow arbitrary file reading from the device provided that the path to file is known. Total of 17 products are affected.
TWSL2018-003: Finding 1: Post-Authentication Command Injection
This one affects six products and reflects a root level OS command execution via the device_name parameter on the lan.cgi page, although the attack requires authentication.
TWSL2018-003: Finding 2: Authentication Bypass
This also affects large set of products (17) and is trivial to exploit. Authentication is bypassed if “&genie=1” is found within the query string.
TWSL2018-003: Chained Attack: Command Injection
This is a three-stage attack leveraging three separate issues: CSRF token recovery vulnerability and the two findings in TWSL2018-003. As a result, any user connected to the router can run OS commands as root on the device without providing any credentials.
TWSL2018-004: Command Injection Vulnerability on D7000, EX6200v2 and Some Routers
Only 6 products are affected, this allows to run OS commands as root during short time window when WPS is activated.
PLANNED WORK: Core switch upgrades.
Following the rapid growth on the Merula network we have been planning a major upgrade to our core switching platform to Juniper devices. These will allow us to dramatically increase the capacity between sites and add further stability to our network. Over the coming week, work will take place to introduce them into our network.
This will in due course lead to some limited downtime as links are migrated. We will update here with specific slots for this but wanted to provide some initial details for this work.
Any initial questions, please email support via the normal means.