[CLOSED]: Routing issues esp. affecting VOIP circuits

UPDATE 28th May: the link was re-enabled overnight and was monitored. It is now stable & we are confident that the problems it caused have now been fixed by LINX.
UPDATE: we have temporarily disabled this link and routes are now quickly recovering. We will update when this issue is fixed at LINX and we re-enable this connection.
We (and other ISPs) are seeing some instability at one of the LINX peering points today, which may cause slowness to some locations etc and strange routing paths. This is particularly affecting many VOIP circuits but other transits are also showing problems.
We are monitoring and waiting for an update from LINX – if the instability becomes too great we will disable our peering; however as other ISPs also connect there, this may not resolve all issues and we’re actively investigating alternatives.

SECURITY: possible consumer router vulnerabilities

If you have a domestic-grade router you should check by reference to this report, whether or not it may be affected by a recently notified vulnerability https://www.sec-consult.com/fxdata/seccons/prod/temedia/advisories_txt/20150519-0_KCodes_NetUSB_Kernel_Stack_Buffer_Overflow_v10.txt

If you believe this may be the case you should contact the manufacturer direct or check their web-site to see if your model has been patched and proceed accordingly.

 

Router Reload [10/05/2015]

Due to a memory leak one of our core routers (thn-gw2) has just rebooted. We are seeing all services back as they should be.

As this is one of a resilient pair of routers there should not have been any outage – BUT there may have been a  brief ‘blip’ while the network switched over top the backup router (and another when thn-gw2 returned to service).

We are monitoring the issue and will update this post if we see any further issues

 

[CLOSED] Intermittent packet loss on one transit circuit

UPDATE:

Through investigations Level3 isolated the trouble to overutilization on a link from Toronto to Chicago. Traffic was rerouted to avoid the trouble and services are now restored. Level3 will continue monitoring services to ensure continued stability.

Please be advised that one of our transit providers, Level3, are experiencing an issue that may result in packet loss to some destinations. We will keep you updated as and when we receive more information. If necessary, we will disable this link until they resolve the problem

Planned work on one of our back-haul links 14th May

Between 23:00 and 05:00 on the night of 14th running into 15th, one of our back-haul providers is making some configuration changes. At some point during this window there may be a brief (30 second or so) increase in lost packets but there should be no loss of Internet connectivity. During this same brief window, the connections to any servers hosted at our Manchester POP will be at risk.