Issues overnight

Between approx 23:30 on 21/12 and 0:30 on the 22/12 there was a partial outage in our Sovereign House hosting facility. This affected servers hosted in the building (numbering about 4) and a few IP ranges hosted off of this router.

All ADSL and other hosted services were fine. the cause turned out to be a faulty PSU in one server (actually the one hosting this site…). Once we isolated the faulty server the rest of the rack came back – we then swapped th PSU on this server and all appears to be fine now.

If you are still seing any issues please let us know

Network issue

We are aware of an issue preventing access to some IP ranges on our network at this time. We are urgently investigating this and expect service to be back very shortly

Network issue today

We are aware of a network issue today that limited connectivity to some servers and sites. While we have not identified the exact cause as yet we believe the problem is now cleared. We believe thius may have originated outside out network but we are investigating this as a matter of urgency

Recent email issues

We are aware that some users may have seen delays to emails over the last week or two. We have been investigating the cause and now have a plan of action to resolve these once and for all.

Earlier today we made a major change to the config on one of our AV /SPAM servers int he cluster here. Assuimg this is successful the same change will be rolled out to all remaining servers tomorrow and Monday and this should see an end to this issue.

We will update this post as soon as we have more details

‘At Risk’

During some initial tests on our generator today in advane of the maintanance work on Friday engineers have found a suspect control board. This controls and monitors the generator and is not directly involved in the power feed to our racks.

As part of the work on Friday it is planned to replace this board and resolve the data issue currently being seen. This work should not cause any issues at all, but since this is working on live equiment there is always a risk of an unexpected issue.

We currently don’t have a time for this work to be completed but we will update here as soon as possible

Generator maintanance / test

As part of the maintenance at our data centre in Huntington we will have engineers performing maintenance on  one of our generators on the morning of July 11th. There should be no outage or problems related to this but it will reduce resilience  while this takes place.

Mail Scanning issues

Due to a huge increase in spam and a hardware failure on one of the mail scanning servers here we are aware that some email has been delayed over the last 24 hours. We now have all servers back up and running and an extra server will be live later this evening.

This issue has also made us bring forward an upgrade to the scanning engine – this has been in testing for a while bit is now live – this should also reduce the amount of spam received – and the accuracy will increase as time goes on.

if you are still seeing any issues please call or email and we will look into the issues as soon as possible

Brief network bounce this morning

As part of our preperation to perform more diagnostics on one of our routers in Telehouse North following the issues we have been seeing with it we have made some changes to the network configuration. These will also reduce the reliance on this router and therefore mean that any further issues will not be customer affecting.

During the config change you *may* have seen a very brief network issue to some locations however this should not have lasted more than 2-3 minutes at a maximum. All work is now complete and if you are still seeing any issues please contact support in the normal manner.

We apologise for the lack of notice here but wanted to ensure that any furter issue did not cause any problems

Central Pipes unbalanced

Due to a BT incident earlier for which we are still waiting clarification we are currently seeing an imbalance on the central pipes used for the  @adsl.merula and @adsl.wizards realms. if your login is one of the above and you are seeing slow speeds you may like to diconnect and re-login in an attempt to change to the less congested central.

You may need to do this more than once since you have a 50/50 chance od re-connecting to the congested cental.

if we can get moew details from BT we will post them here

Possible Disruption Friday 16/5

Further from the recent router issues we have determined that this is caused by a memory issue on the router hardware. We have now sourced replacement RAM and we will have engineers on site on Friday to replace this. The router itself way be down for 10-15 minutes but most if not all traffic should work round this and there should be little if any noticeable issues