There appears to have lost power and/or a switch failure in a single rack in our Huntingdon Data Centre. This dropped approx 2am this morning. This rack houses a small number of Merula and customer servers
We are aware and will investigate and resolve this asap. We are planning to be on site approx 7am and will resolve this issue then
We apologise for any issues this may cause and will update this as soon as we have more details
[update 8:15am]
The issue appears to be related to the switch in the rack – after being offline for approx 90 minutes the switch came back up and connectivity was restored to most servers in the rack. However we are seeing issues with connectivity via a couple of servers in this rack. We have therefore taken the decision to manually reboot the switch to see if this restores service given that the servers themselves look OK and have not rebooted. This will unfortunately result in a loss of connectivity to all services in this rack for a couple of minutes. We will update this as we know more
[Update 8:53]
The switch was rebooted – and the latest saved config has been re-applied. This we believe has restored service to the services we are aware of that had an issue. We are continuing to check for anything else with an issue and are investigating the cause of the switch outage further. We may have to schedule a swap out of the switch if we cannot locate an obvious issue here. However we believe that currently all services in Huntingdon should now be restored. Please do email support if you continue to see any issues
[Update 9:20]
This affected switch appears to have failed again. We will now start swapping this out for a replacement switch. We will have an update within the next 45 minutes
[Update 11:30AM]
The switch was replaced and we believe all services have recovered. We are checking for any remaining issues. If you are are seeing any issues please do raise them with support@merula.net. We will update this further later in the day or if we locate any remaining issues