This morning at approx. 6am UK local time, we had a switch failure of switch DS-194. This caused an outage for any directly connected customer. It also triggered an event in another switch which could affect connectivity to some virtual machines. At approx 8:45am, this switch was recovered and all… Continue reading
LINX (London Internet Exchange) are experiencing issues with their LON1 network today. This is causing brief periods of instability and BGP session outages across their network and has been seen across multiple switches now. As a result, we have shutdown our LON1 peering until LINX confirm the issue is resolved…. Continue reading
At 5:13am this morning we saw a brief power event on our main UPS in BOL8 datacentre followed by Switch DS-105 going offline. Although no issues are reported surrounding the power event, any customers directly connected to switch DS-105 are currently offline. DS-105 is a legacy switch stack and not… Continue reading
At approx 11:45pm tonight (16th Dec), one of our core switches (DS-193) crashed and rebooted. This had the same error code as the other switch earlier this week and runs the same IOS version. As a result, we will now schedule IOS upgrades on all core switches.
At 1:34pm today, one of our switches (DS-162) in BOL23 crashed and rebooted. Investigation looks like a bug in the Cisco IOS image being triggered causing the automatic reboot. Any customers directly connected to this switch would have seen a brief period of disconnection during the reboot. No further issues… Continue reading
We currently have a dark fibre outage between two datacentres in Manchester. Our fibre provider is investigating. No outages to customers are expected, however this fibre outage does remove resilience at our “LD2” Manchester hosting. Customers hosted within Manchester are now “at risk” although no outages are expected. We operate… Continue reading
We have identified an issue with one of our core network switches, DS-129, located in LDeX 2 Manchester. This switch has hit a known Cisco IOS-XE bug and requires a reboot to activate an IOS-XE update. As this issue is currently service affecting, we will perform an emergency upgrade/reload of… Continue reading
We are currently seeing a distributed denial of service attack against a customer since around 12:15am (UK local time) – this appears to be an LDAP reflection amplification attack. Some mitigation has been applied at our network borders to reduce traffic levels to minimise disruption for the affected customer. This… Continue reading
Openreach will be performing planned engineering works from 11pm 2/Sep/2020 until 7am on 3/Sep/2020 in Manchester. The openreach supplied reason for maintenance is “node/joint remake for faults/quality purposes” This maintenance will cause the loss of one of our metro ethernet circuits during the maintenance window. No impact is expected due… Continue reading
At 14:27 (UK local time) today, we saw our peering sessions via LINX Manchester flap (restored at 14:31). Further reports came in of a loss of power feeds (both A and B feeds) for some customers of Equinix MA1, however this has not affected our power feeds for equipment in… Continue reading