We have identified an issue with one of our core network switches, DS-129, located in LDeX 2 Manchester. This switch has hit a known Cisco IOS-XE bug and requires a reboot to activate an IOS-XE update. As this issue is currently service affecting, we will perform an emergency upgrade/reload of… Continue reading
We are currently seeing a distributed denial of service attack against a customer since around 12:15am (UK local time) – this appears to be an LDAP reflection amplification attack. Some mitigation has been applied at our network borders to reduce traffic levels to minimise disruption for the affected customer. This… Continue reading
Openreach will be performing planned engineering works from 11pm 2/Sep/2020 until 7am on 3/Sep/2020 in Manchester. The openreach supplied reason for maintenance is “node/joint remake for faults/quality purposes” This maintenance will cause the loss of one of our metro ethernet circuits during the maintenance window. No impact is expected due… Continue reading
At 14:27 (UK local time) today, we saw our peering sessions via LINX Manchester flap (restored at 14:31). Further reports came in of a loss of power feeds (both A and B feeds) for some customers of Equinix MA1, however this has not affected our power feeds for equipment in… Continue reading
At 1:19am this morning (UK local time), one of our core switches rebooted due to a software bug. This switch connected to one of our core routers which caused a brief period of instability whilst our network reconverged. The core switch is back online, however the core router is currently… Continue reading
At 7:04am this morning, we had a dark fibre failure between two of our Manchester datacentres. The cause of this is currently being investigated by the fibre provider. Unfortunately, our backup fibre path failed to take over leading to one of our Manchester datacentres becoming isolated. This was caused by… Continue reading
At approx 4:55pm UK local time, we experienced a loss of our LINX (London) connectivity. This is provided by a “Connexions” partner, and we suspect an issues lies somewhere within their network. A brief period of instability would have occurred whilst BGP reconverged automatically to our failover paths. No further… Continue reading
At approx 6:45am this morning, our monitoring triggered reports of instability for connectivity via our LINX connection (London Internet Exchange) in London. As this link was not saturated in any way, we took the decision to shutdown the link to cause all traffic to re-route via alternative paths. This connection… Continue reading
At 1:43pm, our monitoring identified a link failure on our metro ethernet ring between Bolton and Manchester. As designed, our ring reconverged via backup links and no outage was experienced – however the ring now has reduced resilience. We have raised a fault with Virgin Media to investigate the cause… Continue reading
At approx. 8:15pm this evening, our alerting saw a flap of our protected fibre link between Manchester and London (provided as an MPLS pseudowire) which affected connectivity between LDeX2 and LDeX1 datacentres. This link is used solely for our connectivity to LINX LON1 and LINX LON2 peering networks, and caused… Continue reading