At approximately 6:07pm, we were alerted to a Cisco Switch failure of DS-173 on part of our older core network. A simultaneous dual fan failure caused the device to power down after 30 seconds. We arrived on site and migrated all affected ports on DS-173 to our newer core network,… Continue reading
At 10:26pm UK local time, we started to receive automated reports of increased latency and packet loss to some destinations. After some diagnostics, we discovered that traffic taking an inbound path via Cogent (AS174) to us was seeing increased latency of +900ms and 25-35% packet loss. Outbound data was unaffected…. Continue reading
At approximately 9:17am UK local time, we started receiving network status reports from our BOL23 datacentre indicating an issue with connectivity. We quickly identified this as a dual fan failure in switch DS-178 which caused it to shutdown after 60 seconds to prevent an overheat condition. This affected connectivity on… Continue reading
At approx 8:25pm on 30th May 2024 we identified some network disruption to our core network. After investigation, we discovered that we had lost power to a rack in Manchester that houses some connectivity and also one of our core routers. Generally this would cause a couple of minutes of… Continue reading
At approx 5:20am this morning, we experienced a dual Cisco device failure on our older switched network which caused an outage for any customers still present on that network. Customers and devices connected to our newer network were unaffected. Ports on the older network are being migrated to the newer… Continue reading
We’ve noticed some instability of peering connections between our router and LINX LON2 network which has caused a few periods of packet loss and latency to some locations. This does not seem to be affecting our LINX LON1, LINX MAN1, or Equinix-IX Manchester peering groups. We have temporarily shutdown our… Continue reading
We have detected increased latency for paths travelling via AS3257 (GTT) to/from the global internet. This seems to be intermittent at the moment. We have reduced our reliance upon this circuit to try to reduce the effects of this increased latency. If we continue to experience any issues relating to… Continue reading
Overnight we experienced a failure of DS-178 is BOL23 datacentre. Unfortunately, within a few seconds of this failing, we also experienced a network interface failure on another device within the network ring when it changed state from failover to active. This caused an isolation of various network devices within BOL23…. Continue reading
At 12:14am UK local time, our monitoring reported an outage on two Vodafone provided fibre circuits delivered into our BOL8 location. These are both delivered over the same fibre bundle. These circuits provide our main office telephone service, and one of our metro ethernet fibre ring legs. After internal investigation… Continue reading
A core switch failure in our U23 DC will have affected all connectivity briefly this morning, we are currrently investigating the cause, and are remedially migrating a couple of significantly affected hosts as a priority. A fuller RFO will follow.