At approx 5:20am this morning, we experienced a dual Cisco device failure on our older switched network which caused an outage for any customers still present on that network. Customers and devices connected to our newer network were unaffected. Ports on the older network are being migrated to the newer… Continue reading
We’ve noticed some instability of peering connections between our router and LINX LON2 network which has caused a few periods of packet loss and latency to some locations. This does not seem to be affecting our LINX LON1, LINX MAN1, or Equinix-IX Manchester peering groups. We have temporarily shutdown our… Continue reading
We have detected increased latency for paths travelling via AS3257 (GTT) to/from the global internet. This seems to be intermittent at the moment. We have reduced our reliance upon this circuit to try to reduce the effects of this increased latency. If we continue to experience any issues relating to… Continue reading
Overnight we experienced a failure of DS-178 is BOL23 datacentre. Unfortunately, within a few seconds of this failing, we also experienced a network interface failure on another device within the network ring when it changed state from failover to active. This caused an isolation of various network devices within BOL23…. Continue reading
At 12:14am UK local time, our monitoring reported an outage on two Vodafone provided fibre circuits delivered into our BOL8 location. These are both delivered over the same fibre bundle. These circuits provide our main office telephone service, and one of our metro ethernet fibre ring legs. After internal investigation… Continue reading
A core switch failure in our U23 DC will have affected all connectivity briefly this morning, we are currrently investigating the cause, and are remedially migrating a couple of significantly affected hosts as a priority. A fuller RFO will follow.
At 12:12pm today, our monitoring reported a failure of our VirginMedia metro ethernet leg within our metro ring. We have raised this as a fault with VirginMedia for investigation. We have a resilient ring so no outage or degration of service is in effect, only a reduction in resilience.
At approx 9:37pm tonight one of our SAN devices in BOL23 (EQL-SAS-4) reported that its primary controller was not responding. The controller triggered a kernel panic which caused the controller to reboot. The failover controller took over once it had confirmed the primary controller was not responding. This may have… Continue reading
At 10:03am this morning, we saw a collection of BGP connections drop in Equinix MA1 (Manchester). These were spread across LINX MAN1, EquinixIX, and our GTT IP Transit connection. This caused a couple of minutes of instability whilst BGP reconverged and rerouted traffic automatically. A few minutes later, we saw… Continue reading
We have identified a failure of one of our customer facing network switches in BOL8. Any customers connected directly to DS-173 in BOL8 will be affected. Other switches and customers are unaffected. We are investigating the cause and hope to have this switch back online shortly.