We currently have a 10Gbit/s Openreach fibre circuit between BOL23 and MA1 offline. Openreach are investigating. Our ring automatically rerouted traffic to prevent an outage to customers. No outages are anticipated, but the ring is operating at reduced resilience in the interim.
At 11:15am, Bolton suffered a large power cut which caused a momentary loss of grid power to our DC. This was handled by our UPS systems as expected and was brief enough to not require backup generation. Electricity northwest (ENWL) are en-route to the area to investigate so a further… Continue reading
We have gracefully shutdown our LINX LON1 and LON2 LAN peering while our LINX Connexions provider performs some maintenance. All traffic has re-routed via other paths (LINX IXManchester, or IP Transits) without any disruption to traffic. We will bring these peers back online once our Connexions provider completes their maintenance… Continue reading
At approx 1:10pm today, we experienced a network issue. After investigating, we saw what appeared to be a Layer 2 loop within our network. We have mitigation against these loops, so this was unusual. After detailed investigation, we discovered that a Cisco 3850 series switch located in Manchester MA2 had… Continue reading
We’ve just been informed of some maintenance tomorrow between 7am and 7pm UK local time that may affect 2 legs of our metro ethernet ring in Manchester. Our metro ethernet ring should automatically re-route traffic via its alternate routes (BT, VirginMedia, Vodafone), but may cause a couple of periods of… Continue reading
To continue our network upgrades, we have ordered upgrades of our metro ethernet circuits, and IP Transit feeds. This will bring our metro ethernet rings up to a minimum of 10Gbit/s per segment. It will also increase our external connectivity to multiples of 10Gbit/s. Lead times for 10Gbit/s upgrades… Continue reading
At 10:07pm this evening, our monitoring detected the loss of our Metro Ethernet Ring links from one our suppliers. Our ring automatically reconverged as designed. At 10:13pm this evening, we saw these links reconnect. No planned maintenance is known, so we have raised a query with the supplier and… Continue reading
At approx 2:56pm UK local time today, we saw packet loss across our entire metro network for around 4 minutes. Initial diagnostics showed that one of our core network ring Cisco 3850-series switches had failed (it was stuck in a boot loop with failing self-tests). Ordinarily, this should not have… Continue reading
At 14:56 we had a core switch failure, and a consequential period of reconvergence, lasting approximately 4 minutes. This event will have been service affecting for some cutomers on our network. We are investigating the root cause of the hardware failure, and will publish a fuller RFO on our status… Continue reading
We saw our Vodafone Fibre circuit between BOL8 and LD2 drop at 8:38pm this evening. We have reported the fault to Vodafone to investigate. Our ring automatically re-routed via its alternate paths as expected without any customer outages. More updates will be added via comments as we receive them.