At 10:53am UK local time, we saw our physical link to Level3 (AS3356) drop for 5 minutes. This caused our routers to immediately terminate the BGP routing session with Level3 and start to reconverge via our alternative paths leading to 2-3 minutes of instability whilst routes updated. The link to… Continue reading
Earlier today, we had a failure of one of our firewalls pairs. This occurred at 2:43pm UK local time. Our firewalls operate in pairs in a failover capacity. Unfortunately, today this failover took much longer than expected. The failover took place within a few seconds as designed, however this triggered… Continue reading
We have received monitoring alerts showing a core problem at one of our IP Transit providers (Level3). While trying to confirm this alert, we can’t reach Level3’s London POP at all so feel that the issue at their end is a lot wider than a local issue. As a result,… Continue reading
We experienced a fibre outage within the BT network this morning at 8:33am. There is a fibre break in one of our core ring circuits between Bolton and Manchester. Our network automatically failed over to its redundant path, and continues to operate normally. Once the fibre break has… Continue reading
Hello customer Outage time frame Friday, 9/9/2013 start time 2.58 pm fix time 3.03 pm Duration 5 minutes Description, Network BGP (border gateway protocol) instability. A small configuration change on the data switches combined with a flush of its routing tables let the network reconverge and should prevent any future… Continue reading
Our supplier appears to have completed their maintenance early. The circuit has been back in operation since approx. 10pm without any issues. All traffic restored to its relative preferred path automatically. We will continue to monitor the circuit until the end of the supplier maintenance window (midnight UK local time)… Continue reading
One of our core network partners, Cable&Wireless, are currently performing routine maintenance on one of the segments of our core inter-building networks. All traffic has been automatically diverted via its failover path. During this maintenance window, our resilience is reduced. There is also the possibility of a brief disturbance to… Continue reading
This morning at 8:10am, we started seeing increased latency and packet loss between our network and BT internet. This settled down at 10:34am. Some diagnostics showed the problem to be around the London area, possibly at the interconnect of Global Crossing and BT. BT have raised this as a fault… Continue reading
The virgin media circuit was restored around 3:30pm after a 2 hour outage. Traffic has stabalised between the circuits. We are awaiting a further detailed explanation from virgin media.
Today we saw the number of active IPv6 routes on the internet hit 10,000. Each route is (more or less) one ISP on the internet. This shows the growth of providers taking up the next generation protocol. As you may be aware, the availability of IPv4 (current) IP addresses is… Continue reading