Continuing from http://status.netnorth.net/2015/05/06/us-based-connectivity-issue/ We just started to receive monitoring reports showing the same level of packet loss as received earlier this morning. We have removed the Level3 connectivity from our network immediately forcing re-routing of traffic. We will contact Level3 for more information.
Level3 based network issues are often very difficult to diagnose. This is due to Level3 operating their BGP network via MPLS (Multi-Protocol Label Switching), a wonderful and useful technology (but difficult to diagnose when your network spans the entire world!) Take this snippet of a traceroute for example: 2 ge-6-18.car1.Manchester1.Level3.net… Continue reading
We have received reports this morning of network alerts from 3rd party providers indicating trouble reaching the Netnorth network. After further investigation, we narrowed the issue to part of the Level3 network (traceroutes included below) located in the US – we raised this with Level3 who confirmed they currently have… Continue reading
Today’s routine generator test passed as expected. The generator was started manually, then run off-load 12:04 – 12:24 All measured values were within their normal ranges.
Today’s routine generator test passed as expected. The generator was started manually, then run off-load 11:25 – 11:49 All measured values were within their normal ranges.
After a routine check of our facilities, we have discovered that the Netnorth ‘Cold Storage‘ has reached a 90% capacity threshold. No services will be affected by this event, and we expect that the general trend in daily temperatures will deplete ‘Cold resources’ naturally, thereby moderating the problem. NN Support
Today’s routine generator test passed as expected. The generator was started manually, then run off-load 10:50 – 11:12 All measured values were within their normal ranges.
Today’s routine generator test passed as expected. The generator was started manually, then run off-load 10:06 – 10:28 All measured values were within their normal ranges.
At 1:34pm we received alerts regarding some destinations being unreachable. Further investigation showed that our fibre link to Level3 was down unexpectedly. This caused a BGP routing event. Our configuration started the BGP reconvergance process as soon as the router saw the fibre link drop and outbound traffic was re-routed… Continue reading
Today’s monthly on-load generator test passed as expected. The generator was started automatically after detecting the mains isolation, then ran on-load 10:43 – 11:03 The generator detected recovery of mains and shut down automatically in the expected timeframe All measured values were within their normal ranges.