At 11:55pm tonight, our connection to Level3 appeared to disconnect. The symptoms are identical to the previous occasions where the Level3 router rebooted so we assume this to be the case again tonight.
The connection is slowly coming back online, which further supports this suspicion.
As Level3 appear to be unconcerned by a router that reboots itself, we have already made plans to migrate to an alternative transit supplier. This is due to take place early February 2017.
Once this replacement is online, we will likely retire our Level3 connectivity entirely.
Some routes may have experienced a brief period of packet loss or increased latency during this transit fault whilst routers around the internet changed paths to our alternate transit links. The majority of destinations would have been entirely unaffected however.
As this is caused by routers outside of our control, we have no way to prevent this brief packet loss to some destinations.
Any paths via our peering links or other transit connectivity are unaffected.
No further issues have been seen overnight (and none were expected)
Given the time of day and our network layout, this should have had minimal impact on our network connectivity – although some monitoring systems are primarily connected via Level3 in the USA so may have caused brief outage reports if not multihomed at their end.