Earlier today, we had a failure of one of our firewalls pairs.
This occurred at 2:43pm UK local time.
Our firewalls operate in pairs in a failover capacity. Unfortunately, today this failover took much longer than expected.
The failover took place within a few seconds as designed, however this triggered some routing issues within the failover firewall which prevented traffic from flowing normally and presented itself as an outage for the IP Address protected by that firewall pair.
This routing action is contrary to the documentation from the vendor and is being investigated further. A workaround has been put in place in the meantime.
Full routing was restored by 2:51pm.