On 07th July 2024, we got alerted at 01:14 am (CEST) about down bgp-sessions at FFM2 (Interwerk). Thorough analysis suggested either a power or network connectivity issue with the equipment within this points of presence.

Following quick analysis, we were able to pinpoint the exact root cause onto a outage of darkfiber connectivity towards Interxion, impacting multiple operators at once. Unfortunately, the outage also took down our out-of-band access, which is provided over another network operator being also affected.

Together with datacenter on-site personnel, we were able to gain access to our equipment and restored connectivity over a adhoc deployed backup link, allowing to take other operators at this site live again.

According to the datacenter operator, the reason for this outage was a human error, causing darkfiber crossconnect connectivity to be terminated at Interxion, following a inventory clean-up of the cabinet operator. This event was out of our control and caused upstream.