Elevated Reports of Connection Issues in parts of Europe

Incident Report for Discord

Resolved

Service is operating as expected at this time.
Posted Nov 27, 2020 - 16:02 PST

Monitoring

As of this time the rate of connection reports has returned to nominal levels, Oncall Engineering will continue to monitor the network health.
Posted Nov 27, 2020 - 15:42 PST

Update

Additional details have come to light, the issue with the damaged fiber run was exacerbated by a BGP table leak that caused two tier 1 network peers to incorrectly send traffic through the damaged run and through connections insufficient to handle the diverted throughput.

The BGP leak occurred at approximately 21:18UTC and traffic routing was corrected at 22:47UTC.

The impact was mostly localized to France's Free ISP and Turkey's TurkNet ISP.
Posted Nov 27, 2020 - 15:41 PST

Identified

A likely cause for the issue has been identified. The issue is upstream of Discord and our CDN, likely cause is physical damage to backbone fiber. Traffic is being routed around this damaged fiber run.
Posted Nov 27, 2020 - 15:35 PST

Update

Oncall Engineers are continuing to investigate this issue to determine the scope and cause. Issue seems to be isolated to Western Europe. We are continuing to collect data to determine root cause.
Posted Nov 27, 2020 - 14:50 PST

Investigating

Investigating elevated reports of Connection Issues from France and Italy.
Posted Nov 27, 2020 - 14:10 PST