Chicago networking outage

Incident Report for BitLaunch

Resolved

We have received confirmation from our upstream that this event is resolved:

> One of the TOR switches experienced a failure, impacting the virtual chassis and causing downtime. Our network engineers determined the switch is beyond recovery and requires replacement. To restore connectivity, we have moved your uplink to a functioning switch. Connectivity has been successfully restored.
Posted Mar 16, 2025 - 17:41 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Mar 16, 2025 - 17:31 UTC

Update

We are continuing to work on a fix for this issue.
Posted Mar 16, 2025 - 17:31 UTC

Update

We believe the switch is replaced at this point and we're monitoring the situation.
Posted Mar 16, 2025 - 17:27 UTC

Update

We are continuing to work on a fix for this issue.
Posted Mar 16, 2025 - 16:01 UTC

Update

A fix is being deployed and we are working with our upstream to restore the connection.
Posted Mar 16, 2025 - 14:33 UTC

Update

We are continuing to push our upstream for updates. Unfortunately the issue is largely out of our control at this point - we'll provide updates as soon as we have them.
Posted Mar 16, 2025 - 14:09 UTC

Update

Our upstream are reporting an issue with one of their switches which they are working to resolve.
Posted Mar 16, 2025 - 12:00 UTC

Update

We are continuing to push our upstream for a resolution.
Posted Mar 16, 2025 - 11:16 UTC

Update

We are continuing to push our upstream for a resolution.
Posted Mar 16, 2025 - 10:21 UTC

Identified

We have confirmed that this is an issue with our upstream and we have contacted them for a resolution.
Posted Mar 16, 2025 - 09:43 UTC

Investigating

We are currently investigating this issue.
Posted Mar 16, 2025 - 09:39 UTC
This incident affected: BL Networks (CHI1).