[Quickbase US Status] Notice: Service Not Available (Unplanned) - Down Time

Incident
04/11/2023, 01:00pm EDT

[Quickbase US Status] Notice: Service Not Available (Unplanned) - Down Time

Status: closed
Start: 03/13/2023, 11:24am EDT
End: 03/13/2023, 12:35pm EDT
Duration: 1 hour 11 minutes
Affected Components:
Quickbase Service - US Region Quickbase Audit Logs - US Region Quickbase Automations - US Region Quickbase Billing - US Region Quickbase Pipelines - US Region Quickbase Platform Analytics - US Region Quickbase RESTful APIs - US Region Quickbase Sync - US Region Quickbase Webhooks - US Region
Update

03/13/2023, 11:24am EDT

03/13/2023, 11:24am EDT

As of 11:24 AM Eastern US Time, some functionality of the US Instance of the Quickbase platform is not working.  We are actively investigating and will post an update by 12:00 PM Eastern US Time.

Update

03/13/2023, 12:05pm EDT

03/13/2023, 12:05pm EDT

As of 12:05 PM Eastern US Time, we are continuing to see errors for certain requests to the platform.  We believe the issue is with our cloud hosting partner and we are in contact with them.  We will post an update within 30 minutes.

Update

03/13/2023, 12:33pm EDT

03/13/2023, 12:33pm EDT

As of 12:20 PM Eastern US Time, most functionality in the US Instance of the Quickbase platform is functioning normally.  However, we are still working to resolve issue with Pipelines, Sync (connected tables), Automations, and Webhooks.

Resolved

03/13/2023, 12:35pm EDT

03/13/2023, 12:35pm EDT

As of 12:35 PM Eastern US Time, functionality for Pipelines, Sync (connected tables), Automations, and Webhooks in the US Instance of the Quickbase platform is functioning normally.

At this time, all functionality is working normally.

This incident is closed.

We will publish a root cause once we have it.

Root Cause

04/11/2023, 01:00pm EDT

04/11/2023, 01:00pm EDT

Our hosting partner, Amazon AWS, had a network device failure that resulted in a loss of Internet connectivity for certain parts of the Quickbase platform including Pipelines, Sync (connected tables), Automations, and Webhooks.  The network is designed to be redundant and to withstand the failure of a single device by rerouting traffic to other network devices.  However, a bug in the operating system of the failed network device caused traffic to not reroute properly.

Amazon implemented monitoring on March 14, 2023 to more quickly detect traffic routing issues and they have deployed a fix to the operating system of the network device as of April 10, 2023.