[Resolved] [Hazard Notice] Intermittent database issues affecting APIs

We are experiencing intermittent database deadlocks with our databases powering the API servers. Most API calls are unaffected, as the servers can retry a failed database connection.

However, we are seeing some cases that have user impact.

  1. Magnum clusters might fail to spin up. Deleting the cluster and retrying works
  2. Some loadbalancers member might error out. We are cleaning them up and restoring them to a healthy state.

We are focused on fixing this and will provide more information when we can.

Login or Signup to post a comment