Elevated error rate for integration runner invocations in US region

Incident Report for Prismatic

Resolved

All indication are that performance across the platform has been restored. The incident has been resolved.
Posted Jun 27, 2023 - 02:54 UTC

Update

The team has completed rollout of the fix and things appear to be operating normally. We will continue to monitor the situation to ensure that no regressions appear.
Posted Jun 27, 2023 - 02:35 UTC

Update

We are continuing to monitor for any further issues.
Posted Jun 27, 2023 - 02:33 UTC

Update

Rollout of the expected fix continues and the team continues to monitor the performance of the platform.
Posted Jun 27, 2023 - 01:30 UTC

Update

Mitigation efforts continue on a failing piece of critical piece of infrastructure. API and application performance are negatively impacted and Integration Runner is operating at a lower capacity than normal. A fix has been implemented and rollout continues.
Posted Jun 27, 2023 - 00:44 UTC

Monitoring

Our team is continuing to monitor integration runner performance.
Posted Jun 27, 2023 - 00:20 UTC

Update

A fix has been put in place, and the integration runner service is returning to normal operation. Our engineering team is monitoring the service.
Posted Jun 26, 2023 - 23:49 UTC

Update

Our engineering team is continuing to implement a fix
Posted Jun 26, 2023 - 23:15 UTC

Update

An issue has been identified and our engineering team is implementing a fix.
Posted Jun 26, 2023 - 22:46 UTC

Update

The engineering team is continuing to investigate issues related to the integration runner.
Posted Jun 26, 2023 - 22:26 UTC

Update

We are continuing to investigate this issue.
Posted Jun 26, 2023 - 22:03 UTC

Investigating

We are currently investigating this issue.
Posted Jun 26, 2023 - 21:49 UTC
This incident affected: GraphQL API, Web App, and Integration Runner.