envzero - Investigating AWS Outage Affecting Our Services – Incident details

This incident is now resolved. Our cloud provider, AWS, has confirmed recovery for the underlying SQS issue that was affecting our deployment triggers. We have verified that all services, including PR-Plan and CD deployments, are now operating normally and all queued jobs have been processed.

We have narrowed down the scope of the performance degradation. We have confirmed that **PR-Plan and CD deployments** are not triggered. All other deployment triggers, including **UI, API, PR Comment, and scheduled tasks, are working normally.** This issue remains a direct result of the ongoing AWS outage in the US-EAST-1 Region. We are continuing to monitor the situation and will post an update as soon as performance for PR-Plan and CD deployments is restored.

Our core deployment service is currently experiencing degraded performance. Users may experience significant delays or errors when triggering new deployments. This is a direct result of the ongoing AWS outage affecting EC2 instances in the US-EAST-1 Region. Our team is actively monitoring the situation and AWS's recovery progress. We will post an update as soon as we see performance for our deployment service return to normal.

We are seeing significant signs of recovery. Our cloud provider, AWS, has reported that most requests are now succeeding and they are working through a backlog. Our internal end-to-end tests confirm that our application is back online and operating as expected. We are moving this incident to "Monitoring" as we continue to observe the situation to ensure full stability and recovery. We will post a final update once we are confident the issue is fully resolved.