# \#post-incident-review **URL:** https://discuss.circleci.com/tag/post-incident-review/324.md [Latest](https://discuss.circleci.com/latest.md) · [Categories](https://discuss.circleci.com/categories.md) · [Tags](https://discuss.circleci.com/tags.md) --- ## [Post Incident Report: Oct 28, 2026 - Delays starting Gen 2 Docker jobs that use IP ranges](https://discuss.circleci.com/t/post-incident-report-oct-28-2026-delays-starting-gen-2-docker-jobs-that-use-ip-ranges/54959)
**Author:** [@anthonycastro](https://discuss.circleci.com/u/anthonycastro)\ **Replies:** 0\ **Last updated:** [October 31, 2026, 12:21am UTC](https://discuss.circleci.com/t/post-incident-report-oct-28-2026-delays-starting-gen-2-docker-jobs-that-use-ip-ranges/54959 "2026-10-31T00:21:34Z")
Summary From 14:20 to 19:08 UTC on October 28, customers running Gen 2 Docker jobs using IP Ranges saw increased wait times for their jobs to start. From 14:20 until 17:02, the delays were intermittent, and from 17:02 to… --- ## [Post Incident Report: Oct 21, 2026 - Increased wait times for Gen 2 Docker jobs](https://discuss.circleci.com/t/post-incident-report-oct-21-2026-increased-wait-times-for-gen-2-docker-jobs/54940)
**Author:** [@anthonycastro](https://discuss.circleci.com/u/anthonycastro)\ **Replies:** 0\ **Last updated:** [October 22, 2026, 11:33pm UTC](https://discuss.circleci.com/t/post-incident-report-oct-21-2026-increased-wait-times-for-gen-2-docker-jobs/54940 "2026-10-22T23:33:52Z")
Summary From 15:44 to 20:55 UTC on October 21, customers running Gen 2 Docker jobs saw increased wait times for their jobs to start. Delays were longest on the 2xlarge resource class, where jobs waited up to 20 minutes. … --- ## [Post Incident Report: Oct 9, 2026 - Delay in data appearing in UI and API](https://discuss.circleci.com/t/post-incident-report-oct-9-2026-delay-in-data-appearing-in-ui-and-api/54919)
**Author:** [@amitm](https://discuss.circleci.com/u/amitm)\ **Replies:** 0\ **Last updated:** [October 12, 2026, 10:40pm UTC](https://discuss.circleci.com/t/post-incident-report-oct-9-2026-delay-in-data-appearing-in-ui-and-api/54919 "2026-10-12T22:40:18Z")
Summary Between 11:15 UTC and 23:47 UTC on October 9, CircleCI customers experienced hours-long delays and failures across pipelines, workflows and jobs, along with delays in the CircleCI UI, API and notifications. Fro… --- ## [Post Incident Report: Oct 1 to 8, 2026 - Delays and failures starting jobs](https://discuss.circleci.com/t/post-incident-report-oct-1-to-8-2026-delays-and-failures-starting-jobs/54915)
**Author:** [@anthonycastro](https://discuss.circleci.com/u/anthonycastro)\ **Replies:** 0\ **Last updated:** [October 9, 2026, 10:30pm UTC](https://discuss.circleci.com/t/post-incident-report-oct-1-to-8-2026-delays-and-failures-starting-jobs/54915 "2026-10-09T22:30:04Z")
Summary Between October 1 and October 8, 2026, CircleCI customers experienced five separate incidents in which jobs waited longer than normal to start, or failed to start. Three of the incidents were driven by limited in… --- ## [Post Incident Report: Oct 6, 2026 - Delays in pipelines, workflows, UI data, and notifications](https://discuss.circleci.com/t/post-incident-report-oct-6-2026-delays-in-pipelines-workflows-ui-data-and-notifications/54908)
**Author:** [@amitm](https://discuss.circleci.com/u/amitm)\ **Replies:** 0\ **Last updated:** [October 7, 2026, 10:19pm UTC](https://discuss.circleci.com/t/post-incident-report-oct-6-2026-delays-in-pipelines-workflows-ui-data-and-notifications/54908 "2026-10-07T22:19:29Z")
Summary On October 6, 2026, from 15:30 to 20:30 UTC, CircleCI customers experienced delays creating pipelines and starting workflows and jobs. From 15:48 to 16:38 UTC, no workflows or jobs started, and pipelines triggere… --- ## [Post Incident Report: July 20, 2026 - Errors with Github API Delaying Workflows](https://discuss.circleci.com/t/post-incident-report-july-20-2026-errors-with-github-api-delaying-workflows/54816)
**Author:** [@amitm](https://discuss.circleci.com/u/amitm)\ **Replies:** 0\ **Last updated:** [July 23, 2026, 9:23pm UTC](https://discuss.circleci.com/t/post-incident-report-july-20-2026-errors-with-github-api-delaying-workflows/54816 "2026-07-23T21:23:15Z")
Summary On July 20, 2026 from 00:22 to 01:45 UTC, CircleCI customers using our GitHub integrations experienced failures running pipelines and experienced workflows getting stuck. During this incident, some pipelines trig… --- ## [Post Incident Report: 2026-07-13 Network Routing Issues Cause Intermittent macOS Job Failures](https://discuss.circleci.com/t/post-incident-report-2026-07-13-network-routing-issues-cause-intermittent-macos-job-failures/54785)
**Author:** [@rob\_hartsock](https://discuss.circleci.com/u/rob_hartsock)\ **Replies:** 0\ **Last updated:** [July 16, 2026, 2:00am UTC](https://discuss.circleci.com/t/post-incident-report-2026-07-13-network-routing-issues-cause-intermittent-macos-job-failures/54785 "2026-07-16T02:00:47Z")
Summary From 18:00 UTC on July 13, 2026 to 23:18 UTC on July 14, 2026, some customers running jobs on CircleCI’s macOS fleet experienced jobs that hung or failed to progress during GitHub fetch steps. For these customers… --- ## [Post Incident Report: 2026-07-14 Bitbucket OAuth Change Causes Login Failures and Workflow Issues](https://discuss.circleci.com/t/post-incident-report-2026-07-14-bitbucket-oauth-change-causes-login-failures-and-workflow-issues/54781)
**Author:** [@rob\_hartsock](https://discuss.circleci.com/u/rob_hartsock)\ **Replies:** 0\ **Last updated:** [July 14, 2026, 10:36pm UTC](https://discuss.circleci.com/t/post-incident-report-2026-07-14-bitbucket-oauth-change-causes-login-failures-and-workflow-issues/54781 "2026-07-14T22:36:00Z")
Summary From 01:33 UTC to 05:18 UTC on July 14, 2026, customers with a Bitbucket-linked identity were unable to log in to CircleCI, and some customers experienced workflows that failed to start or update, due to a change… --- ## [Post Incident Report: July 2, 2026 - Outage impacting web experience and pipelines starting](https://discuss.circleci.com/t/post-incident-report-july-2-2026-outage-impacting-web-experience-and-pipelines-starting/54759)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [July 10, 2026, 7:03pm UTC](https://discuss.circleci.com/t/post-incident-report-july-2-2026-outage-impacting-web-experience-and-pipelines-starting/54759 "2026-07-10T19:03:50Z")
Summary On July 2, 2026, from 15:12 UTC to 16:44 UTC, some CircleCI customers experienced failures starting pipelines and difficulty accessing the CircleCI UI, including an inability to log in. Customers triggering pipel… --- ## [Post Incident Report: June 23, 2026 - Delays starting Docker jobs](https://discuss.circleci.com/t/post-incident-report-june-23-2026-delays-starting-docker-jobs/54750)
**Author:** [@anthonycastro](https://discuss.circleci.com/u/anthonycastro)\ **Replies:** 0\ **Last updated:** [June 27, 2026, 12:15am UTC](https://discuss.circleci.com/t/post-incident-report-june-23-2026-delays-starting-docker-jobs/54750 "2026-06-27T00:15:08Z")
Summary On Tuesday, June 23, 2026, our engineering team identified that a single Docker job cluster was operating below expected performance levels. Our infrastructure is organized into routing groups, each responsible f… --- ## [Post Incident Report: December 2, 2025 - Jobs not starting, Jobs stuck in running state, Pipelines page not loading](https://discuss.circleci.com/t/post-incident-report-december-2-2025-jobs-not-starting-jobs-stuck-in-running-state-pipelines-page-not-loading/54291)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [December 18, 2025, 9:37pm UTC](https://discuss.circleci.com/t/post-incident-report-december-2-2025-jobs-not-starting-jobs-stuck-in-running-state-pipelines-page-not-loading/54291 "2025-12-18T21:37:31Z")
Summary All timestamps are in UTC On December 2, 2025, at 16:20, we deployed a code change to workflow status computation that introduced a race condition in how workflows are terminated. This caused the following impac… --- ## [Post Incident Report: May 1, 2025 - Delays in starting some jobs](https://discuss.circleci.com/t/post-incident-report-may-1-2025-delays-in-starting-some-jobs/53307)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [May 22, 2025, 4:02pm UTC](https://discuss.circleci.com/t/post-incident-report-may-1-2025-delays-in-starting-some-jobs/53307 "2025-05-22T16:02:14Z")
Summary On May 1, 2025, from 22:20 UTC to May 2, 2025 02:00 UTC, CircleCI customers experienced delays in starting most jobs. Jobs affected were contained to the following resource classes: Docker large, Docker medium, D… --- ## [Post Incident Report: April 4, 2025 - CircleCI UI Loading & build triggering issues](https://discuss.circleci.com/t/post-incident-report-april-4-2025-circleci-ui-loading-build-triggering-issues/53208)
**Author:** [@amitm](https://discuss.circleci.com/u/amitm)\ **Replies:** 0\ **Last updated:** [May 2, 2025, 2:20pm UTC](https://discuss.circleci.com/t/post-incident-report-april-4-2025-circleci-ui-loading-build-triggering-issues/53208 "2025-05-02T14:20:47Z")
Summary On April 4, 2025, from 00:16 to 01:49 UTC (approximately 1 hour and 33 minutes), CircleCI experienced a service disruption affecting both our user interface and build capabilities. During this time, customers wer… --- ## [Post Incident Report: April 4, 2025 - Delays in starting workflows](https://discuss.circleci.com/t/post-incident-report-april-4-2025-delays-in-starting-workflows/53113)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [April 16, 2025, 1:41pm UTC](https://discuss.circleci.com/t/post-incident-report-april-4-2025-delays-in-starting-workflows/53113 "2025-04-16T13:41:43Z")
Summary On April 3, 2025, from 22:08 UTC to 23:45 UTC, CircleCI customers experienced increased latency and some failures with starting and canceling workflows and jobs. During this time customers may have experienced de… --- ## [Post Incident Report: January 23, 2025 - Some commit status updates were not published](https://discuss.circleci.com/t/post-incident-report-january-23-2025-some-commit-status-updates-were-not-published/52697)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [January 31, 2025, 9:42pm UTC](https://discuss.circleci.com/t/post-incident-report-january-23-2025-some-commit-status-updates-were-not-published/52697 "2025-01-31T21:42:42Z")
Summary On January 23, 2025, from 19:48 UTC to 20:43 UTC, customers using CircleCI GitHub OAuth and Bitbucket projects stopped receiving commit status updates. This was due to a code change deployed at 19:48 UTC that neg… --- ## [Post Incident Report: January 22, 2025 - Delays starting and canceling workflows and jobs, and viewing them in the UI](https://discuss.circleci.com/t/post-incident-report-january-22-2025-delays-starting-and-canceling-workflows-and-jobs-and-viewing-them-in-the-ui/52696)
**Author:** [@shelly](https://discuss.circleci.com/u/shelly)\ **Replies:** 0\ **Last updated:** [January 31, 2025, 9:37pm UTC](https://discuss.circleci.com/t/post-incident-report-january-22-2025-delays-starting-and-canceling-workflows-and-jobs-and-viewing-them-in-the-ui/52696 "2025-01-31T21:37:52Z")
Summary From January 21, 2025 at 23:50 UTC to January 22, 2025 at 00:56 UTC, CircleCI customers experienced increased latency with starting and canceling workflows and jobs, and experienced delays and difficulty viewing … --- ## [Post Incident Review: July 2nd Errors loading CircleCI](https://discuss.circleci.com/t/post-incident-review-july-2nd-errors-loading-circleci/51497)
**Author:** [@kami](https://discuss.circleci.com/u/kami)\ **Replies:** 0\ **Last updated:** [July 19, 2024, 4:04pm UTC](https://discuss.circleci.com/t/post-incident-review-july-2nd-errors-loading-circleci/51497 "2024-07-19T16:04:06Z")
Summary and Impact: On July 2, 2024, at 07:34 UTC, a deployment change in production using the MicroFrontends (MFEs) migration Ingress guide caused Kong routing to break for the circleci.com host. This led to the unavai… --- ## [Incident Report: 2023-03-14 - Delays Starting Jobs](https://discuss.circleci.com/t/incident-report-2023-03-14-delays-starting-jobs/47555)
**Author:** [@naoya](https://discuss.circleci.com/u/naoya)\ **Replies:** 0\ **Last updated:** [March 24, 2023, 1:34am UTC](https://discuss.circleci.com/t/incident-report-2023-03-14-delays-starting-jobs/47555 "2023-03-24T01:34:20Z")
Incident Report: 2023-03-14 - Delays Starting Jobs Summary: On March 14, 2023, at approximately 18:00 UTC, all customers’ pipelines began to experience delays in starting, ultimately resulting in jobs not running. This i… --- ## [Incident Report: November 8, 2021 - Jobs stuck in a "not running" state](https://discuss.circleci.com/t/incident-report-november-8-2021-jobs-stuck-in-a-not-running-state/41890)
**Author:** [@yannCI](https://discuss.circleci.com/u/yannCI)\ **Replies:** 0\ **Last updated:** [November 11, 2021, 1:53pm UTC](https://discuss.circleci.com/t/incident-report-november-8-2021-jobs-stuck-in-a-not-running-state/41890 "2021-11-11T13:53:47Z")
Incident Report: November 8, 2021 - Jobs stuck in a “not running” state Summary Customer jobs were blocked in a “not running” state from approximately 18:50 UTC to 19:43 UTC on Monday, November 8, 2021, due to a database… --- ## [Incident Report: October 27, 2021 - VMs failing to be created](https://discuss.circleci.com/t/incident-report-october-27-2021-vms-failing-to-be-created/41769)
**Author:** [@JonathanC503](https://discuss.circleci.com/u/JonathanC503)\ **Replies:** 0\ **Last updated:** [November 2, 2021, 9:31pm UTC](https://discuss.circleci.com/t/incident-report-october-27-2021-vms-failing-to-be-created/41769 "2021-11-02T21:31:05Z")
Summary Customers experienced lengthy delays in job starts and stuck builds from 17:00 UTC October 27 to 01:16 UTC October 28. All jobs were affected, starting with machine jobs and continuing into docker executor as th… --- ## [Incident Report: September 27, 2021 - Delays in Workflows](https://discuss.circleci.com/t/incident-report-september-27-2021-delays-in-workflows/41443)
**Author:** [@JonathanC503](https://discuss.circleci.com/u/JonathanC503)\ **Replies:** 0\ **Last updated:** [October 5, 2021, 9:40pm UTC](https://discuss.circleci.com/t/incident-report-september-27-2021-delays-in-workflows/41443 "2021-10-05T21:40:12Z")
Incident Report: September 27, 2021 - Delays in Workflows Summary: On September 27, 2021, from 07:05 to 14:46 UTC CircleCI customers experienced delays in workflows. Due to our efforts to fix the initial incident, some c…