Upgrading the underlying operating system for the Docker executor - 18.04 to 20.04

After successfully upgrading the kernel version of the operating system that is used to run customer containers as part of the Docker executor on CircleCI, CircleCI will be upgrading the full underlying operating system for jobs that use the Docker executor starting December 7, 2021. This change is critical to ensuring the underlying infrastructure that runs your jobs continues to provide reliable and performant execution.

This upgrade should be effectively invisible to customers. The current operating system version used is Ubuntu 18.04 and it will be upgraded to Ubuntu 20.04.

CircleCI is gradually rolling out this change for all Docker executor customers beginning on December 7, 2021.

Relevant Information:

  • CircleCI does not expect this to be a breaking change for the overwhelming majority of Docker executor jobs as the jobs are isolated in their own containers.
  • As part of this upgrade, the underlying kernel will be changing from 5.4 to 5.11
  • If there are other issues observed as a result of this change, please comment on this post immediately so they can be analyzed and addressed.
    • It is possible to view which kernel and operating system versions a job uses during execution by outputting “ uname -r”
  • This upgrade will not impact Remote Docker jobs

Please comment on this post or reach out to your support contact if there are any questions about this change. Thank you!

This upgrade is now complete for 100% of customers. This upgrade included a kernel upgrade from 5.4 to 5.11. If you are experiencing any issues with the new OS or kernel, please comment below.

I want to provide an update to all of our customers on how this went and the observed performance improvements. At CircleCI, we strive to make sure that the underlying infrastructure that runs our customer workloads is up to date, as is best practice across the industry. This roll-out of an updated underlying OS and associated kernel is one example that successfully enables us to maintain up-to-date systems and high Quality of Service.

The OS upgrade had an added benefit of improving spin-up performance for jobs that use our Docker executor. To understand the improvements, a little bit of context is required to understand how we define “slowness” in our systems. For reference, below are two of the subtasks that are a part of the “container spin-up” step and how we define “slow” for each one:

  • Extract: Extract steps are entirely dependent on the resources available to the VM running Docker. We define 10 MiB/s to be a good indicator that extract performance has degraded enough to cause noticeable user impact.
  • Create: For create steps, we set a target that containers should take no longer than 10 seconds to create.

The OS upgrade has resulted in 47% fewer tasks with a “slow" create step and 47% fewer tasks with a “slow" extract step across all jobs that use the Docker executor.

Feel free to reach out to me with any questions or observations you’re seeing with the latest upgrade.