Attach workspace significantly slower on windows-server-2022-gui:2023.06.1

[RunningOstrich]

Edit: This is fixed as of the September image

On the latest windows-server-2022-gui:2023.06.1 windows image attaching workspaces from previous jobs is taking upwards of 6 minutes on our builds, when previously it would complete in about 40s. We have observed this behavior on builds other than ones from just today.

I ran the same build job with the following image versions back to back:

windows-server-2022-gui:2023.04.1 - Attach workspace took 38s
windows-server-2022-gui:2023.05.1 - Attach workspace took 37s
windows-server-2022-gui:2023.06.1 - Attach workspace took 5m37 and 6m7s (multiple runs)

In our build job, attach workspace downloads 3 .tar.gz files from previous jobs, of sizes ~25mb, ~425mb and ~100mb respectively. When I watched the logs for the slow attached workspace job I could see that the download of the jobs completed fairly quickly, however the “Applying workspace layers” step in particular seemed to take a large amount of time - even for the smallest 25mb layer.

I tried following the instructions to set up a ramdisk for windows from a Circle CI support article see if this made a difference, but saw no real change to the attach workspace behavior.

[RunningOstrich]

We’re also seeing this on the latest July image - 2023.07.1 with 1 run taking upwards of 7 minutes on attaching workspace.

Staying on 2023.05.01 still completes in about 40s but isn’t a great long term solution.

[jalex]

Hi @RunningOstrich,

I’ve added a note on our latest release post, which you’ve also commented on, but I wanted to summarize for you here:

It is currently believed that this upstream issue, issue 2, is the culprit.

If the upstream issue is not resolved, we will be utilizing git version 2.40.1 in the September release.

As for the 40s timing on the2023.05.1 image, would you be able to share what you have been seeing before, when it changed, and if that is the expectation you’re looking for?