[Product Update] Speeding up code checkout

We are testing out a new default method to checkout code (blobless clones) which aims to speed up checking out source code for all users who use our built-in checkout command.

If you experience issues with cloning your repo when using the built-in checkout step, please comment on this thread or email oran@circleci.com.

My team’s builds started mysteriously and consistently failing today, with no obvious code-related or config-related cause. This is the only thing in CircleCI’s changelog and the only reason I suspect it to be the cause is that the timing matches up.

@abbyhowell can you let me know your org name and project name and send me a job link of a failure as well please?

You can send to oran@circleci.com and we can investigate/make sure you’re opted out until we figure out the root cause.

I’m seeing a similar issue where jobs have started randomly failing around a step that uses git diff. I assume it’s caused by this change since parts of commit history would be missing.

Is there a way to disable this option in config.yml to verify it’s the cause?

@davidalbers can you send me an email at oran@circleci.com with a job link please and we will opt you out while we investigate

Questions - why blobless and why change the default action?

I’m going to be in the minority as I use self-hosted runners, but this allows me to persist a copy of the git repo on the runner host with the result that the full git clone performed by the checkout command is an update rather than a full checkout for each future job run. As none of my repos have blobs the change does not impact my current usage, but for the future it would seem better to be able to choose how checkout operates, rather than just having a hardcoded selection.

If I were using the standard runner environment with a large repo I would likely be asking why you have not gone with a treeless clone as the local repos produced would be deleted at the end of the job anyway.

As a side note.

It has been a long time since I have had to worry about the checkout step, but in the past when trying to understand what actions were being performed it was very easy as the executed shell script was included in the output of the checkout step shown in the GUI.

At some point, this must have been dropped, but I am not aware of any other way to see what actions the script is performing. Obscuring such information may make the output look cleaner, but it does not do anything to help understand what is going on if there is an issue.

Looking at past logs this detail was lost in the last 2 months.

We have noticed that sonar cloud steps are failing in our builds while using blobless clones. This is a huger blocker for us as sonar is a gatekeeper for getting our PRs merged across projects.

Sonar community is starting to be aware of this issue, but they do not have any permanent solutions yet.

How can we revert to old checkout methods?

If you look at the step output from any workflow that is older than about 3 months you will be able to see the script run for the checkout step and so duplicate it.

I do not know why, but the latest update to the checkout step has also removed this detail as I noted in a post a few weeks ago.

Hi @brettjohnsen-sy,

I can help out with disabling blobless checkout for you. Send the name of your organization and the project(s) you’d like us to turn this off for to oran@circleci.com. Thanks!

We are also experiencing blocking issues related to Sonar as described above by brettjohnsen-sy.

[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin: (default-cli) on project xxx: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: org.eclipse.jgit.errors.MissingObjectException: Missing blob 95ec47e800000000000009ab539724 -> [Help 1]

Can this be turned off for our org?