I recently updated my local build-agent to latest, and while trying to run my configuration locally I’m now getting an error on code checkout. Everything was working fine before updating the build-agent. Any ideas on how to solve this problem?
Does this path exist? I don’t usually specify the path at all and just use whatever the default ends up being, I think its a much simpler approach rather than having to keep track of the current working directory.
Could you try to remove working_directory and the path argument? What happens then?
❯ tar --version
tar (GNU tar) 1.32
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
My cli should be the latest version (circleci update) just exits with “Done”.
And my checkout step doesn’t have a path:, just checkout.
Relevant logs:
====>> Checkout code
#!/bin/bash -eo pipefail
mkdir -p /home/circleci/project_name && cd /tmp/_circleci_local_build_repo && git ls-files -z | xargs -0 tar -c | tar -x -C /home/circleci/project_name && cp -a /tmp/_circleci_local_build_repo/.git /home/circleci/project_name
xargs: tar: terminated by signal 13
Error: Exited with code 125
Step failed
We can see the command being run here. Does this work if you try it manually? Two different OS versions, two different tar versions. If this were a issue with withe the CLI in general we would be getting a lot more reports, but it seems there is some form of edge case you both are hitting.
The more things we try, the more we can isolate the bug report and figure out a fix.
Or maybe a really large one? Will have to narrow it down by removing files, I’m afraid. Might be a good use case for https://git-scm.com/docs/git-bisect to aid in troubleshooting sanity
Unfortunately I’m not willing to bisect a 25k commit codebase
I think I have a lead, the xargs -0 doesn’t seem to be working. If I had -t to xargs to print the commands, I’m only shown two big tar calls, the first with the first 2994 files (counted the number of spaces + 1), and the second one with 732 files.
git ls-files|wc -l yields 3728 (so i’m missing 2 files, but maybe I’m not counting right)
It’s actually the number of files in the repo that triggers the error. Apparently 3700 files is too much
git ls-files -z | xargs -L 1 -t -0 tar -c | tar -x -C ~/tmp-circleci
will fail reliably for every repo, by limiting the number of files per xargs call to 1, thus making more than 1 call to tar, which is apparently not supported when piping.
Thank you so much for that great research and writeup. I’ll get a bug filed. (wonders what % of our customers have that many files AND do local checkouts)
Could you bring light to how it used to work before? Did @wlcopeland just happen to reach that limit of files and thought it was an update to the CLI that triggered the error, or was the way to transfer files really changed and used to work with a large number of files?
Side note, the xargs limit is system dependent, so maybe the other users ran into different limits, hard to know.
I…don’t blame you!
