[Container Agent] Failed to copy circleci-agent into container "primary"

Build Environment
1

Yesterday I had a working container with the image cimg/deploy:2022.09 and today all containers give this error.

I’am running circleci/container-agent:1.0.17278-5c2bd95.

could not start task: failed to copy circleci-agent into container "primary", Error: error executing command /bin/sh -c cat > $HOME/circleci-agent : error executing command /bin/sh -c cat > $HOME/circleci-agent : command terminated with exit code 2

It looks like either a user/id error or something has changed with our permissions.

2

@petermein Taking a look with the team. I’ll report back when I have more. Can you share the version of container-agent you’re using please? It should be on the Self-hosted Runner Inventory screen from the left-hand nav of the CircleCI application

3

@petermein Actually, ignore me. I don’t need the version I see it in the output. We’re still taking a look.

4

One small, thing is there any way you can try downloading the latest helm chart from the new location and seeing if you’re still experiencing this issue? I don’t see any reason why that would be the cause but it’s one thing we can rule out.

the location with packagecloud is the new helm hosting location.

Can you also share a build link if possible? Feel free to send via direct message if you prefer.

5

@sebastian-lerner the issue was that we had a custom runAsUser value which I guess was required for us to run the container agent previously.

          securityContext:
            runAsUser: 1000

Completely removing this resolved the issue.

1 Like
6

@sebastian-lerner the solution @lucasvdh provided doesn’t seem to work for me.

The error message I’m seeing:

CircleCI failed to run this build, check your config. Try re-running the build and if this issue persists, open a Support ticket. Detail: failed to copy circleci-agent into container primary, Error: error executing command mkdir -p $HOME/.local/bin : error executing command /bin/bash -c set -o pipefail ; mkdir -p $HOME/.local/bin 21 | tee /proc/1/fd/1 : command terminated with exit code 1

Self Hosted Infra Info:

Circleci-runner version 3.0.19-5234-08c7340 (circleci-container-runner-container-agent)
Task-agent version 1.0.229736-54e0c9a4
Platform info Kubernetes-Runner (os: linux; arch: amd64; version: v1.27.9-eks-5e0fdde; virtualization: linux/amd64)

Chart Values:

agent:
  resourceClasses:
    org/some-resource-class:
      token: "TOKEN"
      spec:
        securityContext:
          runAsNonRoot: true
          runAsUser: 1000
  containerSecurityContext:
    runAsUser: 1000
    runAsGroup: 1000
    runAsNonRoot: true

CircleCI Config:

jobs:
  some_job:
    docker:
      - image: cimg/base:2021.11
    resource_class: org/some-resource-class
    steps:
      - checkout
      - run: ...

I also get the same error message with cimg/base:current. Any thoughts or ideas?

7

As you are using a self-hosted environment it is going to come down to what UID was assigned when you created the environment. The following writeup covers the detail

While a UID value of 1000 was the right answer for the original question and the result in the above writeup it does not mean it is the right answer for your configuration.

8

Thanks @rit1010 I had the wrong UID.