Fairly new to CircleCI/Docker. I created a Linux 18.04 container with some embedded compilers and command line tools for our CI builds.
The build was failing due to a userns issue:
Original error: failed to register layer: Error processing tar file(exit status 1): Container ID 391824504 cannot be mapped to a host ID
I fixed the problem by modifying my ‘tar’ command line in my Dockerfile. When I attach to my container locally, I can confirm that that UID no longer exists in my container. I’ve pushed it to Docker hub as :latest:
However, my pipeline is still encountering the error above.
When I push my updated container, it appears that all the layers have new hashes:
The push refers to repository [docker.io/hpeyerl/arm-none-eabi-gcc]
246e0aab93f6: Layer already exists
5ea2e4541f9c: Layer already exists
76427ca2d9a4: Layer already exists
f497cfcb2e14: Layer already exists
e91eb9e93840: Layer already exists
03340fad5c7f: Layer already exists
c48583b1563c: Layer already exists
1c7f380da30a: Layer already exists
4c1e4a6fd1ee: Layer already exists
6f08023613ed: Layer already exists
8e28da4eacee: Layer already exists
e8ba47024730: Layer already exists
e0929b953d38: Layer already exists
21639b09744f: Layer already exists
latest: digest: sha256:1501a4f901db8d8a06e52a50de0f25d919077a4dd4edfc716d7a0831bf19ef99 size: 3265
But the build log doesn’t appear to reference any of those layers:
Starting container hpeyerl/arm-none-eabi-gcc
Warning: No authentication provided, using CircleCI credentials for pulls from Docker Hub.
image cache not found on this host, downloading hpeyerl/arm-none-eabi-gcc
latest: Pulling from hpeyerl/arm-none-eabi-gcc53061382: Already exists
d10c1005: Pulling fs layer
92e3b60d: Pulling fs layer
eddb586a: Pulling fs layer
86ee05b3: Pulling fs layer
633ffb9a: Pulling fs layer
eaf414af: Pulling fs layer
f3054705: Pulling fs layer
76df4692: Pulling fs layer
5aa4c6fa: Pulling fs layer
9faddd9d: Pulling fs layer
40bfb4d1: Pulling fs layer
271f70e9: Pulling fs layer
5aa4c6fa: Extracting [==================================================>] 115.4MB/115.4MBBCircleCI was unable to start the container because of a userns remapping failure in Docker.
This typically means the image contains files with UID/GID values that are higher than Docker and CircleCI can use.
Original error: CircleCI was unable to start the container because of a userns remapping failure in Docker.
This typically means the image contains files with UID/GID values that are higher than Docker and CircleCI can use.
Original error: failed to register layer: Error processing tar file(exit status 1): Container ID 391824504 cannot be mapped to a host ID
Also, when the build pipeline runs, the ‘last pulled’ time on Docker io doesn’t update, and is actually claiming the last pull was several hours prior. So I believe that’s telling me that CircleCI is using a cache of the old broken container even though a newer one exists.
Since the container never starts up, I can’t apparently run it with ‘ssh’ to look inside of it at what’s going on.
I must be misunderstanding something.
Edit: Yes. I was misunderstanding something. Even if the end result does not have a file with an invalid uid, none of the individual layers can have even a temp file with an invalid uid, even if that temp file is removed after it is created. So in my case, ‘tar --owner=0’ didn’t do what anyone would expect it to do. Since the container is run as ‘root’, tar creates the file with whatever the owner is inside the tarball ignoring --owner=0. The answer was to use --no-same-owner instead.
Solved. Though painful.