E: Could not get lock /var/lib/dpkg/lock

This error is so frustrating. Doing an apt-get install on a circleci/classic:201808-01 machine build leads to the following error several times per day:

#!/bin/bash -eo pipefail
sudo killall -9 apt-get || true && sudo apt-get install -y git python3 curl python-pip jq pip install awscli --upgrade --user

E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable) E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? Exited with code 100

The sudo killall -9 apt-get was a work around suggested in another thread, but does not work. The other suggestion I’ve seen is to downgrade to an earlier circleci/class machine version, but I can’t do that as I need circleci/classic:201808-01 for docker-compose 3.4 configs.

Does anyone have a solution to this that is not just adding a random sleep to the build?

I’m also experiencing this on my builds.

I remember trying to help someone with this, and I could not reproduce it. To fix it, I was planning to write a Bash loop that waits a few seconds, tries the operation again, and keeps going until a specified timeout.

It’s one of those things where it doesn’t happen every build, but when it happens, it seems to happen like 20 times in a row before it succeeds. This morning was particularly bad.

Really hoping to find a solution that doesn’t involve just putting in a sleep/retrying.

I hear ya, I have an intermittent issue to debug too!

I recommend you use a sleep method now, so that your builds become reliable now. You can implement a better fix if one comes available.

This occurs when as the error message states, another process is currently using Apt. Off the top of my head I would imagine that update-manager or Apt Cron is running when you try to use Apt yourself.

If that’s the case, then hopefully we can do something about it. I’ll pass this topic around internally.

@FelicianoTech yeah, I figured it was something like that. As you can see I tried doing a sudo killall -9 apt-get && before trying to run apt-get install, but perhaps one of those processes you referred to is not covered by this (maybe it’s just apt?). Regardless, installing software is most likely one of the very first steps any build is gonna perform, so it would be good if jobs didn’t need to worry about this and the vm itself proactively shut off any apt processes like you describe.

Thanks!

Have either of you tried using the newer ubuntu-1604:201903-01 image? Do you experience this issue when using that image?

I have not tried that image. I’m currently using circleci/classic:201808-01 because I read somewhere that that is the image I need to use for docker-compose 3.4 support. Does ubuntu-1604:201903-01 support docker-compose 3.4?

3.4? I think you mean Compose format 3.4.

The ubuntu-1604:201903-01 image contains Docker Compose v1.23.1 and Docker v18.09.3. Since Docker Compose file formats are backwards compatible, this image should work for Compose format 3.4.

For reference: https://docs.docker.com/compose/compose-file/

Yeah, sorry, I meant the 3.4 FORMAT (not version 3.4). Ok cool, I will give this image a try and report back…