Can't apt-get update reliably in builds

When I’m running builds at present I can’t do an sudo apt-get update reliably.

sudo apt-get update output

#!/bin/bash -eo pipefail
sudo apt-get update
Get:1 Index of linux/ubuntu/ focal InRelease [57.7 kB]
Get:2 Index of linux/ubuntu/ focal/stable amd64 Packages [25.7 kB]
Get:3 Index of /git-core/ppa/ubuntu focal InRelease [23.8 kB]
Get:4 Index of /git-core/ppa/ubuntu focal/main amd64 Packages [3,171 B]
Get:5 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Err:6 http://archive.ubuntu.com/ubuntu focal InRelease
Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out [IP: 185.125.190.39 80]
Err:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Unable to connect to archive.ubuntu.com:http: [IP: 185.125.190.39 80]
Err:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Unable to connect to archive.ubuntu.com:http: [IP: 185.125.190.39 80]
Ign:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Ign:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Ign:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Ign:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Ign:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Ign:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Ign:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Ign:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Ign:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Ign:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Ign:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Ign:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Ign:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Ign:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Ign:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Ign:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Ign:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Ign:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Ign:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Ign:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Err:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages
Connection failed [IP: 185.125.190.39 80]
Err:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out [IP: 185.125.190.39 80]
Err:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages
Unable to connect to security.ubuntu.com:http: [IP: 185.125.190.39 80]
Err:12 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
Unable to connect to security.ubuntu.com:http: [IP: 185.125.190.39 80]
Fetched 224 kB in 1min 45s (2,126 B/s)
Reading package lists… Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out [IP: 185.125.190.39 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-updates/InRelease Unable to connect to archive.ubuntu.com:http: [IP: 185.125.190.39 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-backports/InRelease Unable to connect to archive.ubuntu.com:http: [IP: 185.125.190.39 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/restricted/binary-amd64/Packages Connection failed [IP: 185.125.190.39 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/universe/binary-amd64/Packages Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out [IP: 185.125.190.39 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/multiverse/binary-amd64/Packages Unable to connect to security.ubuntu.com:http: [IP: 185.125.190.39 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/main/binary-amd64/Packages Unable to connect to security.ubuntu.com:http: [IP: 185.125.190.39 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.
CircleCI received exit code 0

Although this step succeeds any subsequent step needing to install packages then fails.

install fails

#!/bin/bash -eo pipefail
sudo apt-get install libmagickwand-dev imagemagick
Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Unable to locate package libmagickwand-dev

Exited with code exit status 100
CircleCI received exit code 100

I’m using the next-gen convenience image cimg/ruby:2.7.6-browsers.

My expectations on circleci using a next gen convenience images are that:

  • ubuntu packages would already be up to date so apt-get update would be a no-op (or unrequired)
  • circleci are managing their access to ubuntu (maybe having their own local mirror) given that they have no problem.

I’ve tried ssh-ing into the build to investigate and tried re-running sudo apt-get update but this mostly fails (though sometimes connects very slowly - it hasn’t yet succeeded altogether).

It may of course be a networking problem. Hard to diagnose without dnsutils package (which I can’t install…)

Oops - looks like this was a specific canonical outage:

https://status.canonical.com/#/incident/KNms6QK9ewuzz-7xUsPsNylV20jEt5kyKsd8A-3ptQFdhagDnjfaXXVpElWI5MToWkxoAA9XuP4NaRBzB2kxAQ==

And that cimg hasn’t been rebuilt for 2 years so it’s a bit out of date (Docker) - makes sense if I build my own Docker from that image.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.