Lingering apt timeouts after Ubuntu incident

Ubuntu’s status shows as Ok now, however, on highly parallel jobs, I’m still seeing some lingering slowness updating / installing packages from apt. Is it possible they’ve implemented some additional rate-limiting which is affecting CircleCI when a lot of requests are coming from a small number of nodes?

Also, a note, in case it’s helpful to others: using apt-get can cause indefinite hanging on certain types of timeouts that seems to ignore no_output_timeout. I talked to support, and, after some back and forth, they suggested that apt-get is rewriting the same line by default, and to add -q to the options for apt-get, which should suppress that behavior.

https://support.circleci.com/hc/en-us/articles/37474192881179-Resolving-Unable-to-connect-to-archive-ubuntu-com-403-Forbidden-Error-in-CircleCI

that’s not mentioned here, so figured I’d mention it in case it helps others.

I did also add a variation on this, to setup some caching.