# Docker Hub rate limiting - how to prepare? **URL:** \ **Category:** Build Environment\ **Created:** [August 25, 2020, 2:22am UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189 "2020-08-25T02:22:57Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![\_joe](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/_joe/32/25492_2.png) [@\_joe](https://discuss.circleci.com/u/_joe)\ **Post date:** [August 25, 2020, 2:22am UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/1 "2020-08-25T02:22:57Z")
Starting November 1st, Docker Hub will change ToS and rate limit anonymous users to 100 image pulls per hour. The CircleCI convenience images are all hosted on Docker Hub and looking at our own runs we see that the chance of having your images cached locally is close to 0%. Given that many builds run concurrently on your machines, we must assume that this limit will get exhausted quickly and builds will start failing due to that. What is the official recommendation to your users? - Do you have an invisible caching proxy for image pulls? - Do you have any plans to provide the convenience images from a CircleCI registry? - Should we start mirroring the images we use to our own registry? - Should we get a Docker Hub subscription? ---
**Author:** ![thekatertot](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/thekatertot/32/25169_2.png) [@thekatertot](https://discuss.circleci.com/u/thekatertot)\ **Post date:** [August 25, 2020, 9:36pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/2 "2020-08-25T21:36:41Z")
Thank you for posting @_joe! We are working on answers to all of these questions, and will update as soon as we can. We know how important this is. I will update you here when I know more. ---
**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\ **Post date:** [August 25, 2020, 11:03pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/4 "2020-08-25T23:03:58Z")
For our OSS project, while the number of pulls from CI jobs can exceed the current limits, the number of triggered CI workflows is well below this. It would be nice if dependent jobs in a workflow could be qued to the same CI worker such that the image needn’t be re-pulled if within a user defined time window for that executor. This would have the added benefit of speeding up consecutive jobs that use the same image by skipping the minutes of spin-up time in downloading the docker-image/file-caches for the next dependent job container, as well as help ensure all jobs in the same workflow are using the same sha of the docker image tag, resolving related issues where an image may update between jobs in the same workflow. > [@Introspect image info from inside docker executor](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620): > > I’m trying to introspect into the image used by the current docker executor from inside the same container. I’d like to use this to derive a unique key to later recognise if the same docker image is used in future CI jobs; thus I could restore the cache that is unique and binary compatible to container’s image. Knowing the digest or immutable identifier, aka the RepoDigests from docker image inspect would be one such example. A CIRCLECI\_ ENV including this sha256 could help ID and image. Th… Alternatively, CircleCI could erect a transparent registry between CI workers and DockerHub and interpret the same user defined time window parameter (e.g. 06h00m00sec) for that executor config to determine when the cache of the image-tag should be considered stale and re-pull the manifest before discarding old/unused layers. This would be different from the current Docker Layer Caching feature as it would be for caching the pulling of docker image for running the job, not the layers created when building in the job. [https://circleci.com/docs/2.0/docker-layer-caching/](https://circleci.com/docs/2.0/docker-layer-caching/) As an concrete example, we use a nightly job to update the CI image on DockerHub, then pull that image repeatedly for the graph of jobs in every workflow, sparing resources and maximize caching. [https://app.circleci.com/pipelines/github/ros-planning/navigation2?branch=main](https://app.circleci.com/pipelines/github/ros-planning/navigation2?branch=main) ---
**Author:** ![\_joe](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/_joe/32/25492_2.png) [@\_joe](https://discuss.circleci.com/u/_joe)\ **Post date:** [August 26, 2020, 12:05am UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/5 "2020-08-26T00:05:21Z")
The problem is that you have no knowledge what other users are doing on the shared hosts, just for the fun of it I could create a job to pull a hundred different images in a few seconds breaking builds for everybody that gets a build scheduled on the same host. We don’t pull that much in a single run either but the machines are shared and the limit is per IP. But before I start changing the config of hundreds of projects I’d prefer to know what the preferred way of going forward is. IMHO the cleanest solution would be for CircleCI to have their own registry. They could even intercept pulls of their own images transparently so that no change on the user side is necessary. ---
**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\ **Post date:** [August 26, 2020, 12:59am UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/6 "2020-08-26T00:59:54Z")
> [@\_joe](#): > > The problem is that you have no knowledge what other users are doing on the shared hosts, just for the fun of it I could create a job to pull a hundred different images in a few seconds breaking builds for everybody that gets a build scheduled on the same host. I was thinking of executors using docker images, as I don’t think CircleCI supports docker-in-docker. As soon as the first job is finished, the next dependent job running the same executor could take its place on the same host. This would require more intelligent scheduling, and would still be conditional upon workflow graph and sequential job ordering, but biasing jobs towards workers with a relevantly warm docker engine could save local bandwidth and cpu time for the CI cluster. > [@\_joe](#): > > the machines are shared and the limit is per IP. At first I was going to clarify that the limit was per docker hub repo: > **[Scaling Docker to Serve Millions More Developers: Network Egress | Docker](https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/)** > > Learn from Docker experts to simplify and advance your app development and management with Docker. Stay up to date on Docker events and new version But on second read, an independent 100pulls/6hr limit is applied to unauthenticated IPv4/v6 address? > For anonymous (unauthenticated) users, pull rates are limited based on the individual IP address. Ooftah! That makes image docker caching even more necessary, even just for free tier CI users. ---
**Author:** ![adrianmui](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/adrianmui/32/25470_2.png) [@adrianmui](https://discuss.circleci.com/u/adrianmui)\ **Post date:** [August 26, 2020, 11:50pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/7 "2020-08-26T23:50:58Z")
This is really important and if there’s no alternative it’s hard to imagine using Circle as our preferred CI tool. We’re not able to automate what we can already produce with our existing Jenkins infrastructure. Please advise. > [@Caching Docker Base Image in CircleCI](https://discuss.circleci.com/t/caching-docker-base-image-in-circleci/37067/2): > > We are in the same boat; speeding up builds by caching the env is a tradeoff with larger image sizes. Status: Downloaded newer image ... pull stats: download 2.226GiB in 26.833s (84.94MiB/s), extract 2.225GiB in 2m46.246s (13.71MiB/s) time to create container: 510ms ... Perhaps the upcoming pull rate limits in Docker Hub will encourage CI services to cache image layers. You’d think local caching would be cheaper than CPU time + inbound bandwidth, saving us timer_clock & moneybag. ---
**Author:** ![alexey](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/alexey/32/42_2.png) [@alexey](https://discuss.circleci.com/u/alexey)\ **Post date:** [August 27, 2020, 5:22pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/8 "2020-08-27T17:22:44Z")
Hi folks, I’m one of the product managers here at CircleCI. We’re taking the Docker Hub Terms of Service change seriously, and our engineering team is working on a detailed plan for how we’ll handle this change. Our goal is to minimise any disruption for our customers. We will inform all customers, if they are affected by the change, about any action they need to take with as much advance notice as possible. As I mentioned, we are working on a plan will share more details as soon as we have them. Expect an update from us next week. Kate and others from the team are, of course, available to hear your questions and concerns in the meantime. ---
**Author:** ![mqchau](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/mqchau/32/12804_2.png) [@mqchau](https://discuss.circleci.com/u/mqchau)\ **Post date:** [August 28, 2020, 4:57am UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/9 "2020-08-28T04:57:34Z")
I have an idea: CircleCI team use a dedicated paid DockerHub user to pull images that do not explicitly indicate username and password. This DockerHub user will be by itself, has no access to any private images. The cost is small: $60/year. There is no security risk here because it can only access public images anyway. This method will only apply to docker executor and not the docker commands like docker build. For docker build I think the developers need to login with some credentials to push anyway, then the limit is on that user. ---
**Author:** ![\_joe](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/_joe/32/25492_2.png) [@\_joe](https://discuss.circleci.com/u/_joe)\ **Post date:** [August 28, 2020, 2:30pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/10 "2020-08-28T14:30:09Z")
This could (and should) be considered account sharing. CircleCI should do the right thing here, they have basically shifted cost to Docker Hub for years and this is clearly one of the reasons why the terms are changing for everybody. ---
**Author:** ![\_joe](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/_joe/32/25492_2.png) [@\_joe](https://discuss.circleci.com/u/_joe)\ **Post date:** [September 10, 2020, 12:38pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/11 "2020-09-10T12:38:51Z")
Two weeks have passed, any update? ---
**Author:** ![thekatertot](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/thekatertot/32/25169_2.png) [@thekatertot](https://discuss.circleci.com/u/thekatertot)\ **Post date:** [September 10, 2020, 3:43pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/12 "2020-09-10T15:43:50Z")
We’re still actively working on this @_joe! ---
**Author:** ![\_joe](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/_joe/32/25492_2.png) [@\_joe](https://discuss.circleci.com/u/_joe)\ **Post date:** [September 25, 2020, 3:39pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/13 "2020-09-25T15:39:45Z")
One month has passed, we still haven’t heard from you. Time is running out! Edit: We are now also contacting our account executive, time is running out. ---
**Author:** ![thekatertot](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/thekatertot/32/25169_2.png) [@thekatertot](https://discuss.circleci.com/u/thekatertot)\ **Post date:** [September 25, 2020, 8:51pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/14 "2020-09-25T20:51:53Z")
@_joe Thanks for checking in. As you can imagine, there’s a bit of red tape for us to get through as we navigate this change. We’re absolutely working on it, and will have statements out as soon as we can. The team is hard at work on this! ---
**Author:** ![thekatertot](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/thekatertot/32/25169_2.png) [@thekatertot](https://discuss.circleci.com/u/thekatertot)\ **Post date:** [September 29, 2020, 8:11pm UTC](https://discuss.circleci.com/t/docker-hub-rate-limiting-how-to-prepare/37189/15 "2020-09-29T20:11:33Z")
Hi! We have our official update on Docker Hub rate limiting here! [Authenticate with Docker to avoid impact of Nov. 1st rate limits](https://discuss.circleci.com/t/authenticate-with-docker-to-avoid-impact-of-nov-1st-rate-limits/37567/5)