The Docker image tag exists in Docker, but I am getting the following error message all the sudden. Does anyone know what’s going on?
Build-agent version 0.0.7546-3d4d9eb8 (2018-07-20T14:36:52+0000)
Starting container circleci/android:api-25-alpha
image cache not found on this host, downloading circleci/android:api-25-alphaError response from daemon: manifest for circleci/android:api-25-alpha not found
Just started encountering the same error on a PHP build.
I was using circleci/php:7.2.0-cli-node-browsers … which seems to longer exist, although I did a build with it earlier today… tried using the SHA256 hash and that didn’t work either.
Also tried, circleci/php:7.2-cli-node-browsers and circleci/php:7.2-node… confirmed that these tags are on Docker Hub (see https://hub.docker.com/r/circleci/php/tags/)
Same here. Seems to be a problem with either Circle builds or docker hub.
Some other circleci repo images like circleci/node works though.
I am also having this issue with circleci/php:7.1-node-browsers and have yet to find a php image that is working.
I went out for lunch, came back and the problem disappeared magically. It would be great if CircleCI could update on https://status.circleci.com/ for blocker issues like this. 
If anyone wants more stability, get the hash of a working image, and commit that in your image YAML key, in the form imagename@sha256:longhash. This is called “pinning to hash” and will lock your image to a specific version.
That said, I don’t know if that will protect you against (accidental) image deletion. The “alpha” thing makes me wonder if it should be used in production CI too… 
We’ve been using circleci/php:7.2.3-node-browsers image for a while. It started failing 22h ago with the same error:
Starting container circleci/php:7.2.3-node-browsers
image cache not found on this host, downloading circleci/php:7.2.3-node-browsers
Error response from daemon: manifest for circleci/php:7.2.3-node-browsers not found
The image doesn’t exist in docker hub: https://hub.docker.com/r/circleci/php/tags/. Seems get deleted?
Urgh, I expect it was automatically deleted then, based on CircleCI’s automated build+deploy systems. I see that issue pop up in the forum from time to time.
You could try 7.2-apache-node-browsers if you think your build will be stable right across 7.2.*, or 7.2.8-apache-node-browsers if you would rather fix to a specific version.
However, if you want to improve your stability, you can either:
Thank you for your suggestion. I will need to look into hash pinning.
Side note : The reason I am using “alpha” is only the alpha images are available now for Android. Docker. Yes i agree… Maybe this flakiness is expected because the Android images are in alpha.
One thing I am not sure of is whether hash pinning works if the image is deleted from a registry. I am not sure if the remote Docker keeps old versions behind, or whether they are inaccessible when the latest copy is removed. I’d advise caution here even if it initially works, since it may depend on caching that can still break.
As per my advice to gedex, once you obtain the image you need, you can push it to your own public/private registry and pull it from there. It’s slightly less convenient, and a bit slower, but maybe a bit more reliable - these are common trade-offs.
Hi folks,
Apologies for the inconvenience! Let me provide some context here and hopefully clear some things up.
We are slowly migrating our Docker Hub convenience images from manual to automated build repositories.
The benefits of automated builds are that we get integrated/automated Docker Hub support for Dockerfiles and READMEs.
So, for example, our Android repository now has a full description, which was automatically pulled from a GitHub repository, along with an example Dockerfile, so you can see exactly how these images are built.
We are making these changes to provide better visibility into and transparency around our convenience images—a top customer request for a number of months now.
Unfortunately, because, believe it or not, it is impossible to rename a Docker Hub repository, and impossible to convert a regular repository to an automated build repository, this process involved deleting and recreating the repository on Docker Hub.
We have been doing this successfully, without losing any of our images, by pulling all variants of a particular image into a long-running CircleCI job, then deleting and recreating the repository on Docker Hub, then pushing all variants back up to that new automated build repository.
@mayu you managed to catch us during the few minutes of downtime while this migration process was occurring… In the future, we will be working to ensure that these migrations occur during times when there is as little usage of these images as possible. Zero downtime is probably impossible, but we will work to come as close to zero as we can. And in the long run, the usability of these images will be greatly increased after these migrations.
@halfer unfortunately, as you say, pinning the sha256 of a specific variant doesn’t quite work here, as, when the images are deleted and re-pushed, they have new SHAs.
That said, in the case of the Android images, all the tags were restored shortly after your original post @mayu.
The PHP images are a separate concern @aautar @alvarollmenezes @bdwilliams @gedex… As you can see from Docker Hub, we have not migrated PHP to an automated build repository yet…
We are not sure why some tags are causing errors for folks, but we believe this is a Docker Hub issue and we are investigating. We are also working to re-build and re-push all old PHP tags, just in case—that work should be done in the next day or two.
Finally, just as a heads-up about what is coming down the line from CircleCI, we are working towards disentangling ourselves a bit from our dependency on Docker Hub, so that these kinds of issues do not impact end users in the future. We’ll keep you updated as to what that solution will look like!
And for future reference, we will be using the #convenience-images tag on Discuss to post relevant announcements about future changes to images.
Thanks everyone for your questions and concerns!
I also just want to second @halfer’s comment here. Hosting your own image is as simple as creating an account on Docker Hub or another Docker registry, then doing docker pull circleci/$image, re-tagging the image, and then docker push’ing it to your own Docker Hub account or private registry.
While we of course try to be as reliable and available as possible, speaking as a software developer, the idea of owning my own code/infrastructure just seems very intuitive—I would encourage folks to try out mirroring/hosting these images themselves.
From there, it’s a surprisingly easy path to extending them—adding additional features/dependencies, etc.—which all goes a long way towards making your CI/CD pipeline faster, smarter, and more efficient! 
We also have a great video on this subject (thanks @fernfernfern!):
For folks getting started and looking for something free of charge, I recommend GitLab. Private projects are free and each one comes with 10G of total storage, totalled over a Git repo and Docker registry. Docker Hub only allows one free registry, but GitLab lets you have as many as you like.
Data transfer, when pulling from GitLab to Circle, is a very healthy 1G/min, and has been rock solid (maybe one network failure in nine months, with at least one build/day, sometimes many). Of course, keep your images miniature, folks 
I’m missing the circleci/php:7.1.14-cli-node-browsers image as wel… When I updated to a newer version I wasn’t able to boot up my machine with circleci/php:7.1.20-cli-node-browsers (rest of the config is the same) I get this error message:
CircleCI was unable to start the container because the container entrypoint or command failed to start.
This typically means that the passed entrypoint or command is not found or is not valid. Try clearing the entrypoint/command values.
Original error: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:301: running exec setns process for init caused \"exit status 21\"": unknown
I’m using workflows, but my entry point is called “build”. Not sure where to look.
May we see your config.yml (in a Markdown code block please)? I’d look in the docker key to see if you’re adding any custom commands or entrypoints for this image. Or, if you are not customising those things, they should have defaults, defined by CircleCI - maybe there is something wrong with this image.
# This configuration was automatically generated from a CircleCI 1.0 config.
# It should include any build commands you had along with commands that CircleCI
# inferred from your project structure. We strongly recommend you read all the
# comments in this file to understand the structure of CircleCI 2.0, as the idiom
# for configuration has changed substantially in 2.0 to allow arbitrary jobs rather
# than the prescribed lifecycle of 1.0. In general, we recommend using this generated
# configuration as a reference rather than using it in production, though in most
# cases it should duplicate the execution of your original 1.0 config.
version: 2
# Define jobs
jobs:
build:
working_directory: ~/project/directory
shell: /bin/bash --login
# CircleCI 2.0 does not support environment variables that refer to each other the same way as 1.0 did.
# If any of these refer to each other, rewrite them so that they don't or see https://circleci.com/docs/2.0/env-vars/#interpolating-environment-variables-to-set-other-environment-variables .
environment:
CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
# In CircleCI 1.0 we used a pre-configured image with a large number of languages and other packages.
# In CircleCI 2.0 you can now specify your own image, or use one of our pre-configured images.
# The following configuration line tells CircleCI to use the specified docker image as the runtime environment for you job.
# We have selected a pre-built image that mirrors the build environment we use on
# the 1.0 platform, but we recommend you choose an image more tailored to the needs
# of each job. For more information on choosing an image (or alternatively using a
# VM instead of a container) see https://circleci.com/docs/2.0/executor-types/
# To see the list of pre-built images that CircleCI provides for most common languages see
# https://circleci.com/docs/2.0/circleci-images/
docker:
- image: circleci/php:7.1.20-cli-node-browsers
environment:
MYSQL_HOST: 127.0.0.1
MYSQL_DB: circle_test
MYSQL_USER: root
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_PASSWORD:
command: /sbin/init
- image: circleci/mysql:5.6-ram
environment:
MYSQL_USER: root
MYSQL_ALLOW_EMPTY_PASSWORD: true
steps:
# Machine Setup
# If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
# The following `checkout` command checks out your code to your working directory. In 1.0 we did this implicitly. In 2.0 you can choose where in the course of a job your code should be checked out.
- checkout
# Prepare for artifact and test results collection equivalent to how it was done on 1.0.
# In many cases you can simplify this from what is generated here.
# 'See docs on artifact collection here https://circleci.com/docs/2.0/artifacts/'
- run: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
# This is based on your 1.0 configuration file or project settings
- run:
working_directory: ~/project/directory
command: 'echo ''Europe/Amsterdam'' | sudo tee -a /etc/timezone; sudo dpkg-reconfigure -f noninteractive tzdata;'
# update apt
- run:
name: Update apt
command: sudo apt-get update
# add ruby
- run:
name: Install Ruby and other packages
command: sudo apt-get install ruby ruby-dev libicu-dev zlib1g-dev cmake libzip-dev libxml2-dev libcurl3 libsqlite3-dev mysql-client -y -q
# php extensions
- run:
name: Add required php extensions
command: sudo docker-php-ext-install soap pdo pdo_mysql pdo_sqlite bcmath mbstring
# Identify the system
- run:
name: PHP version
command: php -v
- run:
name: Node version
command: node -v
- run:
name: NPM version
command: npm -v
- run:
name: Ruby version
command: ruby -v
# Dependencies
# This would typically go in either a build or a build-and-test job when using workflows
# Restore the dependency cache
- restore_cache:
keys:
# This branch if available
- v1-dep-{{ .Branch }}-
# Default branch if not
- v1-dep-staging-
# Any branch if there are none on the default branch - this should be unnecessary if you have your default branch configured correctly
- v1-dep-
# This is based on your 1.0 configuration file or project settings
- run:
name: Create config file
command: php bin/config -e "empty" -f circle_ci -u root -p "" -n circle_test
# This is based on your 1.0 configuration file or project settings
- run:
name: Setup the environment
command: make setup-ci
# Save dependency cache
- save_cache:
key: v1-dep-{{ .Branch }}-{{ epoch }}
paths:
# These cache paths were specified in the 1.0 config
- vendor
- composer.lock
- ./node_modules
# This is based on your 1.0 configuration file or project settings
- run:
name: Write doctrine proxies
command: php vendor/bin/doctrine orm:generate-proxies
#
# Test commands
#
# This would typically be a build job when using workflows, possibly combined with build
# This is based on your 1.0 configuration file or project settings
- run:
name: Test code syntax
command: make quality
# This is based on your 1.0 configuration file or project settings
- run:
name: Run unit tests
command: make unit
# Your existing circle.yml file contains deployment steps.
# The config translation tool does not support translating deployment steps
# since deployment in CircleCI 2.0 are better handled through workflows.
# See the documentation for more information https://circleci.com/docs/2.0/workflows/
deploy_staging:
machine:
enabled: true
working_directory: ~/project/directory
# deploy using sh scripts
steps:
- checkout
- run: chmod +x deploy/staging.connect.sh
- run: ./deploy/staging.connect.sh
# deployment for master
deploy_master:
machine:
enabled: true
working_directory: ~/project/directory
# deploy using sh scripts
steps:
- checkout
- run: chmod +x deploy/production.connect.sh
- run: ./deploy/production.connect.sh
#teardown:
# If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
# Save test results
# - store_test_results:
# path: /tmp/circleci-test-results
# Save artifacts
# - store_artifacts:
# path: /tmp/circleci-artifacts
# - store_artifacts:
# path: /tmp/circleci-test-results
# Workflows
workflows:
version: 2
test_deploy:
jobs:
# build and test
- build
# deploy staging
- deploy_staging:
# must pass
requires:
- build
# only for staging
filters:
branches:
only:
- staging
# deploy master
- deploy_master:
# must pass
requires:
- build
# only for master
filters:
branches:
only:
- master@hugo.webiq: I’d remove the command from your circleci/php:7.1.20-cli-node-browsers image - it will be overriding the default, which may have changed in this image. The default should work.
Aside: I notice also you’re using an auto-generated YAML file. This is great to get you going but (personal opinion follows) I would not stick with it. There is great value in trimming out the cruft, removing comments that get in the way of readability, and rewriting it by scanning the reference docs and seeing what you do (not) need.
This is borderline ridiculous. Our PHP builds are STILL failing with this. You’ve kind of left your paying customers out to dry on this while we twiddle our fingers and wait for circleci to get back the images we’ve built our entire pipeline around. Almost a week without a good build. Our QA engineers are fumbling trying to test things on their LOCAL systems while our automated deployments to our environments are broken. And you’ve mentioned 1 or 2 days but that time has gone and passed.
Build-agent version 0.1.301-997b7e08 (2018-08-15T16:32:23+0000)
Starting container circleci/php:7.2.3-fpm-node-browsers
image cache not found on this host, downloading circleci/php:7.2.3-fpm-node-browsers
Error response from daemon: manifest for circleci/php:7.2.3-fpm-node-browsers not foundIf I were your QA engineer, I’d look to see how to build circleci/php:7.2.3-fpm-node-browsers myself, push it to a private registry, and then pull it from there. I seem to recall the images are open source, so the Dockerfile for these should be on GitHub.
How long do you think it would take to do my suggestion? My estimation (having not done it before) is 3-4 hours.
@tdondich @hugo.webiq All the PHP tags should be there now:
curl "https://registry.hub.docker.com/v1/repositories/circleci/php/tags" | jq -r '.[].name' | grep "7.2.3-fpm-node-browsers"
curl "https://registry.hub.docker.com/v1/repositories/circleci/php/tags" | jq -r '.[].name' | grep "7.1.20-cli-node-browsers"
If you’re having a specific issue, can you open a support ticket and CC me (rose@circleci.com)?
Thank you!