# Introspect image info from inside docker executor

**URL:** <https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620>\
**Category:** Build Environment\
**Tags:** envars, container-image, docker\
**Created:** [July 29, 2019, 9:48pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620 "2019-07-29T21:48:43Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [July 29, 2019, 9:48pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/1 "2019-07-29T21:48:43Z")

</div>

I’m trying to introspect into the image used by the current docker executor from inside the same container. I’d like to use this to derive a unique key to later recognise if the same docker image is used in future CI jobs; thus I could restore the cache that is unique and binary compatible to container’s image.

Knowing the digest or immutable identifier, aka the `RepoDigests` from `docker image inspect` would be one such example. A `CIRCLECI_` ENV including this sha256 could help ID and image.

> **[docker pull](https://docs.docker.com/engine/reference/commandline/pull/#pull-an-image-by-digest-immutable-identifier)**
>
> Description Pull an image or a repository from a registry Usage docker pull \[OPTIONS\] NAME\[:TAG|@DIGEST\] Options Name, shorthand Default Description --all-tags , -a Download all tagged images in the repository...

* * *

The build I’m working with at hand is very principled, with a wide set of system dependencies. This is automated, as source dependencies are self declared. But to reduce the setup time in CI jobs, we pre-install many such depnedies in the CI docker image.

I’d like to add build caching that reuses the build workspace shipped in the CI image, or using ccache between consecutive builds on a branch/PR. However, we update the CI image nightly to keep dependencies fresh, so I’d like be able to determine what docker image is being used, so I don’t load cache for an different docker image.

I’d like to avoid having to alter the CI image, such as baking in DateTime environment variable or other such nonces via image layers. Many images we use are outside our control or from external communities; thus generalizing to any docker image would be preferable.

---

<div class="post-metadata">

**Author:** ![halfer](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/halfer/32/9083_2.png) [@halfer](https://discuss.circleci.com/u/halfer)\
**Post date:** [July 29, 2019, 10:13pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/2 "2019-07-29T22:13:51Z")

</div>

Could you determine the folders that contain your dependencies, and do a `ls -lR` on them, and then feed that output to a hashing program?

---

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [July 29, 2019, 10:16pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/3 "2019-07-29T22:16:48Z")

</div>

They are a mix of system dependencies installed from debian packages, pip installs and who know what else from upstream parent images in the hierarchy. That’s why I’d just like to pin the cache key to the image digest and keep it simple while accounting for any environment changes.

---

<div class="post-metadata">

**Author:** ![halfer](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/halfer/32/9083_2.png) [@halfer](https://discuss.circleci.com/u/halfer)\
**Post date:** [July 29, 2019, 10:19pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/4 "2019-07-29T22:19:16Z")

</div>

You could have a look at your env vars to see if anything is available, such as an image digest. I’ve not tried it.

The other thing you can do - and which I would do myself - is to Dockerise the process, and do your build using Docker in Docker. To speed it up, you could use layer caching.

---

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [July 29, 2019, 10:36pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/5 "2019-07-29T22:36:07Z")

</div>

> [@halfer](#):
>
> You could have a look at your env vars to see if anything is available, such as an image digest. I’ve not tried it.

I looked, but there doesn’t seem to be a built-in environment variable specific to docker executors:

[https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables](https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables)

The docs do mention the benefits of pining docker images by digest, but do not offer a method for programmatically determining what it was from inside the container:

[https://circleci.com/docs/2.0/circleci-images/#using-a-docker-image-id-to-pin-an-image-to-a-fixed-version](https://circleci.com/docs/2.0/circleci-images/#using-a-docker-image-id-to-pin-an-image-to-a-fixed-version)

> [@halfer](#):
>
> Dockerise the process, and do your build using Docker in Docker. To speed it up, you could use layer caching.

That’s what we largely do already; aside from restoring the workspace first built by DockerHub as a cache, as we can’t determine the image’s immutable identifier to guarantee binary compatibility.

> <https://github.com/ros-planning/navigation2/blob/d253a3e0ed3799ad761e5dfef3a73750a340c814/.circleci/config.yml#L173-L180>

> <https://github.com/ros-planning/navigation2/blob/d253a3e0ed3799ad761e5dfef3a73750a340c814/Dockerfile#L52-L53>

---

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [August 13, 2019, 3:33am UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/6 "2019-08-13T03:33:56Z")

</div>

Any suggestions here?  
Would it be possible to retrieve the print out from the `Spin up Environment` step from the same job? As the print out from that first step does indeed include the `Digest: sha256:...` that I am after to determine if the image pulled for the executor is binary compatible with the cache I could restore.

---

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [August 19, 2019, 7:02pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/7 "2019-08-19T19:02:23Z")

</div>

From the comment section of a related idea:

> **[CircleCI Ideas](https://ideas.circleci.com/ideas/CCI-I-894)**
>
> Post your ideas and vote on those you would most like to see in CircleCI.

@ndintenfass mentions pipeline-level variables:

> **[Using pipeline variables in your configuration](https://circleci-public.github.io/pipeline-preview-docs/pipeline_variables_overview.html)**
>
> Preview documentation about pipeline variables and their uses

Would it be possible to use pipeline-level variables to convey the immutable digest of the docker image?  
E.g:

```yaml
<< pipeline.executor.docker.image.digest >>

```

Another issue I’m encountering is that while I’m separating my build and test stages into sequential jobs in my workflow, so that I may enable parallelism for only the testing stage, it is possible for the test job to pull a difference docker image compared to the build job it depends on, given the image tag on the docker registry could have been updated over the span of time for the build job. This of course brakes the caching strategy, as then docker executors are not the same across jobs. Would it be possible for circleci to first query the registry for the immutable identifier upon config expansion at the start of the workflow, so that all jobs would attempt to pull the exact same docker image.

For example, an `affix` parameter could be added to the `docker` config, that would express that the image should be affixed with the discovered digest when interpreting the config for the workflow.  
added E.g.:

```diff
executors:
  some_exec:
    docker:
      - image: library/ubuntu:bionic
+ affix: true
    working_directory: /foo/bar

```

This would expand the tag to include the digest, so that later jobs, either later in the same workflow or from re-run of failed jobs, would pull from the same repeatable image. Above would expand to this:

```diff
executors:
  some_exec:
    docker:
+ - image: library/ubuntu:bionic@sha256:d1d454df0f579c6be4d8161d227462d69e163a8ff9d20a847533989cf0c94d90
    working_directory: /foo/bar

```

I think this expansion should only be done once at the start of the workflow, and `Rerun from failed` should keep the same expansion from the workflow it originated from.

---

<div class="post-metadata">

**Author:** ![ruffsl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ruffsl/32/21229_2.png) [@ruffsl](https://discuss.circleci.com/u/ruffsl)\
**Post date:** [August 20, 2019, 6:50pm UTC](https://discuss.circleci.com/t/introspect-image-info-from-inside-docker-executor/31620/8 "2019-08-20T18:50:14Z")

</div>

> [@ruffsl](#):
>
> Would it be possible to retrieve the print out from the `Spin up Environment` step from the same job?

I was looking into the API and discovered that its does in fact expose a `steps->actions->output_url` field for a single job, allowing a job to introspect its own startup and acquire the sha.

> **`GET` Request:** Returns the full details for a single job. The response includes all of the fields from the job summary. [https://circleci.com/docs/api/#single-job](https://circleci.com/docs/api/#single-job)

This is a bit of a roundabout hack, so I’d still prefer a simpler method using pipeline variables.
