Can I use cache feature to skip expensive installations?


#1

Disclaimer: I just started to use CircleCI and did not grok all the details yet - sorry!

Imagine a configuration that defines

step-library:
  - &save-cache
      save_cache:
        key: 'v1/{{ .Environment.CIRCLE_JOB }}/{{ .Branch }}/{{ checksum ".circleci/config.yml" }}'
        paths: [ "~/cmake-3.9.1-Linux-x86_64" ]

  - &restore-cache
      restore_cache:
        keys:
          - 'v1/{{ .Environment.CIRCLE_JOB }}/{{ .Branch }}/{{ checksum ".circleci/config.yml" }}'
          - 'v1/{{ .Environment.CIRCLE_JOB }}/master/{{ checksum ".circleci/config.yml" }}'

  - &install-cmake-3-9-1
      run:
        name: Install CMake 3.9.1
        command: |
          curl -sSL https://cmake.org/files/v3.9/cmake-3.9.1-Linux-x86_64.tar.gz | sudo tar -xzC ~ 
[...]

jobs:
# ------------------------------------------------------------------------------
  build-with-clang-and-test:
    machine: true
    environment:
      [...]
      CMAKE_CXX_COMPILER: clang++5
      CMAKE_C_COMPILER: clang-5
    steps:
      - checkout
      # Caching does not work for directories where you need privileges, 
      # therefore we use it only for our local copy of CMake
      - *restore-cache
      - *install-cmake-3-9-1
      - *save-cache
      - *install-whatever
      - *init-git-submodules
      - *execute-cmake-and-make
      - *perform-unit-tests

I expected the cache feature to skip *install-cmake-3-9-1 on the second run if I only change a source code file and have circle-ci rerun after git push. But from the output I get the impression that install-cmake-3-9-1 is performed anyway.

So now I wonder whether I made a mistake or - if not - what caching is for in the end.

Also I noticed that cache restore does not work on directories where i need sudo rights for writing. Any hint how to circumvent this?

The final goal i want to achieve is that installing cmake, llvm/clang, etc. is cached in order to reduce reaction time for a test run. Any hint how to do this?

Thanks, Markus


#2

There was a small bug in the cache key generation, but my question is not resolved anyway

step-library:
  - &save-cache
      save_cache:
        key: 'v1/{{ .Branch }}/{{ checksum ".circleci/config.yml" }}'
        paths: [ "~/cmake-3.9.1-Linux-x86_64" ]

  - &restore-cache
      restore_cache:
        keys:
          - 'v1/{{ .Branch }}/{{ checksum ".circleci/config.yml" }}'
          - 'v1/master/{{ checksum ".circleci/config.yml" }}'

#3

If you have many dependencies to install, or any which are time consuming, I’d recommend building a custom Docker image and utilizing that with CircleCI 2.0. We have quite a few dependencies that we install, and this method has helped us greatly.


#4

Here’s an article on using custom Docker images with CircleCI 2.0:
https://circleci.com/docs/2.0/custom-images/


#5

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.