Unable to cache dependencies installed during test step

Build Environment
1

I’m seeing the problem with Clojure projects using either Boot or Leiningen, but it is probable that other tools also have the same problem.

As the cache is saved just after dependency step, only files installed during dependency step are cached. Many Boot and Leiningen plugins install more dependencies during run-time and these dependencies are not cached. These dependencies are not declared in a way that they could be downloaded beforehand. Only way to load these dependencies is to run the tests.

E.g. Travis saves the cache at the end of build so all files will be cached.

Is there maybe someway to manually add files to cache after test step?

One example here: https://circleci.com/gh/adzerk-oss/boot-cljs/2
The test step always downloads some dependencies as they are not cached.

Some examples of dependencies which are downloaded only when related task is run:

  • boot-http downloads http server when run
  • lein-cloverage downloads cloverage library only when run

Also, as both Boot and Lein are relatively slow to start, instead of downloading any dependencies in separate step, I would rather just invoke them once and let it do all the work on single run. Separate dependency step with Lein or Boot adds something like 5 to 15 seconds to build time…

Now that I think, I could probably just set test step as no operation and run tests on dependencies step. Is there any better way?

7 Likes
3

Same here, i’m using facebook’s jest to test my code and want to cache the preprocessor result, but i couldn’t

1 Like
4

I have a similar problem with Gradle. I’d like to have runtime dependencies like the in-memory database or database drivers cached as well.

5

I also have a the same problem. The tests do some work that I would like to cache.

6

I’m been trying to do the exact same thing with Jest without any luck.

1 Like
7

@scottbartell have you come across any solutions for properly caching Jest between builds?

8

Is there any downside to running the tests in the dependency phase? At least the lack of parallelism, I guess? (I don’t think Boot or Lein support it anyway, but they could. I don’t know about Jest.)

9

I managed to cache jest’s cache across builds.
Previously my jest tests took over 4 minutes, now they take 25 seconds. :thumbsup:

If anyone’s interested, here’s my setup:

circle.yml

machine:
  node:
    version: 7.5.0

dependencies:
  pre:
    - mkdir -p ./.tmp
    - yarn
  override:
    # hacky way to cache jest, put it in dependencies
    - ./node_modules/.bin/jest
  cache_directories:
    - ~/.cache/yarn
    - ./.tmp #jest cache

test:
  override:
  # use local runners instead of globals
    - ./node_modules/.bin/eslint src/*

jest config in package.json

"jest": {
		"cacheDirectory": ".tmp/jest"
	}

@reedflinch @scottbartell

1 Like