We have a project that’s broken up into several packages. Instead of building every package on every commit, we’d like to only build and test the packages that have changes. Is there any way to support this behavior with CircleCI?
Thanks!
It is indeed possible.
Basically, you need a cache of some kind. CircleCI has a built-in one, but it is a bit limited (you provide a hash to it and it fetches a file, but you cannot inspect it, delete items from it, or zap it entirely). In my opinion it is better to use an S3 bucket, a Git repo, maybe even a Docker registry.
To do it, produce a package in your normal way. I don’t know what that means in your context, so let’s assume it is a tarball (package-a.tar.gz). When you produce it, push it to the cache with a hash so it can be retrieved for that hash.
If you have a one:one relationship between Git repos and packages, then you can use the Git commit hash. If you have several packages in a repo, then create an MD5 hash of the contents of each folder that produces each package.
Your packages will then look a bit like e866566849ab36485947df-package-a.tar.gz. When it comes to building a package, obtain the hash again, and try to fetch it from your cache. If it exists already, skip the build. You can do that with CircleCI 2.1 conditional steps, or a simple Bash conditional.
Thus, your CI will always run when you push your repo, but if a package does not need to be built, it will skip and finish in 10 seconds or so.
1 Like
Haven’t tried this yet, but it seems reasonable
1 Like