Cache clearing can be very confusing in 2.0

cache

#1

I just helped one our more experienced Ruby devs fix an issue in Circle CI by updating a random gem in the Gemfile. He has 10 years of experience in Ruby, but he needed me because I migrated the repo to Circle 2. The issue was that a Ruby gem created a shared object that linked to libmysqlclient.so.18. This now appears to be libmysqlclient.so in the latest ruby container. So the cached MySQL shared object failed.

The solution was to either change the cache key or bump a random gem to force the checksum to change and install the gem which builds against the new SO. I feel like this requires a bit too much understanding about how CircleCI 2.0 actually works. The cache is not really valid if the underlying OS and libs can change underneath it. You may have reasons for not having the “Rebuild without caches” button, and I may agree with your decision if I understood them. But we’ve been a customer of yours since you were beta and that particular developer has never asked me for help on Circle before.


#2

I had a thought: what if we could bake the container build signature into the cache key somehow? So if we end up picking up a new container the cache key changes?


#3

Thanks for sharing this info. The issue described above (and a number of related issues) happened because the default Ruby Docker images that we base our images off changed from Debian Jessie to Stretch: https://hub.docker.com/_/ruby/

To prevent breakage you can pin the image you specify by being more specific - e.g: - image: circleci/ruby:2.3-jessie

Pinning by point versions is also possible: - image: circleci/ruby:2.3.7-jessie
though breakages between point releases should be less common.

It’s possible to be even more specific and pin to the SHA of the image you want: - image: circleci/ruby:@sha256:b7c0bf563de65c69c5e8fcc7495d2ac24c08b3e94e9ed3228a4a89883f57f7d7


#4