Error uploading archive: cache at :storage/caches

[jamesbraza]

In my save_cache step, I am getting the following printout:

Creating cache archive...
Uploading cache archive...

Error uploading archive: cache at :storage/caches/123-456/cache-id
.tar.gz failed: cache at :storage/caches/123-456/cache-id
.tar.gz failed: an unexpected storage error occurred (operation id: abc123)

What is the origin of this error? The message is not useful as there is little information included.

I found save_cache step fails with “Error uploading archive: MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size” which seems relevant. It’s message “MetadataTooLarge: Your metadata headers exceed the maximum allowed metadata size” is related to the message I am seeing “an unexpected storage error occurred”.

Can I get a confirmation that the core issue here is a too-large cache key, a too-large cache value, both, or something else entirely? I believe I have eliminated a too-large cache key being the issue. I believe it’s a too-large cache value included in paths.

Also, can CircleCI update their save_cache function to catch this .tar.gz error and present a more useful message?

[ChristopherDrum]

Came in here looking for a solution to this same problem. Just hit this today, and it’s occurring multiple times for us now. Seems strange to me that save_cache would fail the build; that should only be a warning at worst, IMHO.

[jamesbraza]

Alright good news… I’ve figured it out! It seems having a newline at the end of the cache key is the issue underlying this vague caching exception.

Old: use of >. This used to work, but apparently it doesn’t work any more, this will raise the error encountered (see original message).

          key: >
            deps1-{{ .Branch }}-{{ .Environment.CACHE_VERSION }}-
            {{ checksum "foo.txt" }}-
            {{ checksum "bar.txt" }}-
            ...
            {{ checksum "last.txt" }}

New: use of >-. This - (per YAML) strips the final newline in the output. This works for me!

          key: >-
            deps1-{{ .Branch }}-{{ .Environment.CACHE_VERSION }}-
            {{ checksum "foo.txt" }}-
            {{ checksum "bar.txt" }}-
            ...
            {{ checksum "last.txt" }}