# Persist\_to\_workspace vs \`save\_cache\` performance **URL:** \ **Category:** Tips, Tricks, and Hacks\ **Created:** [July 16, 2019, 11:35pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391 "2019-07-16T23:35:34Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![jamesarosen](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/jamesarosen/32/22628_2.png) [@jamesarosen](https://discuss.circleci.com/u/jamesarosen)\ **Post date:** [July 16, 2019, 11:35pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/1 "2019-07-16T23:35:35Z")
I’m trying to optimize our build. Up until recently, we had 1. A `dependencies` _task_ that checked out from git, ran `bundle install` and `yarn install`, and `yarn compile`, with caching around each of the four steps 2. A `dependencies` _job_ that just ran that task 3. Three jobs that ran in parallel after the first job. Each ran the `dependencies` task before running tests. Because of the workflow graph, they would (almost) always get cache hits. Based on [this article](https://circleci.com/blog/persisting-data-in-workflows-when-to-use-caching-artifacts-and-workspaces/), we recently changed that to use the workspace for persistence: 1. A `dependencies` _job_ that pulls from git, runs `bundle install` and `yarn install`, then `yarn compile`, and pushes the whole working directory into the workspace. This job uses caches for all the dependencies. 2. Three jobs that run in parallel after the first job. Each pulls the working directory from the workspace. It turns out that’s slower. Unfortunately the `persist_to_workspace` task doesn’t offer any transparency into what’s taking time but it definitely takes longer to persist to workspace than to write to cache. Additionally, [this post](https://discuss.circleci.com/t/glob-patterns-document-persist-to-workspace-paths/17442/2) suggests that `persist_to_workspace` can accept glob patterns, but I don’t see that information in the docs. So my questions: 1. Why is `persist_to_workspace` slower than `save_cache`? 2. Should we use cache or workspaces for git? For ruby gems? For node modules? For compiled assets? 3. Does `persist_to_workspace` accept a glob? Does it accept a _negative_ glob so I can ignore `./node_modules/` and `./.git/` but persist everything else? 4. Can you update the docs with general guidelines about when to use each tool? ---
**Author:** ![excenter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/excenter/32/18610_2.png) [@excenter](https://discuss.circleci.com/u/excenter)\ **Post date:** [July 22, 2019, 6:05pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/2 "2019-07-22T18:05:53Z")
Things I use caches for: - git directories (for the speed issues you’ve identified) - node\_modules folders (cannonical use case. a cache of node\_modules with a yarn.lock hash for the name is the official recommendation) Things I use workspaces for: - handing folders off to orbs that expect them (check out wealthforge/cypress@1.0.0, it’s just like regular cypress but with more features) - storing mutable variables within a build your `dependencies` job sound an awful lot like our `code_switch` one ``` code_switch: executor: default working_directory: /home/circleci/build-dir steps: - checkout: path: /home/circleci/build-dir # populate-deploy-branch and checkout-diffs are commands for handling building from a mono-repo - populate-deploy-branch - checkout-diffs - run: rm -rf .git # get rid of that big excess folder - save_cache: key: git-sha-{{ .Revision }} paths: - /home/circleci/build-dir ``` and on the receiving side, one of the mono-projects will do something like ``` ui_ruby: executor: default # you can work from any arbitrary file path, and caches will populate appropriately working_directory: /home/circleci/build-dir/clients/ui-ruby steps: # bail-if-current will gracefully kill the job if there are no changes on the detect-path conditionally - bail-if-current: detect-path: "clients/ui-ruby" # - restore_cache: key: git-sha-{{ .Revision }} # another command that will build and push a docker image. - docker-build-deploy: repository-image: "ui-ruby" docker-layer-caching: true # dlc is a life saver on ruby gems ``` hopefully this is helpful ---
**Author:** ![excenter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/excenter/32/18610_2.png) [@excenter](https://discuss.circleci.com/u/excenter)\ **Post date:** [July 22, 2019, 7:22pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/3 "2019-07-22T19:22:35Z")
Here is `persist_to_workspace` accepting 2 files, it’s not a literal glob, but get’s the same stuff done. ``` - persist_to_workspace: root: /tmp/dir paths: - FILE_TO_STORE - ALT_FILE ``` ---
**Author:** ![KeKs0r](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/keks0r/32/25967_2.png) [@KeKs0r](https://discuss.circleci.com/u/KeKs0r)\ **Post date:** [November 6, 2020, 2:11pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/4 "2020-11-06T14:11:43Z")
I just stumbled upon this issue after going down the same path @jamesarosen went through. Is there some kind of documentation or info about the persist\_to\_workspace performance? ---
**Author:** ![Stefano.Officine](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@Stefano.Officine](https://discuss.circleci.com/u/Stefano.Officine)\ **Post date:** [January 21, 2022, 10:53am UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/5 "2022-01-21T10:53:59Z")
I’m also interested in this performance issue. I would like to use cache and workspace according to the article mentioned by OP. If I were to follow the article I would save my node\_modules to workspace for the next jobs, while still potentially hitting cache for my yarn install from previous runs. In reality, saving node\_modules to cache and restoring it in every job is much faster. I think that since workspace has less features than cache (it just lasts one run and there only is one at a time) it should at least be faster. So either I’m not using it correctly (but I’m using it just like the users above), it is not functioning as expected or is not very useful. ---
**Author:** ![maschwenk](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/maschwenk/32/30065_2.png) [@maschwenk](https://discuss.circleci.com/u/maschwenk)\ **Post date:** [May 4, 2022, 4:37pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/6 "2022-05-04T16:37:59Z")
I’m finding the cache restore step to be painfully slow ![image](https://global.discourse-cdn.com/circleci/original/3X/7/b/7be886671edd864356f1d9bdbf73a9392ecd3e50.png) Downloading a 500 MiB file should take ~5 seconds on a gigabit connection, so what is the remaining ~20 seconds? ---
**Author:** ![rit1010](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rit1010/32/34738_2.png) [@rit1010](https://discuss.circleci.com/u/rit1010)\ **Post date:** [May 4, 2022, 5:43pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/7 "2022-05-04T17:43:39Z")
The docs indicate that Maven repositories are used as the storage location, so there is a lot more going on than just the raw copying of the files you want cached. The store is also likely to be a shared store so performance is likely affected by other customers. Without knowing what you are trying to do I can not add much, beyond the link to circleci’s docs on usage strategies in case you have not yet come across them. > **[Caching Strategies - CircleCI](https://circleci.com/docs/2.0/caching-strategy/)** > > This document is a guide to the various caching strategies available for managing dependency caches in CircleCI. Beyond the persist and cache options, depending on what equipment you have access to another option could be a self-hosted runner as that can then hold complete copies of file sets that are needed between builds. ---
**Author:** ![maschwenk](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/maschwenk/32/30065_2.png) [@maschwenk](https://discuss.circleci.com/u/maschwenk)\ **Post date:** [December 19, 2022, 5:01pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/8 "2022-12-19T17:01:54Z")
I just ripped out my workspace implementation with a `zstd` compression + upload to S3 solution and the workspace persistence aspect is 2-2.5x faster. So far it doesn’t seem like we need workspace features that we’re losing by this switch. ---
**Author:** ![sibelius](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sibelius/32/13734_2.png) [@sibelius](https://discuss.circleci.com/u/sibelius)\ **Post date:** [November 3, 2023, 3:21pm UTC](https://discuss.circleci.com/t/persist-to-workspace-vs-save-cache-performance/31391/9 "2023-11-03T15:21:16Z")
@maschwenk is your solution open source? can you share your final workflow ?