Sharing xcode derived data folder across jobs / workflows

Build Environment Mobile
1

If we build multiple times the same code on our local system, it takes very less time as xcode caches the ouput from previous build command. It could be a good optimization for circle ci cache if it can provide additional handling for sharing xcode cache i.e. Derived data.

In current options available, at max we can try to include the derived data path too while saving cache. But! this wont work, as xcodebuild command also notices the file modification date, which changes while pulling the cache.

2

We’ve also had this issue; the other problem is that our DerivedData is huge, well beyond the 500MB it suggests for caching (I think it’s really hard for DerivedData for anything decently sizable to not be above that, to be honest)

3

We had the same issues. It often made the build take longer since we had to upload and download the files when we persisted parts of the workspace. We just build again for simplicity

4

Would it be possible for CircleCI restore_cache step to have an option to preserve file attributes to avoid xcodebuild to invalidate the cache?

1 Like
5

That (or something like that) would be AMAZING.

For what it’s worth, caching derivedData in a way that xcode wouldn’t immediately invalidate is something Bitrise claims to have solved: https://blog.bitrise.io/60-percent-faster-builds-force-xcode-use-caching

I have had little luck trying to do mimic the approach on CircleCI but would be interested to hear if anybody else has had better luck.

6

:wave: You could probably do something along the following lines:

- run: find ~/Library/Developer/Xcode/DerivedData -exec touch -t 2004210000 {} \; || true

If you place this step after your cache restore, then it will set all directories and files within DerivedData to the date specified (in YYMMDDHHMM format).

The || true at the end of the command will prevent the job from failing if DerivedData is empty (for example if there is no cache to restore).

I tested this myself and saw marginal gains, though the app I was testing is extremely small, so I would be interested to see if this benefits users with larger apps.

7

This seems to work. Just that the touch command takes around the same amount of time saved in the build step :wink:

So it would still be great if the files was restored in a correct way instead of having to go through that step.

Thanks.

8

Much faster solution would be to tar DerivedData in posix format, and revert git file modification time (around 30s on CI, depends of your project), see here for more: CircleCI 60% faster builds: use Xcode DerivedData for caching! | by Michał Zaborowski | Dec, 2021 | Medium