I noticed slow mv performance from the lower to the upper layer. Moving a 500MB directory containing small files which comes with the Docker image I am running in takes 15 seconds. Doing this operation on the same layer is instant (because the inode doesn’t change).
This leads me to believe that the kernel of CircleCI host systems doesn’t have CONFIG_OVERLAY_FS_REDIRECT_DIR enabled.
With this option enabled, a move between layers should be instant.
See section renaming directories in https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt