Have a look at the CircleCI cache system in the docs. You define a key, which is usually a checksum of your dependencies specification file, and you specify what path is to be cached.
I don’t use Gradle, but this should download everything in situ such that when you do the next install, Gradle connects to its repo but realises there is nothing to do, and exits early.