Instant git checkout in your workflow

Helpful Resources Tips, Tricks, and Hacks
,
1

We have a huge git repository (1+GB) with lots of history.

In a workflow with lots of jobs, it took 2+ mins on each job to do the checkout. It can lead to a lot of added time.

Here’s what you can do to only have to use “checkout” once:

Define a job that will checkout your repository and persist it in a workspace:

persist-checkout-to-workspace:
  docker:
    - image: python
  steps:
    - checkout
    - run: rm -rf .git
    - persist_to_workspace:
        root: /root
        paths:
          - project

Define a fast-checkout anchor:

fast-checkout: &fast-checkout
  attach_workspace:
    at: /root

You can now use the instant fast-checkout step in any job following persist-checkout-to-workspace in your workflow:

sample-job:
  docker:
    - image: node
  steps:
    - *fast-checkout
    - [...]

workflows:
  version: 2
  some-workflow:
    jobs:
      - persist-checkout-to-workspace
      - sample-job:
          requires:
            - persist-checkout-to-workspace
2 Likes
2

That looks excellent. I’d also say that a checkout probably does a full clone, when you probably only need a shallow clone, i.e. with a --depth=1. That would be very fast too.

3

I don’t think you can customize the checkout step, so you are stuck to the full clone.

4

Correct, but it’s optional in Circle 2.0. :slightly_smiling_face: So you can just swap it out for an ordinary run step containing the Git command you want.

5

And what about all the other stuff done in the checkout step?

6

Ooh, I didn’t know it did anything else. What else does it do?

7

You can look at what it does at any of your build, expand the checkout step.

8

The advice to remove the optional checkout step and swap it to a custom Git command has been given by a Circle employee here before, so it may be worth a try?

Anyway, best of luck with whatever you decide!

9

You can try to cache the .git source, so it will only fetch new updates

      - restore_cache:
          keys:
            - repo-source-{{ .Branch }}-{{ .Revision }}
            - repo-source-{{ .Branch }}-
            - repo-source-
      - checkout
      - save_cache:
          key: repo-source-{{ .Branch }}-{{ .Revision }}
      paths:
        - ".git"
10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.