Renaming a repository/fixing a broken cache


#1

I recently renamed my repo from “blah_project” to “project”

At first, I thought circle ci wouldn’t track the rename, so I unfollowed “blah_project” in Circle CI and then followed “project” again. However, it CircleCI knows the repo name was renamed and it appears that the cache directories are still being used from “blah_project”.

My circle.yml file looks similar to

version: 2
executorType: docker
containerInfo:
  - image: debian:8
    cmd: ["bash"]
stages:
  build:
    workDir: /vxl
    steps:
      - type: shell
        pwd: /
        name: Installing git
        command: |
          apt-get update
          apt-get install -y git

      - type: cache-restore
        key: vxl_src
      - type: checkout
      - type: cache-save
        key: vxl_src-{{ checksum "/vxl/core/vxl_version.h" }}
        paths:
          - /vxl/.git

When I “Build” I get the error message

Warning: Permanently added the RSA host key for IP address '104.192.143.1' to the list of known hosts.

repository does not exist.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Exited with code 128

The script from the checkout contains the section

if [ -e /vxl/.git ]
then
  cd /vxl
else
  mkdir -p /vxl
  cd /vxl
  git clone $CIRCLE_REPOSITORY_URL .
fi

git fetch --force origin sattel:remotes/origin/sattel

What I believe is happening is that the since I cache restore the /vxl/.git directory, the clone is not run, and the new remote url is not set (it’s still trying to fetch from the blah_project url which doesn’t exist anymore). I’ve run “Build without cache” many times, but every new branch I rerun on, uses the wrong cache again. “Build without cache” does not purge ALL cache for that project like I thought it does.

One simple solution to this would be changing your checkout script to.

if [ -e /vxl/.git ]
then
  cd /vxl
  git remote set-url origin $CIRCLE_REPOSITORY_URL
else
  mkdir -p /vxl
  cd /vxl
  git clone $CIRCLE_REPOSITORY_URL .
fi
  1. Is there a way to purge all the cache?
  2. Is cache always shared by name among all projects?
  3. Is there another way to solve this problem (other than changing the cache names in the circle.yml file)

#2

Not 100% from the UI at this time. A support rep can clear it, though.

No, it’s per-project.

Instead of caching the names, just remove the cache restore for a single build to overwrite the cache.


#3

Thanks @rohara

So when I “Build without cache” not only does it not load the cache, it doesn’t save it either?


#4

I’m not sure how technically tied that button is to all your caches in 2.0. It’s definitely meant more for 1.0 since the cache is so versatile and complex in 2.0.


#5

I think

key: vxl_src-{{ checksum "/vxl/core/vxl_version.h" }}

Was my problem then. Even though I has a successful build, it wasn’t resaving the cache because that version of the cache already existed. Changing it to

key: vxl_src-{{ checksum "/vxl/core/vxl_version.h" }}-{{ epoch }}

Build, and change back with cache restore removed worked. Thanks!

Is there any change that the checkout script might be updated to always restore the origin “just in case” for situations like this?


#6

Are you asking if the checkout step will automatically cache your source? I think that’s a great post for the feature requests :slight_smile:


#7

No, I was not asking that at all. (I’m actually 100% happy with choosing to cache the source dir myself by adding that to the circle.yml file)

The “step” of type “checkout” runs a script. That script contains the following code

if [ -e /vxl/.git ]
then
  cd /vxl
else
  mkdir -p /vxl
  cd /vxl
  git clone $CIRCLE_REPOSITORY_URL .
fi

git fetch --force origin sattel:remotes/origin/sattel

(Where /vxl is whatever the “workDir” was set to)

The script is already smart enough to anticipate someone optionally caching the source directory. So I’d say that’s a good thing. The problem I had could have been resolved with the small change to

if [ -e /vxl/.git ]
then
  cd /vxl
  git remote set-url origin $CIRCLE_REPOSITORY_URL
else
  mkdir -p /vxl
  cd /vxl
  git clone $CIRCLE_REPOSITORY_URL .
fi

git fetch --force origin sattel:remotes/origin/sattel

Since Circle CI is smart enough to know that when a repository was renamed it is still the same project, if any other user of Circle CI caches the source directory and renames the repo name on bitbucket (or github I imagine) they would run into the same cache problem I ran into here.

So I’m merely suggesting that by adding that one line to the checkout script, the checkout script will reassert that the origin is $CIRCLE_REPOSITORY_URL, and the cached source directory won’t break the build. I can’t think of any side effect, and it fixes this corner case.


#8

Awesome! Thanks for going into depth. I opened a ticket internally for this.


#9