Git submodule url isn't playing nice with the cache

Feedback & Bug Reports
1

I was directed here by the customer support widget.

I’m running into an issue with the cache. I sometimes change my remote urls in .gitmodules. I will be using a submodule, decide I need a new feature, fork it and update my repo to point to my fork. I will make the change and submit a pull request upstream but if/until it gets approved I need to use my fork to use this feature. At whichever point it gets received I will switch back to the source repo.

During this time, the cache of .git seems to cause problems. It will always point to the old url no matter what I try to do to fix it. I have tried adding git submodule sync && git submodule update --init --recursive to my post-checkout commands, I have tried deleting the submodule entirely from my repo, committing that, and then re-adding next commit. By the time it goes to check out my submodules it always complains that the commit I’m looking for is not found in the repo. I rebuild with ssh, log in, and check the remote for that submodule, and it’s still pointing to that old url, despite that url not existing anywhere in my codebase.

I even tried to change it in ssh and rebuild hoping it would cache the result of the ssh run, but no dice.

I had to wait almost 2 weeks to get my cache cleared so builds would start passing again (I’ve had to disable my check for a passing build in my deploy script this whole time).

So 1: is this a bug that you guys should/can fix? Or 2: what can I do to avoid this in the future? I will definitely be continuing to change .gitmodules urls so if it’s not something you can fix maybe I can do something differently. Or at least: can you add a way to clear my cache myself?

2

Sorry for the delay in getting back to you.

I don’t think we can offer anything better than clearing the cache right now, so please do contact us in support if you need that to be done for any of your projects.

Thanks.

3

Hi Chris,

We use a similar repo structure, and use the following:

checkout:
    post:
        - git submodule sync
        - git submodule update --init --recursive || (rm -fr .git/config .git/modules && git submodule deinit -f . && git submodule update --init --recursive)

Broken down, if submodule update fails (which it will, because the forked submodule commit is unknown to the original repo), then blow away and restore submodules. Hope that helps.

2 Likes
4

Thanks @jgraettinger for the blow it away solution.

I’ve actually investigated this problem (before finding this post). It’s very strange. I have sshed into the CI session, and typed git submodule sync several times, and ls and cat .gitconfig, The ubuntu user definitely appears to have write permissions (664), and yet the .git/submodule_path/config files remains unchanged.

However after the CI has failed, and I’m sshed for my 30 minutes, I can type in git submodule sync and the .git/submodule_path/config file is updated successfully and git submodule sync succeeds?!

Furthermore, the following worked…

  - |
    git submodule sync
    git submodule update --init || :
  - |
    git submodule sync
    git submodule update --init

And the first command fails (as we have observed), but I || : so that the build continues. And to my surprise, the second group of commands succeeds? It was all very confusing. It’s like there is some weird file sync issue going on. Moving to the dependencies-override section did not make it any better.

5

The solution has worked well for me so far as well. I also found it very strange, it was tough to track down.