Git errors failing the build when deploying to Heroku

Deployments
1

We’ve been noticing Git errors at the end of our deployments in multiple Heroku environments. Apparently these may be related to pushing shallow clones

remote: Verifying deploy... done.
error: Could not read ce337a9c142cc6e96cff79b407359137080d08f9
fatal: Failed to traverse parents of commit be1f7a904e49b1f61fd9ea418f67d5a32d215148
error: failed to run repack
To [redacted]
 ! [remote rejected] fba4351cdc0b482743f180ad4b53cdadfc3b587d -> master (missing necessary objects)

We seem to have been able to fix it by doing some push/pull directly on the Heroku repositories and it all seems to be working again, but this has come up a few times so I wanted to ask if there is anything we can do to improve the reliability of our deployments.

I’ve heard that Heroku now has a deployment API and I was wondering if Circle has plans to integrate with that?

1 Like
2

That is exactly right—this is most likely caused by the repo being a shallow clone on the CircleCI side.

To work around this, please add the following to your circle.yml right before pushing to Heroku:

- "[[ ! -s \"$(git rev-parse --git-dir)/shallow\" ]] || git fetch --unshallow"

We’ll see if we can use the deployment API at some point in the future. Thanks.

EDIT: Wrapped the command in double quotes, as otherwise YAML complains about the pipe characters.

3

Thanks I’ll use that workaround, however is there any reason that this isn’t the default behaviour for Circle deploying to Heroku. It seems like a bad idea to have this intermittent bug around when there is a workaround.

This can create some pretty critical errors as the code is deployed to the server, but the build doesn’t run any post deploy commands like migrations. We actually had some minor data loss today as a result of this issue. So I would say it is pretty serious.

4 
- [[ ! -s "$(git rev-parse --git-dir)/shallow" ]] || git fetch --unshallow

I get an parsing error when adding this to circle.yml

expected chomping or indentation indicators, but found |.

5

We do that automatically if you are using the built-in Heroku integration, however we don’t do it if you are pushing to Heroku with git push syntax.

Where exactly in your circle.yml are you putting this command? It should go inside the commands section of the deployment block, if you are pushing to Heroku there:

deployment:
  production:
    branch: master
    commands:
      - "[[ ! -s \"$(git rev-parse --git-dir)/shallow\" ]] || git fetch --unshallow"
      - git push heroku master

Thanks.

EDIT: Wrapped the command in double quotes, as otherwise YAML complains about the pipe characters.

1 Like
6

Just realised that my suggestion was wrong, could you please wrap that command in double quotes as I now indicate above? Sorry for the confusion.

7

I’m also having this issue, but it just appeared. That makes me nervous because nothing has changed in our circle.yml or Heroku configs lately. We’re on build 2000+, and our first shallow clone failure was on Monday, and every subsequent build has failed.

Would that be an expected pattern of failure?

I’m switching over to the suggested solution now, just curious more than anything though.

8

Due to the way we maintain the source cache, it could have been the case where your cache contained all the necessary commits all the time, but this stopped being true on Monday. I am just guessing, but this does sound probable to me. Thank you for going for the proposed solution right away.

9

Has there been any updates on this issue? Just ran into it as well. I tried the fix above to no avail. Also tried rebuilding without the cache. We are using circle ci to deploy to our staging app on heroku, instead of production.

This is the error:

[[ ! -s "$(git rev-parse --git-dir)/shallow" ]] || git fetch --unshallow returned exit code 128

fatal: error in object: unshallow <SHA> Action failed: [[ ! -s "$(git rev-parse --git-dir)/shallow" ]] || git fetch --unshallow

Here is the deployment portion of my circle.yml:

deployment:
  staging:
    branch: master
    commands:
      - "[[ ! -s \"$(git rev-parse --git-dir)/shallow\" ]] || git fetch --unshallow"
      - git push git@heroku.com:<APP NAME> $CIRCLE_SHA1:refs/heads/master
10

This is also happening to me. Did you find a solution?

I opened a ticket just for this:

1 Like
11

Hi all,

any feedback on the git fetch --unshallow error ?

I have the same error as @westonhankins

git fetch --unshallow fatal: error in object: unshallow 0c2704a38fd3976a5c2fb52f0a5303eb621e7d7d

  • Googling is of no help unfortunately.
  • Rebuild without cache does not change neither.
  • git fetch --unshallow --force does not work.
  • logging in with ssh to edit the .git/shallow file is not a solution

I opened a support request but still have no feedback.

regs,
Yann

12

Hello,

Any updates on this?

I’m experiencing the same “could not read” and “could not traverse parent” errors discussed above, culminating in:

! [remote rejected] master -> master (shallow update not allowed)

After applying the fix suggested by @alexey, I receive a new error like @hmcfletch’s:

[[ ! -s "$(git rev-parse --git-dir)/shallow" ]] || git fetch --unshallow returned exit code 128

fatal: error in object: unshallow f93afe2711db6dce73b9a09f780002802d38f6d1 Action failed: [[ ! -s "$(git rev-parse --git-dir)/shallow" ]] || git fetch --unshallow`

I also tried rebuilding without the cache, and that did not work.

Any advice is greatly appreciated!

13

I would like to bump this thread as this problem is currently happening quite frequently across the entire dev team.

14

Hi,

maybe not related to our problem, but do any of you know how to clean/reset the CircleCI source cache ?

The CHECKOUT steps runs

  • restore source cache

  • checkout
    But then I have errors:

    error: Could not read f4e6d03ae3a8c70442298476231d34e935356d92
    fatal: Failed to traverse parents of commit 76d87db51d6dc6406afaf7cb974f9a1b2873d976
    error: failed to run repack

Could this be the origin of other problems ?
I have also noticed that the local repo contains the entire branches history (even the deleted ones), probably due to the source cache.
So, is there a way to clean or reset the source cache ? in order to start on fresh repo :wink:

thanks for your feedbacks,
Yann

Related post: CircleCi restore source cache questions & pb

15

Following up on this, this script to do unshallow fetching worked fine for a long time, but has recently stopped working. I’ve replaced it with:

[[ -e \"$(git rev-parse --git-dir)/shallow\" ]] || git fetch --unshallow

(changing ! -s to -e) and it works correctly again.