I am using Gitflow to manage code commits, so I have 2 working branches:
Upon a successful build, I also push a built docker image to Docker Hub repo. Now let’s see what will happen during Gitflow stages:
- New feature: I create a new feature branch with “feature” prefix. When merged to “develop” branch, assuming that the software version is something like “1.0.0-dev.1”. A docker image with the same tag “1.0.0-dev.1” is built and push. No issue so far.
- Release: A release branch is created from “develop”, and when ready, will be merge to both “main” and “develop”. Now, both “main” and “develop” branches will trigger Docker build with the same tag, and will push the same image to Docker Hub twice.
- Hotfix: similarly, a branch is created from “main”, and then merged to both “main” and “develop”, and we have the same problem.
Now, what I want to achieve is, either “main” or “develop” triggers a build, but not both. I can see a couple of logic to do this:
- Only build when the branch is “main”, and version number (as a variable) does not contain “dev” string, or when the branch is “develop”, and version number contains “dev” string.
- Only build when the branch is “main”, and the merge was from a branch with either “release” or “hotfix” prefix, or when the branch is “develop”, and the merge was from a branch with “feature” prefix.
Is it possible to express this logic in CircleCI?
I am still learning this, so I am not sure I am on the right path, but I was thinking about a similar scenario so I can share my ideas.
- My “main” branch doesn’t trigger any build in my current workflow. I use it only to store the latest versions.
- I have branches for different minorversions like “1.1” and “2.0”
- I also have a branch for the development of the supported minor versions: “2.0-dev”
- When I create a feature, I can create a new branch from "2.0-dev"and this is where I merge it back
- I build only “2.0-dev” and “2.0” at the moment and here is where I trigger the same build twice when I merge “2.0-dev” to “2.0”.
- Every Docker image gets the commit hash as its first tag. Everything else is just an alias.
- When I build the same commit again, I can try to pull
<image_name>:<commit_hash>. If I can, it means it was built and I can decide if I want to build it again for some reasons for example I want to update the latest images using the updated parent.
- From 2.0-dev" I set only “2.0-dev” as an alias.
- When I add a tag to a commit hash, it triggers a build but it doesn’t build the image again, only pulls the built image with the tag matching with the commit hash, sets the alias and pushes that.
- My versions are not coming from manually variables (your dev versions are if I understood you correctly) but from commit tags and branch names. “-dev” suffix used only for dev branches without any number at the and. When I want to release a “pre-release” version I add it as a tag like
1.0.0-alpha.1 in addition to the commit hash tag.
So based on the above facts, I could say I accept that the build will be triggered, but in my build script executed by CircleCI I detect that the tag already exists and immediately stop the build process exiting with “0” so the build would be sucessful without actually using too many resources and without pusing to the registry. If I want to rebuild that tag in a cronjob, I can set a pipeline parameter (I did that but now I am not sure there isn’t a built in solution) so I can build when
CI_EVENT_TYPE is “cron”.
On the other hand I could actually say that I want to build only “2.0-dev” (develop in your case) and not “2.0” when I push the code to GitHub and I would trigger a build for semantic versions like “1.0.0” and
In your case I guess you could trigger the build for the “develop” branch and the tags following semantic versioning. When you merge into “main”, you know all of your commits were built and tested so you don’t need to do it again.
In the end I also generate multiple tags for my Docker image based on the semantic version but only for stable versions. So when I build add a pre-release tag or push to a branch I use only the name of the tag or the name of the branch (currently I add “2.0-dev” to the images built from “2.0”). When I add a stable version like “2.1.23” I also add “2.1” when and it there is no
2.2.* version I add “2” too. “latest” added only when this is the latest major version.
There may be some error in my logic, but this is how I would try to avoid building twice.
Circleci’s processing is currently limited to the basic filter checks at the workflow level - which currently go as far as checking branch name or if ignore was used as a filter then go and check the tags. This is made even more restrictive by the fact that if the filter fires on a branch filter it does not populate the tag parameter and if it fires on a tag filter it defines the branch parameter as empty.
So there are only 2 real solutions,
- Firstly, use a bash script within the circleci job to define the state of the repo to see if a build is required or not. This would result in circleci running the check everytime the repo is modified. This may cause resource costs as it would result in a runner being started even if it is for a few seconds.
- Secondly, write an external monitoring tool that watches for webhook events and then does the same type of checks as done in the bash script - if it finds a build is required it can then add tags to repo that circleci can then correctly process.
Neither of these options is ideal and your choice would most likely depend on what your skill set is. If you have the right skills using the webhook interface would give you a lot more flexibility.