After spending considerable time investigating failing builds, it appears that (after SSHing) into a container, the source code being used is not the most current code.
How can I be absolutely certain my config is setup PROPERLY to be pulling down fresh source code?
If you are using checkout: we should always pull the commit we receive the webhook for. In addition, if you expand that step, you will see the SHA that we reset the project to after checkout. Can you check and see if these are matching? You can find the SHA of the webhook in the top right of the job details page.
hi drazisil:
thanks for your response. The SHA seems to be the correct one. The problem is apparent when I ssh into the container and can see that the code is not current, for example, files that I renamed are still there. What could be the reason for this?
Are you attempting any type of source caching that may be overwriting/adding files after checkout?
To be honest, I suspect my cache strategy is to blame, would you mind looking at my config for a bad config?
version: 2.1
jobs:
build-dependencies:
docker:
- image: circleci/node:10
working_directory: ~/gamma
environment:
CI: false
steps:
- checkout
- restore_cache:
keys:
- dependency-cache-v3-{{ .Branch }}-{{ checksum "package.json" }}-{{ .Revision }}
- run:
name: Install Dependencies
command: yarn install --frozen-lockfile
- save_cache:
key: dependency-cache-v3-{{ .Branch }}-{{ checksum "package.json" }}-{{ .Revision }}
paths:
- ~/.cache
- run:
name: Build 'api'
command: yarn build:api
- run:
name: Build 'web'
command: CI=false yarn build:web
- save_cache:
key: source-code-v4-{{ .Environment.CIRCLE_SHA1 }}-{{ .Branch }}-{{ .Revision }}
paths:
- gamma
- persist_to_workspace:
root: ~/
paths:
- gamma
- .cache/Cypress
lint:
docker:
- image: circleci/node:10
working_directory: ~/gamma
steps:
- attach_workspace:
at: ~/
- restore_cache:
keys:
- dependency-cache-v3-{{ .Branch }}-{{ checksum "package.json" }}-{{ .Revision }}
- source-code-v4-{{ .Environment.CIRCLE_SHA1 }}-{{ .Branch }}-{{ .Revision }}
- run:
name: Lint
command: yarn lint
test-e2e:
docker:
- image: cypress/base:8
environment:
TERM: xterm
working_directory: ~/gamma
steps:
- attach_workspace:
at: ~/
- restore_cache:
keys:
- dependency-cache-v3-{{ .Branch }}-{{ checksum "package.json" }}-{{ .Revision }}
- source-code-v4-{{ .Environment.CIRCLE_SHA1 }}-{{ .Branch }}-{{ .Revision }}
- run:
name: Verify Cypress is installed
command: yarn cy:verify
- save_cache:
key: dependency-cache-v3-{{ .Branch }}-{{ checksum "package.json" }}-{{ .Revision }}
paths:
- ~/.cache
- run:
name: Start e2e-tests
command: |
yarn test:e2e
background: false
- store_artifacts:
path: cypress/videos
- store_artifacts:
path: cypress/screenshots
build-prod:
docker:
- image: circleci/node:10
working_directory: ~/gamma
environment:
CI: false
steps:
- attach_workspace:
at: ~/
- restore_cache:
key: source-code-v4-{{ .Environment.CIRCLE_SHA1 }}-{{ .Branch }}-{{ .Revision }}
- run:
name: Build api
command: yarn build:api
- run:
name: Build web
command: yarn build:web
- save_cache:
key: v2-prod-build-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}-{{ .Revision }}
paths:
- ./web
- ./api
- ./shared
prepare-pr-deployment-images:
machine: true
working_directory: ~/gamma
steps:
- checkout
- restore_cache:
key: v2-prod-build-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}-{{ .Revision }}
- run:
name: Parameterize ENV variables
command: |
echo 'export TAG=${CIRCLE_SHA1:0:7}' >> $BASH_ENV
echo 'export API_IMAGE=api' >> $BASH_ENV
echo 'export WEB_IMAGE=web' >> $BASH_ENV
echo 'export DOCKER_HUB_ORG=gammastaging' >> $BASH_ENV
- run:
name: Docker Sign In
command: |
echo $DOCKER_PWD | docker login -u $DOCKER_USER --password-stdin
- run:
name: Create web image
command: |
docker build -t ${DOCKER_HUB_ORG}/${WEB_IMAGE}:${TAG} -f "Dockerfile.web" .
- run:
name: Create api image
working_directory: ~/gamma/packages/api
command: |
docker build -t ${DOCKER_HUB_ORG}/${API_IMAGE}:${TAG} -f "Dockerfile.api" .
- run:
name: Push web image
command: |
docker push ${DOCKER_HUB_ORG}/${WEB_IMAGE}:${TAG}
- run:
name: Push api image
command: |
docker push ${DOCKER_HUB_ORG}/${API_IMAGE}:${TAG}
deploy-to-now-pr-staging:
docker:
- image: circleci/node:10
working_directory: ~/gamma
steps:
- checkout
- run:
name: Install now-cli
command: |
sudo npm i -g --unsafe-perm now
- run:
name: Parameterize ENV variables
command: |
echo 'export TAG=${CIRCLE_SHA1:0:7}' >> $BASH_ENV
echo 'export API_IMAGE=api' >> $BASH_ENV
echo 'export WEB_IMAGE=web' >> $BASH_ENV
echo 'export DOCKER_HUB_ORG=gammastaging' >> $BASH_ENV
- run:
name: Generate 'latest' api dockerfile to be deployed to staging
command: |
cd packages/api
/home/circleci/gamma/update.sh ${API_IMAGE} ${TAG} ${DOCKER_HUB_ORG}
- run:
name: Deploy api to now
command: |
cd packages/api
echo export API_STAGING_URL="$(now -t "$NOW_TOKEN" -A now.api-staging.json)" >> "$BASH_ENV"
source $BASH_ENV
echo $API_STAGING_URL
/home/circleci/gamma/deploy-pr.sh web ${API_STAGING_URL}
cat /home/circleci/gamma/now.ui-staging.json
- run:
name: Generate 'latest' web dockerfile to be deployed to staging
command: |
./update.sh ${WEB_IMAGE} ${TAG} ${DOCKER_HUB_ORG}
- run:
name: Deploy web to now
command: |
echo export DEPLOY_PREVIEW_URL="$(now -t $NOW_TOKEN -A now.ui-staging.json)" >> "$BASH_ENV"
source $BASH_ENV
echo $DEPLOY_PREVIEW_URL
./sendurl.sh ${DEPLOY_PREVIEW_URL}
prepare-prod-deployment-images:
machine: true
working_directory: ~/gamma
steps:
- checkout
- restore_cache:
key: v2-prod-build-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}-{{ .Revision }}
- run:
name: Parameterize ENV variables
command: |
echo 'export TAG=${CIRCLE_SHA1:0:7}' >> $BASH_ENV
echo 'export API_IMAGE=api' >> $BASH_ENV
echo 'export WEB_IMAGE=web' >> $BASH_ENV
echo 'export DOCKER_HUB_ORG=gammaprod' >> $BASH_ENV
- run:
name: Docker Sign In
command: |
echo $DOCKER_PWD | docker login -u $DOCKER_USER --password-stdin
- run:
name: Create web image
command: |
docker build -t ${DOCKER_HUB_ORG}/${WEB_IMAGE}:${TAG} -f "Dockerfile.web" .
- run:
name: Create api image
working_directory: ~/gamma/packages/api
command: |
docker build -t ${DOCKER_HUB_ORG}/${API_IMAGE}:${TAG} -f "Dockerfile.api" .
- run:
name: Push web image
command: |
docker push ${DOCKER_HUB_ORG}/${WEB_IMAGE}:${TAG}
- run:
name: Push api image
command: |
docker push ${DOCKER_HUB_ORG}/${API_IMAGE}:${TAG}
deploy-to-now-prod:
docker:
- image: circleci/node:10
working_directory: ~/gamma
steps:
- checkout
- run:
name: Install now-cli
command: |
sudo npm i -g --unsafe-perm now
- run:
name: Parameterize ENV variables
command: |
echo 'export TAG=${CIRCLE_SHA1:0:7}' >> $BASH_ENV
echo 'export API_IMAGE=api' >> $BASH_ENV
echo 'export WEB_IMAGE=web' >> $BASH_ENV
echo 'export DOCKER_HUB_ORG=gammaprod' >> $BASH_ENV
- run:
name: Generate 'latest' web image to be deployed to production
command: |
./update.sh ${WEB_IMAGE} ${TAG} ${DOCKER_HUB_ORG}
- run:
name: Generate 'latest' api image to be deployed to production
command: |
cd packages/api
/home/circleci/gamma/update.sh ${API_IMAGE} ${TAG} ${DOCKER_HUB_ORG}
- run:
name: Deploy web to now & alias
command: |
now -t $NOW_TOKEN -A now.ui.json && now -t $NOW_TOKEN -A now.ui.json alias
- run:
name: Deploy api to now & alias
command: |
cd packages/api
now -t $NOW_TOKEN -A now.api.json && now -t $NOW_TOKEN -A now.api.json alias
workflows:
version: 2
build-test-push-deploy-pr:
jobs:
- build-dependencies:
filters:
branches:
ignore: master
- lint:
requires:
- build-dependencies
filters:
branches:
ignore: master
- test-e2e:
requires:
- build-dependencies
- lint
filters:
branches:
ignore: master
- build-prod:
requires:
- test-e2e
filters:
branches:
ignore: master
- prepare-pr-deployment-images:
requires:
- build-prod
filters:
branches:
ignore: master
- deploy-to-now-pr-staging:
requires:
- prepare-pr-deployment-images
filters:
branches:
ignore: master
build-test-push-deploy-master:
jobs:
- build-dependencies:
filters:
branches:
only: master
- lint:
requires:
- build-dependencies
filters:
branches:
only: master
- test-e2e:
requires:
- build-dependencies
- lint
filters:
branches:
only: master
- build-prod:
requires:
- test-e2e
filters:
branches:
only: master
- prepare-prod-deployment-images:
requires:
- build-prod
filters:
branches:
only: master
- deploy-to-now-prod:
requires:
- prepare-prod-deployment-images
filters:
branches:
only: master
I would totally kill this section. What is your goal of a source cache? If it’s to move code around (checkout: is usually faster) after you change it as part of a step, Workspaces is likely a better option.
ETA: It looks like you are using both?
OK, so I removed all references to source-code-v4-{{ .Environment.CIRCLE_SHA1 }}-{{ .Branch }}-{{ .Revision }} and pushed to my open pull request. Build failed just like previously.
Try changing this to dependency-cache-v3-{{ .Branch }}-{{ checksum “package-lock.json” and change all your keys to V4 to roll them so you are starting with a clean cache. Lets see if that helps.
I dont use a package-lock file. I use yarn and was told yesterday by Kyle @ circle to remove ‘yarn.lock’ as a key.
FYI, the spam filter you guys have on this message board hid my last post
Fixed the hidden post, and then lets keep the package.json, and just try rolling the cache keys.
that didn’t fix the problem.
Every job runs in a clean box, so the next steps would be to remove all forms of caching so there is nothing that can inject files until you find the issue.