CircleCI 2.0 copy files before AWS EBS Elastic Beanstalk Deployment using python3

Hi guys,

After not touching my config.yml for sometime and then getting the push from AWS to upgrade python version from 2 to 3, I’ve had to change my EBCLI setup for python with in the config.yml

What I was doing was depending on the environment that I was deploying to, I would copy in appropriate config file into config directory of the node app so that when I starts it starts knowing that its, i.e. dev or int or syst, prod etc.

As soon as I went from this approach with old Python

            # Install AWS EB cli
            - run:
                name: Install awsEBcli
                command: |
                   python --version
                   sudo apt-get -y -qq update
                   sudo apt-get install -y python-pip libpython-dev python-virtualenv python-setuptools
                   virtualenv venv
                   source venv/bin/activate
                   pip install --upgrade setuptools pip
                   pip install awsebcli urllib3[secure]==1.22 idna==2.6

                   eb --version

            # Copy Environment specific files
            - run:
                name: App Properties and Constant file
                command: |
                    if [ "${CIRCLE_BRANCH}" == "integration" ]; then
                        echo ""
                        rm .gitignore

                       echo ""
                        echo "Move CONFIG JS File"
                        mv .deploy/config.js.integration config/config.js
                        if [ $? != 0 ]; then
                           echo "FAILED: Move Config.js: $? - Unsuccessful"
                        else
                           echo "SUCCESS: Config.js mv done 3 of 4"
                        fi

To Python3 like this

            # Install AWS EB cli
            - run:
                name: Install awsEBcli
                command: |
                    sudo apt-get -y -qq update
                    sudo apt-get install python3-pip python3-dev python3-venv python3-setuptools
                    python --version
                    python3 -m venv venv
                    source venv/bin/activate
                    python --version
                    pip install --upgrade pip
                    pip install wheel
                    pip install awsebcli
                    eb --version


            # Copy Environment specific files
            - run:
                  name: App Properties and Constant file
                  command: |
                      if [ "${CIRCLE_BRANCH}" == "integration" ]; then
                          echo ""
                          rm .gitignore

                          echo ""
                          echo "Move CONFIG JS File"
                          mv .deploy/config.js.dev config/config.js
                          if [ $? != 0 ]; then
                             echo "FAILED: Move Config.js: $? - Unsuccessful"
                          else
                             echo "SUCCESS: Config.js mv done 1 of 2"
                          fi
                      fi

I’ve SSH’d into the build circle build machine and can see that the files have actually moved into the correct directory. However, why ins’t my EBS Deploy using it

# Deploy to the correct Elastic Beanstalk Environment  corresponding to the current branch
            - run:
                name: Deploy to EBS
                command: |
                    source venv/bin/activate
                    if ..........
                    elif [ "${CIRCLE_BRANCH}" == "integration" ]; then
                        echo "START EBS Beanstalk Int Deployment"
                        echo "Start Initialise Region"
                        eb init --region ap-southeast-2 --platform "64bit Amazon Linux 2 v5.4.5 running Node.js 14" [appname]
                        eb use [app-name]-env
                        eb labs cleanup-versions --older-than 2 --debug --verbose --force
                        echo " Start Deploy to Environment"
                        eb deploy [app-name]-env --label "$COMMIT_MESSAGE"
                        echo "COMPLETE EBS Beanstalk Deployment done "
                     fi

Any assistance would be greatly appreciated.

Hi @kontact00 ,

when you mentioned that:

However, why ins’t my EBS Deploy using it?

I understand you mean that the Deploy to EBS step has failed ?
If that is the case indeed, could you share the errors you are seeing?

This would help me identify better the context of the issue :slight_smile:

Thank you!

HI Kelvin,

The deploy to EBS works. It creates a zip file, uploads to S3, and deploys successfully.

The part that isn’t working or occuring, is that the
I can see that this has happened mv .deploy/config.js.dev config/config.js
but the zip file that EB creates doesn’t contain the new config.js based on this move.

Does that help?
Regards
Nick

@kontact00

I see. I understand better now the issue. Thank you for explaining.

Just to be sure, can you confirm that the contents in your .ebignore file, if any?
I would like to rule out this one, firstly.
See Configure the EB CLI - AWS Elastic Beanstalk

Next, can you share with me the ElasticBeanstalk CLI version installed during this run?
We can look up the release history to know if there were changes made that may have impacted this :slight_smile:

I assumed in your previous workflow with Python2, the CLI version installed may have been different.

Hmmmm, ok so I don’t have a .ebignore file

My file structure is like this:

.circleci
  -> config.yml
.deploy
  -> gitignore.ebdeploy
  -> config.js.dev
  -> database.js.dev
.ebextensions
  --> 01_fix_permissions.config

The contents of the .gitignore.ebdeploy is as follows:

.deploy/*
.ebextensions/*
ebdeploy-dev.sh
config/config.js
config/database.js

The EB CLI 3.20.0 (Python 3.5.3)

The last successful time was using EB CLI 3.18.2 (Python 2.7.1)
The .gitignore is exactly the same.

I’ll remove the

config/config.js
config/database.js

and rerun it again to see it that makes a difference.

Removing those lines didn’t make a difference.

So I’ve got to the bottom of this, and not 100% certain which part fixed it. I was missing the .ebignore from the root folder so I added that. It only contains

/node_modules/

I had also removed the

.elastisticbeanstalk → config.yml

And put that back in place.

It is now doing as expected.

Hi @kontact00

Awesome, Glad to hear that you managed to get to the bottom of it! :slight_smile: