I am trying to install awslocal on machine executor, the installation is successful, but when I am running ‘WHICH awslocal’ getting error
jobs:
machine-job:
machine: true
environment:
- *default-env
steps:
- checkout
- run:
name: docker-compose up
command: docker-compose up
background: true
# environment:
# TMPDIR: ~/localstack
# DEBUG: 1
- *install_awscli
- run: which aws
- run:
name: install using pip3
command: |
pip3 install -q localstack awscli-local
- run:
name: add awslocal to bash profile
command: complete -C '/home/circleci/bin/aws_completer' awslocal
- run: which awslocal
Hello
I believe the issue is not within the code that you have shared but either the docker-compose up or *install_awscli.
Below is the config that I used based on your own:
build:
machine: true
steps:
- checkout
- run:
name: install using pip3
command: |
pip3 install -q localstack awscli-local
- run:
name: add awslocal to bash profile
command: complete -C '/home/circleci/bin/aws_completer' awslocal
- run: which awslocal
When running that block the following was output on the step in the job output:
#!/bin/bash -eo pipefail
which awslocal
/opt/circleci/.pyenv/shims/awslocal
CircleCI received exit code 0
Kind Regards
Owen Oliver
Here is my steps for installing aws_cli
install_awscli: &install_awscli
run:
name: install_awscli
command: |
mkdir -p ~/bin
cd ~/bin
wget "https://s3.amazonaws.com/aws-cli/awscli-bundle-1.16.188.zip"
unzip -q awscli-bundle-1.16.188.zip
awscli-bundle/install -b ~/bin/aws
Also docker-compose file
version: '2.1'
services:
localstack:
container_name: "localstack"
image: localstack/localstack
network_mode: bridge
ports:
- 4566:4566
environment:
- SERVICES=s3,dynamodb,lambda,events,iam,logs,sts
- DEBUG=${DEBUG- }
- DATA_DIR=${DATA_DIR- }
- PORT_WEB_UI=${PORT_WEB_UI- }
- LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR- }
- KINESIS_ERROR_PROBABILITY=${KINESIS_ERROR_PROBABILITY- }
- DOCKER_HOST=unix:///var/run/docker.sock
- HOST_TMP_FOLDER=${TMPDIR}
volumes:
- "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
Could you let me know how you are installing aws-cli ?
Hello
Would it be possible to add your docker-compose command to be after the installation of everything else?
I believe the issues are due to the docker-compose file but I believe by adding the later we may be able to avoid the issues.
Kind Regards
Owen Oliver
It looks like you’re installing the utilities into ~/bin, but that directory is not in $PATH.
@owenjoliver I tried moving docker-compose later but didn’t worked.
@zackse Could you let me know how I can add it to the path, I tried lot of things but didn’t worked.
Here is the updated config.yml
version: 2.0
# ----------------------------------------------------------------------------
# aliases
# ------------------------------------------------------------------------------
install_awscli: &install_awscli
run:
name: install_awscli
command: |
mkdir -p ~/bin
cd ~/bin
wget "https://s3.amazonaws.com/aws-cli/awscli-bundle-1.16.188.zip"
unzip -q awscli-bundle-1.16.188.zip
awscli-bundle/install -b ~/bin/aws
default-env: &default-env
- AWS_DEFAULT_REGION: us-east-2
- AWS_SECRET_ACCESS_KEY: foo
- AWS_ACCESS_KEY_ID: bar
# ------------------------------------------------------------------------------
# pipeline
# ------------------------------------------------------------------------------
workflows:
version: 2
jobs:
jobs:
- machine-job
# ----------------------------------------------------------------------------
# jobs
# ------------------------------------------------------------------------------
jobs:
machine-job:
machine: true
environment:
- *default-env
steps:
- checkout
- *install_awscli
- run: echo $PATH
- run: |
pip3 install -q localstack awscli-local
- run: which awslocal
- run:
name: docker-compose up
command: docker-compose up
environment:
TMPDIR: ~/localstack
DEBUG: 1
# - run:
# name: Wait for localstack
# command: |
# sleep 30
- run:
name: run awslocal command
command: |
awslocal dynamodb create-table \
--table-name LocalStackOneStop \
--attribute-definitions \
AttributeName=LoanNumber,AttributeType=S \
--key-schema \
AttributeName=LoanNumber,KeyType=HASH \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5
- run:
name: scan dynamodb table
command: |
awslocal dynamodb scan --table-name LocalStackOneStop
See the last example in this doc section for one approach.
Thanks for the reference @zackse . I could able to resolve the path issue.
But I am facing weird issue, I could only run the awslocal command , in the same step like mentioned below.
- run:
name: install awslocal
command: |
echo 'export PATH=/home/circleci/.local/bin:$PATH' >> $BASH_ENV
source $BASH_ENV
pip3 install -q localstack awscli-local
which awslocal
awslocal dynamodb list-tables
awslocal dynamodb create-table \
--table-name LocalStackOneStop \
--attribute-definitions \
AttributeName=LoanNumber,AttributeType=S \
--key-schema \
AttributeName=LoanNumber,KeyType=HASH \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5
awslocal dynamodb list-tables
If I move the awslocal command to the next run step, I am seeing the same error. How can I persist awslocal command in the entire pipeline
- run: |
echo 'export PATH=/home/circleci/.local/bin:$PATH' >> $BASH_ENV
source $BASH_ENV
pip3 install -q localstack awscli-local
which awslocal
awslocal dynamodb list-tables
- run:
name: run awslocal command
command: |
awslocal dynamodb create-table \
--table-name LocalStackOneStop \
--attribute-definitions \
AttributeName=LoanNumber,AttributeType=S \
--key-schema \
AttributeName=LoanNumber,KeyType=HASH \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5
Here is a sample screenshot
Looks like the docs are inaccurate about persisting and/or sourcing $BASH_ENV on every run step.
Another approach would be to call pip3 with sudo to install globally (likely /usr/local/bin) which would mean you don’t need to modify $PATH:
sudo pip3 install -q localstack awscli-local
