I am trying to set up CI/CD for my repo.
The config.yml:
version: 2.1
orbs:
aws-ecr: circleci/aws-ecr@6.1.0
aws-ecs: circleci/aws-ecs@0.0.8
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/node:12.2.0
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
working_directory: ~/repo
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run: npm install
# - run: $(aws ecr get-login --region eu-west-1)
# - run: npm run build
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
workflows:
version: 2.1
build-and-deploy:
jobs:
- build
- aws-ecr/build-and-push-image:
requires:
- build
account-url: AWS_ECR_ACCOUNT_URL
aws-access-key-id: AWS_ACCESS_KEY_ID
aws-secret-access-key: AWS_SECRET_ACCESS_KEY
create-repo: true
repo: arsia
dockerfile: "Dockerfile"
region: AWS_REGION
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build-and-push-image
family: arsia-service
cluster-name: arsia-cluster
container-image-name-updates: 'container=${AWS_RESOURCE_NAME_PREFIX}-service,tag=latest'
After the aws-ecr is done, i check my AWS ECR and the repo is there, and I can validate it when I manually test the repo (by running a Fargo cluster) and my application runs. Perfect!
But I donât want to go through that hassle of creating a task definition every time, with VPCs, port mapping, etc. Apparently aws-ecs can help with updating that:
I create a cluster, create a task definition using a default Nginx container image, and spinning that cluster.
When circleCI gets to executing the aws-ecs/deploy-service-update stage, it fails at the Retrieve previous task definition and prepare new task definition values step. How can I remedy that? it seems strange that I do have a task definition and yet it is unable to retrieve the task definition! 
snapshot of the error stack:
#!/bin/bash -eo pipefail
PREVIOUS_TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition jaguar-service)
CONTAINER_IMAGE_NAME_UPDATES="$(echo container=${AWS_RESOURCE_NAME_PREFIX}-service,tag=latest)"
CONTAINER_ENV_VAR_UPDATES="$(echo )"
# Prepare script for updating container definitions
UPDATE_CONTAINER_DEFS_SCRIPT_FILE=$(mktemp _update_container_defs.py.XXXXXX)
chmod +x $UPDATE_CONTAINER_DEFS_SCRIPT_FILE
cat > $UPDATE_CONTAINER_DEFS_SCRIPT_FILE <<-EOF
from __future__ import absolute_import
import sys
import json
def run(previous_task_definition, container_image_name_updates, container_env_var_updates):
try:
definition = json.loads(previous_task_definition)
container_definitions = definition['taskDefinition']['containerDefinitions']
except:
raise Exception('No valid task definition found: ' +
previous_task_definition)
# Build a map of the original container definitions so that the
# array index positions can be easily looked up
container_map = {}
for index, container_definition in enumerate(container_definitions):
env_var_map = {}
env_var_definitions = container_definition.get('environment')
if env_var_definitions is not None:
for env_var_index, env_var_definition in enumerate(env_var_definitions):
env_var_map[env_var_definition['name']] = {
'index': env_var_index}
container_map[container_definition['name']] = {
'image': container_definition['image'], 'index': index, 'environment_map': env_var_map}
# Expected format: container=...,name=...,value=...,container=...,name=...,value=
try:
env_kv_pairs = container_env_var_updates.split(',')
for index, kv_pair in enumerate(env_kv_pairs):
kv = kv_pair.split('=')
key = kv[0].strip()
if key == 'container':
container_name = kv[1].strip()
env_var_name_kv = env_kv_pairs[index+1].split('=')
env_var_name = env_var_name_kv[1].strip()
env_var_value_kv = env_kv_pairs[index+2].split('=')
env_var_value = env_var_value_kv[1].strip()
if env_var_name_kv[0].strip() != 'name' or env_var_value_kv[0].strip() != 'value':
raise ValueError(
'Environment variable update parameter format is incorrect: ' + container_env_var_updates)
container_entry = container_map.get(container_name)
if container_entry is None:
raise ValueError('The container ' + container_name +
' is not defined in the existing task definition')
container_index = container_entry['index']
env_var_entry = container_entry['environment_map'].get(
env_var_name)
if env_var_entry is None:
# The existing container definition did not contain environment variables
if container_definitions[container_index].get('environment') is None:
container_definitions[container_index]['environment'] = []
# This env var did not exist in the existing container definition
container_definitions[container_index]['environment'].append({'name': env_var_name, 'value': env_var_value})
else:
env_var_index = env_var_entry['index']
container_definitions[container_index]['environment'][env_var_index]['value'] = env_var_value
elif key and key not in ['container', 'name', 'value']:
raise ValueError(
'Incorrect key found in environment variable update parameter: ' + key)
except ValueError as value_error:
raise value_error
except:
raise Exception(
'Environment variable update parameter could not be processed; please check parameter value: ' + container_env_var_updates)
# Expected format: container=...,image-and-tag|image|tag=...,container=...,image-and-tag|image|tag=...,
try:
image_kv_pairs = container_image_name_updates.split(',')
for index, kv_pair in enumerate(image_kv_pairs):
kv = kv_pair.split('=')
key = kv[0].strip()
if key == 'container':
container_name = kv[1].strip()
image_kv = image_kv_pairs[index+1].split('=')
container_entry = container_map.get(container_name)
if container_entry is None:
raise ValueError('The container ' + container_name +
' is not defined in the existing task definition')
container_index = container_entry['index']
image_specifier_type = image_kv[0].strip()
image_value = image_kv[1].strip()
if image_specifier_type == 'image-and-tag':
container_definitions[container_index]['image'] = image_value
else:
existing_image_name_tokens = container_entry['image'].split(
':')
if image_specifier_type == 'image':
tag = ''
if len(existing_image_name_tokens) == 2:
tag = ':' + existing_image_name_tokens[1]
container_definitions[container_index]['image'] = image_value + tag
elif image_specifier_type == 'tag':
container_definitions[container_index]['image'] = existing_image_name_tokens[0] + \
':' + image_value
else:
raise ValueError(
'Image name update parameter format is incorrect: ' + container_image_name_updates)
elif key and key not in ['container', 'image', 'image-and-tag', 'tag']:
raise ValueError(
'Incorrect key found in image name update parameter: ' + key)
except ValueError as value_error:
raise value_error
except:
raise Exception(
'Image name update parameter could not be processed; please check parameter value: ' + container_image_name_updates)
return json.dumps(container_definitions)
if __name__ == '__main__':
try:
print(run(sys.argv[1], sys.argv[2], sys.argv[3]))
except Exception as e:
sys.stderr.write(str(e) + "\n")
exit(1)
EOF
# Prepare container definitions
CONTAINER_DEFS=$(python $UPDATE_CONTAINER_DEFS_SCRIPT_FILE "$PREVIOUS_TASK_DEFINITION" "$CONTAINER_IMAGE_NAME_UPDATES" "$CONTAINER_ENV_VAR_UPDATES")
# Prepare script for getting task definition values
GET_TASK_DFN_VAL_SCRIPT_FILE=$(mktemp _get_task_def_value.py.XXXXXX)
chmod +x $GET_TASK_DFN_VAL_SCRIPT_FILE
cat > $GET_TASK_DFN_VAL_SCRIPT_FILE <<-EOF
from __future__ import absolute_import
import sys
import json
def run(element_name, task_definition_str):
try:
definition = json.loads(task_definition_str)
task_definition = definition['taskDefinition']
except:
raise Exception('No valid task definition found: ' +
task_definition_str)
str_list_types = ['requiresCompatibilities']
json_arr_types = ['placementConstraints', 'volumes']
if element_name in json_arr_types:
output_value = '[]'
else:
output_value = ''
if element_name in task_definition:
element_value = task_definition[element_name]
if element_name in str_list_types:
for i, list_item in enumerate(element_value):
output_value += list_item.strip()
if len(element_value) - 1 > i:
output_value += ' '
elif element_name in json_arr_types:
output_value = json.dumps(element_value)
else:
output_value = str(element_value)
return output_value
if __name__ == '__main__':
try:
print(run(sys.argv[1], sys.argv[2]))
except Exception as e:
sys.stderr.write(str(e) + "\n")
exit(1)
EOF
# Get other task definition values
TASK_ROLE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'taskRoleArn' "$PREVIOUS_TASK_DEFINITION")
EXECUTION_ROLE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'executionRoleArn' "$PREVIOUS_TASK_DEFINITION")
NETWORK_MODE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'networkMode' "$PREVIOUS_TASK_DEFINITION")
VOLUMES=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'volumes' "$PREVIOUS_TASK_DEFINITION")
PLACEMENT_CONSTRAINTS=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'placementConstraints' "$PREVIOUS_TASK_DEFINITION")
REQ_COMP=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'requiresCompatibilities' "$PREVIOUS_TASK_DEFINITION")
TASK_CPU=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'cpu' "$PREVIOUS_TASK_DEFINITION")
TASK_MEMORY=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'memory' "$PREVIOUS_TASK_DEFINITION")
# Make task definition values available as env variables
echo "export CCI_ORB_AWS_ECS_TASK_ROLE='${TASK_ROLE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_EXECUTION_ROLE='${EXECUTION_ROLE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_NETWORK_MODE='${NETWORK_MODE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_CONTAINER_DEFS='${CONTAINER_DEFS}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_VOLUMES='${VOLUMES}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_PLACEMENT_CONSTRAINTS='${PLACEMENT_CONSTRAINTS}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_REQ_COMP='${REQ_COMP}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_TASK_CPU='${TASK_CPU}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_TASK_MEMORY='${TASK_MEMORY}'" >> $BASH_ENV
rm $UPDATE_CONTAINER_DEFS_SCRIPT_FILE $GET_TASK_DFN_VAL_SCRIPT_FILE
An error occurred (ClientException) when calling the DescribeTaskDefinition operation: Unable to describe task definition.
Exited with code 255
