Aws-ecs fails to `Retrieve previous task definition and prepare new task definition values`

I am trying to set up CI/CD for my repo.
The config.yml:

version: 2.1
orbs:
  aws-ecr: circleci/aws-ecr@6.1.0
  aws-ecs: circleci/aws-ecs@0.0.8
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/node:12.2.0

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run: npm install
      # - run: $(aws ecr get-login --region eu-west-1)
      # - run: npm run build
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

workflows:
  version: 2.1
  build-and-deploy:
    jobs:
      - build            
      - aws-ecr/build-and-push-image:
          requires:
            - build
          account-url: AWS_ECR_ACCOUNT_URL
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
          create-repo: true
          repo: arsia
          dockerfile: "Dockerfile"
          region: AWS_REGION
      - aws-ecs/deploy-service-update:
          requires:
            - aws-ecr/build-and-push-image
          family: arsia-service
          cluster-name: arsia-cluster
          container-image-name-updates: 'container=${AWS_RESOURCE_NAME_PREFIX}-service,tag=latest'

After the aws-ecr is done, i check my AWS ECR and the repo is there, and I can validate it when I manually test the repo (by running a Fargo cluster) and my application runs. Perfect!

But I don’t want to go through that hassle of creating a task definition every time, with VPCs, port mapping, etc. Apparently aws-ecs can help with updating that:
I create a cluster, create a task definition using a default Nginx container image, and spinning that cluster.
When circleCI gets to executing the aws-ecs/deploy-service-update stage, it fails at the Retrieve previous task definition and prepare new task definition values step. How can I remedy that? it seems strange that I do have a task definition and yet it is unable to retrieve the task definition! :thinking:

snapshot of the error stack:

#!/bin/bash -eo pipefail
PREVIOUS_TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition jaguar-service)
CONTAINER_IMAGE_NAME_UPDATES="$(echo container=${AWS_RESOURCE_NAME_PREFIX}-service,tag=latest)"
CONTAINER_ENV_VAR_UPDATES="$(echo )"

# Prepare script for updating container definitions
UPDATE_CONTAINER_DEFS_SCRIPT_FILE=$(mktemp _update_container_defs.py.XXXXXX)
chmod +x $UPDATE_CONTAINER_DEFS_SCRIPT_FILE
cat > $UPDATE_CONTAINER_DEFS_SCRIPT_FILE <<-EOF
from __future__ import absolute_import
import sys
import json


def run(previous_task_definition, container_image_name_updates, container_env_var_updates):
    try:
        definition = json.loads(previous_task_definition)
        container_definitions = definition['taskDefinition']['containerDefinitions']
    except:
        raise Exception('No valid task definition found: ' +
                        previous_task_definition)

    # Build a map of the original container definitions so that the
    # array index positions can be easily looked up
    container_map = {}
    for index, container_definition in enumerate(container_definitions):
        env_var_map = {}
        env_var_definitions = container_definition.get('environment')
        if env_var_definitions is not None:
            for env_var_index, env_var_definition in enumerate(env_var_definitions):
                env_var_map[env_var_definition['name']] = {
                    'index': env_var_index}
        container_map[container_definition['name']] = {
            'image': container_definition['image'], 'index': index, 'environment_map': env_var_map}

    # Expected format: container=...,name=...,value=...,container=...,name=...,value=
    try:
        env_kv_pairs = container_env_var_updates.split(',')
        for index, kv_pair in enumerate(env_kv_pairs):
            kv = kv_pair.split('=')
            key = kv[0].strip()

            if key == 'container':
                container_name = kv[1].strip()
                env_var_name_kv = env_kv_pairs[index+1].split('=')
                env_var_name = env_var_name_kv[1].strip()
                env_var_value_kv = env_kv_pairs[index+2].split('=')
                env_var_value = env_var_value_kv[1].strip()
                if env_var_name_kv[0].strip() != 'name' or env_var_value_kv[0].strip() != 'value':
                    raise ValueError(
                        'Environment variable update parameter format is incorrect: ' + container_env_var_updates)

                container_entry = container_map.get(container_name)
                if container_entry is None:
                    raise ValueError('The container ' + container_name +
                                     ' is not defined in the existing task definition')
                container_index = container_entry['index']
                env_var_entry = container_entry['environment_map'].get(
                    env_var_name)
                if env_var_entry is None:
                    # The existing container definition did not contain environment variables
                    if container_definitions[container_index].get('environment') is None:
                        container_definitions[container_index]['environment'] = []
                    # This env var did not exist in the existing container definition
                    container_definitions[container_index]['environment'].append({'name': env_var_name, 'value': env_var_value})
                else:
                    env_var_index = env_var_entry['index']
                    container_definitions[container_index]['environment'][env_var_index]['value'] = env_var_value
            elif key and key not in ['container', 'name', 'value']:
                raise ValueError(
                    'Incorrect key found in environment variable update parameter: ' + key)
    except ValueError as value_error:
        raise value_error
    except:
        raise Exception(
            'Environment variable update parameter could not be processed; please check parameter value: ' + container_env_var_updates)

    # Expected format: container=...,image-and-tag|image|tag=...,container=...,image-and-tag|image|tag=...,
    try:
        image_kv_pairs = container_image_name_updates.split(',')
        for index, kv_pair in enumerate(image_kv_pairs):
            kv = kv_pair.split('=')
            key = kv[0].strip()
            if key == 'container':
                container_name = kv[1].strip()
                image_kv = image_kv_pairs[index+1].split('=')
                container_entry = container_map.get(container_name)
                if container_entry is None:
                    raise ValueError('The container ' + container_name +
                                     ' is not defined in the existing task definition')
                container_index = container_entry['index']
                image_specifier_type = image_kv[0].strip()
                image_value = image_kv[1].strip()
                if image_specifier_type == 'image-and-tag':
                    container_definitions[container_index]['image'] = image_value
                else:
                    existing_image_name_tokens = container_entry['image'].split(
                        ':')
                    if image_specifier_type == 'image':
                        tag = ''
                        if len(existing_image_name_tokens) == 2:
                            tag = ':' + existing_image_name_tokens[1]
                        container_definitions[container_index]['image'] = image_value + tag
                    elif image_specifier_type == 'tag':
                        container_definitions[container_index]['image'] = existing_image_name_tokens[0] + \
                            ':' + image_value
                    else:
                        raise ValueError(
                            'Image name update parameter format is incorrect: ' + container_image_name_updates)
            elif key and key not in ['container', 'image', 'image-and-tag', 'tag']:
                raise ValueError(
                    'Incorrect key found in image name update parameter: ' + key)

    except ValueError as value_error:
        raise value_error
    except:
        raise Exception(
            'Image name update parameter could not be processed; please check parameter value: ' + container_image_name_updates)
    return json.dumps(container_definitions)


if __name__ == '__main__':
    try:
        print(run(sys.argv[1], sys.argv[2], sys.argv[3]))
    except Exception as e:
        sys.stderr.write(str(e) + "\n")
        exit(1)

EOF

# Prepare container definitions
CONTAINER_DEFS=$(python $UPDATE_CONTAINER_DEFS_SCRIPT_FILE "$PREVIOUS_TASK_DEFINITION" "$CONTAINER_IMAGE_NAME_UPDATES" "$CONTAINER_ENV_VAR_UPDATES")

# Prepare script for getting task definition values
GET_TASK_DFN_VAL_SCRIPT_FILE=$(mktemp _get_task_def_value.py.XXXXXX)
chmod +x $GET_TASK_DFN_VAL_SCRIPT_FILE
cat > $GET_TASK_DFN_VAL_SCRIPT_FILE <<-EOF
from __future__ import absolute_import
import sys
import json


def run(element_name, task_definition_str):
    try:
        definition = json.loads(task_definition_str)
        task_definition = definition['taskDefinition']
    except:
        raise Exception('No valid task definition found: ' +
                        task_definition_str)
    str_list_types = ['requiresCompatibilities']
    json_arr_types = ['placementConstraints', 'volumes']
    if element_name in json_arr_types:
        output_value = '[]'
    else:
        output_value = ''
    if element_name in task_definition:
        element_value = task_definition[element_name]
        if element_name in str_list_types:
            for i, list_item in enumerate(element_value):
                output_value += list_item.strip()
                if len(element_value) - 1 > i:
                    output_value += ' '
        elif element_name in json_arr_types:
            output_value = json.dumps(element_value)
        else:
            output_value = str(element_value)
    return output_value


if __name__ == '__main__':
    try:
        print(run(sys.argv[1], sys.argv[2]))
    except Exception as e:
        sys.stderr.write(str(e) + "\n")
        exit(1)

EOF

# Get other task definition values
TASK_ROLE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'taskRoleArn' "$PREVIOUS_TASK_DEFINITION")
EXECUTION_ROLE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'executionRoleArn' "$PREVIOUS_TASK_DEFINITION")
NETWORK_MODE=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'networkMode' "$PREVIOUS_TASK_DEFINITION")
VOLUMES=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'volumes' "$PREVIOUS_TASK_DEFINITION")
PLACEMENT_CONSTRAINTS=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'placementConstraints' "$PREVIOUS_TASK_DEFINITION")
REQ_COMP=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'requiresCompatibilities' "$PREVIOUS_TASK_DEFINITION")
TASK_CPU=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'cpu' "$PREVIOUS_TASK_DEFINITION")
TASK_MEMORY=$(python $GET_TASK_DFN_VAL_SCRIPT_FILE 'memory' "$PREVIOUS_TASK_DEFINITION")

# Make task definition values available as env variables
echo "export CCI_ORB_AWS_ECS_TASK_ROLE='${TASK_ROLE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_EXECUTION_ROLE='${EXECUTION_ROLE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_NETWORK_MODE='${NETWORK_MODE}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_CONTAINER_DEFS='${CONTAINER_DEFS}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_VOLUMES='${VOLUMES}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_PLACEMENT_CONSTRAINTS='${PLACEMENT_CONSTRAINTS}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_REQ_COMP='${REQ_COMP}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_TASK_CPU='${TASK_CPU}'" >> $BASH_ENV
echo "export CCI_ORB_AWS_ECS_TASK_MEMORY='${TASK_MEMORY}'" >> $BASH_ENV

rm $UPDATE_CONTAINER_DEFS_SCRIPT_FILE $GET_TASK_DFN_VAL_SCRIPT_FILE

An error occurred (ClientException) when calling the DescribeTaskDefinition operation: Unable to describe task definition.
Exited with code 255

Hi @arsia, could you provide a link to the job? :slight_smile:

Hi @stella! Unfortunately i cannot as it is a private repo. What specifically would you want to know/see?

Hi @arsia, could you open a support ticket here sharing the link to the unsuccessful job so that we can have easy access to the details of your job for investigation purposes but still keep them private? https://support.circleci.com/hc/en-us/requests/new

Hi @stella
I opened a ticket. I was wondering if you can have come across this issue before at all. I cannot pinpoint from the error stack where the problem is happening! Do you have an idea?