Workflows - Very similar workflows, reducing redundancy in configuration

workflow

#1

Hello. We have a config file with a lot of redundancy across two builds. Is there a way to merge these and reduce redundancy somehow?


#2

Yep! Search for “YAML references” on this board - it’s a YAML feature built into all compliant parsers.


#3

Hi. I’ve done almost everything else, but I can’t seem to get the checksum into an environment variable to use. It tells me it can’t convert !!map to string.

Alternatively, I can’t put a “template tag inside a template tag” either.


#4

Would you show what you have, on this board? A code/config formatting tool is available. (I tend to discourage external links, as they as susceptible to breakage, which disadvantages future readers).


#5
version: 2

jobs:
  test_steps: &default_test_steps
    - checkout
    - restore_cache:
      keys:
        - miniconda-v1-${ENV_CHECKSUM}
    - run:
        name: Installing Miniconda
        command: |
            if [ ! -d "/home/circleci/miniconda" ]; then
              wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
              bash miniconda.sh -b -p $HOME/miniconda
              export PATH="$HOME/miniconda/bin:$PATH"
              conda config --set always_yes yes --set changeps1 no
            fi
            sudo chown -R $USER.$USER $HOME
    - run:
        name: Configuring conda
        command: |
            export PATH="$HOME/miniconda/bin:$PATH"
            if [ ! -d "/home/circleci/miniconda/envs/${ENV_NAME}" ]; then
              conda update -q conda
              conda env create -f ci/environment-${PYTHON}.yml --name=${ENV_NAME}
              source activate ${ENV_NAME}
            fi
            conda env list
            conda list ${ENV_NAME}
    - save_cache:
        key: miniconda-v1-${ENV_CHECKSUM}
        paths:
          - "/home/circleci/miniconda"
    - run:
        name: Installing package
        command: |
          export PATH="$HOME/miniconda/bin:$PATH"
          source activate ${ENV_NAME}
          pip install -e .[tests]
    - run:
        name: Running unit tests
        command: |
          /home/circleci/miniconda/envs/${ENV_NAME}/bin/pytest
    - store_test_results:
        path: test_results
    - run:
        name: Uploading coverage
        command: |
          export PATH="$HOME/miniconda/bin:$PATH"
          source activate ${ENV_NAME}
          codecov

  build_36:
    machine: true
    environment:
      PYTHON: "3.6"
      ENV_NAME: "py36-sparse-test"
      ENV_CHECKSUM: {{ checksum "ci/environment-3.6.yml" }}
    steps: *default_test_steps

  build_27:
    machine: true
    environment:
      PYTHON: "2.7"
      ENV_NAME: "py27-sparse-test"
      ENV_CHECKSUM: {{ checksum "ci/environment-2.7.yml" }}
    steps: *default_test_steps

workflows:
  version: 2
  run_all_pythons:
    jobs:
      - build_36
      - build_27

Here you go! Thanks so much for the help.


#6

Ah, I don’t know if you can do that. The checksum device is for the cache system - I don’t know if you can use that template syntax in env vars.

I would start by trying to set ENV_CHECKSUM using export in a standard run step. I can’t promise that’s the right lead to follow, but it is where I would start.


#7

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.