Tests fail but build continues and finishes successfully

docker
python

#1

Hi,

In the tests step, I have:

docker exec app python /app/tests/tests.py
echo $?

I have the tests intentionally fail at the moment by raising an exception. The echoed exit code is 1, which should stop the build and mark it as failed. Instead CircleCI continues and deploys/merges.

Is there a step I am missing? Is there something additional I should do so Circle knows the test failed?


#2

Are you using a multi-line yaml string for this set of commands? In other words, does this step look something like this?

        - type: shell
        command: | 
          docker exec app python /app/tests/tests.py
          echo $?

I ask since the echo $? will always mask the return result of the previous command, as will any other command. To get around this, you’ll want to use set -e at the top of each of these sections.

If that wasn’t it, could you DM me with a build link?


#3

Hi @Eric

I saw a similar thing. My coding standards test failed but the overall build still passed. Do I require the set -e command?

https://circleci.com/gh/culturekings/afterpay/216


#4

Hi @jared-fraser!

Yep, you want set -e at the top of each command: | section. Each of those sections is roughly equivalent to copying and pasting a multi-line script into a bash subshell. If you copied and pasted:

$(
ls asdf
ls asdf
)

You would get two errors. If you copy and paste this, you’ll get one:

$(
set -e
ls asdf
ls asdf
)

You’ll see one error. (Without the subshell wrapper, my terminal exits. I think it does something similar, but my terminal closed quickly, so I can’t be sure).


#5