How do I make a background step fail the job?

circle.yml
#1

Hello,

In my job I am running the flake8 linter in a docker container as a background step while my other tests are running:

      # Run linting tests
      - run:
          name: Run flake8
          background: true
          command: eval ${docker_compose} run flake8

      # Run pytest tests using docker-compose
      - run:
          name: Tests
          command: eval ${docker_compose} run pytest

However, I have noticed that if the linting fails (and correctly exits with a non-zero exit code) but the other tests are green, then the whole job will end up green, despite the fact that one of the steps is red.

I spent a long time looking in the documentation to whether this was expected but could not find anything. Then, I found this forum post:

This is pretty annoying. Is there any trick I can do to ensure that the whole job fails?

#2

A step that is backgrounded ought to fail only if running that command fails completely. Once the job passes to background, it will be marked as successful, and the outcome of that process cannot retroactively change the outcome of the step.

I would guess it would be technically possible to change this, and I can see why you’d want it, but it would probably break things for other users.

What you could do is the write the outcome of that step somewhere to the filing system, and then before the end of your job, wait for the background job to finish, and then throw an error if a fail is written by the backgrounded job. Perhaps you could do that after your tests?

The only problem you have is that, assuming your tests take longer than your linting, you will have to wait for your tests to complete even if your linting has failed. What you could do is background the tests too, and then in a final step enter a loop to monitor the file output, and exit/fail if either task returns an error.

#3

Guys. This is embarrassing and very confusing. the least you can do is update your documentation. I spent an hour refactoring my configuration, thinking i could run some steps in background and continue to the next command (independent steps).

Let me even help you:
The background attribute enables you to configure commands to run in the background. Job execution will immediately proceed to the next step, setting the background executed step result as “Successful”, no matter what the result of the command was.

#4

As far as I know, one can. What did you try doing, and what did not work?

This is embarrassing and confusing. The least you can do is ask a clear question! :rofl:

#5

haha :slight_smile: i deserve it.
Anyhow - I ran a shell command with background on, it failed, but the build didn’t fail.
(Related to the original op question).
I too know how to run a command with &, I assumed there’s more logic behind running the command with background mode on, like waiting for the return code.

#6

To follow up: based on halfer’s reply I added a step at the end of job to check that my backgrounded step succeeded. It’s working just fine for me :slight_smile:

    - run:
        background: true
        command: |
          flake8
          touch .flake8-succeeded

    - run:
        command: pytest  # note: always takes longer than flake8

    - run:
        name: Check success of background steps
        command: "[ -f .flake8-succeeded ]"