Allow failure of a run step

[Onumis]

I have some flickering specs (integration specs that rely on browser / JS).

One way to avoid these flickering tests from stopping the workflow and requiring me to do a rerun, is to have a step that runs the test suite again for the specs that failed:

      - run:
          name: RSpec interaction tests
          command: |
            bundle exec rspec [--flags] ./interaction

      - run:
          name: RSpec interaction tests (2nd run)
          when: on_fail
          command: |
            bundle exec rspec [--flags] --only-failures ./interaction

But this does only half of what I need:

As you can see, the 2nd pass runs when the 1st one failed.
And is now green, but this job was still marked as failed and the workflow stopped.

Is there a way to achieve what I want?

I guess that I can do something like

      - run:
          name: RSpec interaction tests
          command: |
            bundle exec rspec [--flags] ./interaction
            bundle exec rspec [--flags] --only-failures ./interaction

But it is not as explicit why I’m doing this, the 2nd rspec command will run every time, even when the 1st run had no errors (taking up some seconds of job execution), and perhaps the job spec has some config that fits this use case.

[halfer]

I would go with a variant of the second example, but put a Bash conditional in it.

My Bash is extremely rusty, so do something like this pseudocode:

bundle exec rspec [--flags] ./interaction
failed=(last task failed)
if (failed) then
    bundle exec rspec [--flags] --only-failures ./interaction
endif

A quick bit of DuckDuckGo-ing will help you fill in the blanks.

Edit

Ah, using bright ideas from here, a simple syntax that even I can cope with:

bundle exec rspec [--flags] ./interaction || \
    bundle exec rspec [--flags] --only-failures ./interaction

This uses the “short-circuit logical OR” to do what you want to do.

[Onumis]

Thanks! That works indeed. Would be great if circleCI had some particular syntax for that logic, but will use that solution for now.

[drazisil]

Hi Nuno. I think this feature request https://ideas.circleci.com/ideas/CCI-I-558 is what you are looking for. If it’s not, can you create a new one explain your use case for our product team to take a look at?

[halfer]

No worries

Would be great if circleCI had some particular syntax for that logic

Maybe, but it depends on whether that would be easier (to read or write) than shell scripting. Shell scripting is well known and well tested, and not having a specialised syntax means there’s less for CircleCI to maintain.