Parallel tests with RSpec

rspec

#1

I’m not quire sure how to get parallel tests working in CircleCI 2.0. I have the following step specified at the end of my config.yml file (with parallelism: 4 set in the build definition), but it seems to still run every single test on each of the 4 running nodes:

- run:
    name: Tests
    command: |
      circleci tests glob "spec/**/*.rb" | \
      circleci tests split --split-by=timings | \
      bundle exec rspec

Am I missing something?


#2

Yes you are, but barely. You can’t pipe in the files like that, despite how nice that’d be. Maybe with xargs.

Either way, this is my solution:

      TESTFILES=$(circleci tests glob "spec/**/*.rb" | circleci tests split --split-by=timings)
      bundle exec rspec -- ${TESTFILES}

Worth noting: if you have too low of parallelism compared to the number of tests, you can hit a bash error where you’re passing too many parameters. It’s unlikely you’ll see it, but possible.

EDIT: Removed space after TESTFILES=.


#3

That’s possibly closer, but now every node fails with Exit Code 1:

$ TESTFILES= $(circleci tests glob "spec/**/*.rb" | circleci tests split --split-by=timings)
  bundle exec rspec -- ${TESTFILES}

/bin/bash: spec/{directory}/{file that failed}.rb: Permission denied

#4

Is the error from circleci or bundle?


#5

Looks like the error is from circleci, as I split out the command into several steps to figure out which one caused the failure, and the first line is the culprit:

$ TESTFILES= $(circleci tests glob "spec/**/*.rb" | circleci tests split --split-by=timings)

  /bin/bash: spec/{ directory }/{ file that failed }.rb: Permission denied
  Exited with code 1

#6

Can you run ls -l path/to/that/file.rb and link me to that build? I definitely want to report this as a bug.


#7

I’m a little saddened to think how any engineers probably looked at this without realizing. I can’t even take credit for this.

You have a space after TESTFILES= - just remove that space and you’re good to go. Bash is evaluating that in a way you don’t necessarily expect- it’s trying to execute your script as a bash script and define TESTFILES to nil.


#8

So the suggested solution works (with the space taken out), but I seem to always get this:

Requested historical based timing, but they are not present.  Falling back to name based sorting

I was under the impression that this timing data was automatically meant to be saved in /.circleci-task-data/circle-test-results/ and made available to successive builds.

I’m happy its at least parallelizing, but would be even happier if it did it using timing! :slight_smile:


#9

You need to upload them https://circleci.com/docs/2.0/configuration-reference/#storetestresults


#10

Okay, is there a specific location to upload it?

I’ve already got store_test_results in there, so will go back and double check.


#11

You just need to specify where your tests results are. It’s specific to your app/config, not our platform.


#12

Ah of course, works now! Thanks :slight_smile:


#13

#14

#15

This topic was automatically closed after 3 days. New replies are no longer allowed.