Access other parallel containers in same job

Build Environment Running Tests
,
1

I have a build much like:

version: 2.1
jobs:
  rspec:
    parallelism: 10
    docker:
      - image: 'circleci/ruby:2.6'
    steps:
      - checkout
      - run: bin/rspec spec # in practice, this is balanced with knapsack today

I’d like to balance specs across these nodes. I understand there are a ton of strategies and have been exploring them. One that I haven’t yet found would be to use a gem like rspec-multiprocess_runner to have one node coordinate and 9 nodes dynamically split the work.

The key requirement to enable this would be for my 10 parallel nodes to be able to communicate with each other.

I’m familiar with using multiple containers in one node (e.g. postgres and ruby containers to run tests against a DB), and that works fine, but would not seem to work when steps are required to be executed on each node - not just a “dumb” server.

Pseudocode for what I’d like:

version: 2.1
jobs:
  rspec:
    parallelism: 10
    docker:
      - image: 'circleci/ruby:2.6'
    steps:
      - checkout
      - (restore caches, etc.)
      - run:
        command: |
          if $CIRCLE_NODE_INDEX -eq 0; then
            run_headnode_config # listen for workers, distribute work
          else
            run_childnode_config # connect to node0 as a worker ready to run rspec commands
          end