'Downloading previous test results' step never appears and leads to always missing timings

We are having trouble splitting tests by timing information. It appears to always complain about missing timings file. And that is understandable as the ‘Downloading tests by timings’ step never appears in the workflow.
We have a following CI configuration:

workflows:
  test:
    .. triggered by tags push
    jobs:
    - ..some setup
    - test-job
...
jobs:
  test-job:
    executor: our-executor
    steps:
    - ..mostly commands
    - test-command
      ..parameters
...
commands:
  test-command:
    steps:
    - ..setup steps
    - ..suite splitting using 'circleci tests run' command like so:
        cat ./tmp/all-tests.txt | circleci tests run --command="xargs echo > ./tmp/tests.txt" --split-by=timings --timings-type=filename
    - run tests from the files
    - store_test_results:
      path: ./results

What works:

  1. Splitting by name works
  2. Rerunning only failed tests also works
  3. Insights work: all tests have correct relative file value and timings (It is a pain to navigate there as the workflow does not seem to be attached to any branch, since it’s triggered by a unique tags every time, so I found a way to view insights for null branch and it’s ok)
  4. Flakiness detection works

What doesn’t work:

  1. There is no ‘Downloading previous test results’ step in the job ever
  2. The splitting by timing always complains about the missing file because of the p.1

I’m asking here because there are too many components in this and I don’t have insights on how the hidden magic works. Please advise. Mainly the question is — why doesn’t the ‘Downloading previous test results’ step does not appear?

  • Is it because the workflow is always triggered by a tag and never runs attached to a branch?
  • Is it because the saving test results is in the command steps and not in the job steps?
  • Is it because of the custom executor?

We’ll appreciate any help. Thank you!

Hi @sergey-kruk-cobalt ,

When the store_test_results runs, do you see confirmation in the step details that the file was successfully uploaded?

Hi @yannCI !

Yes, I see that it’s uploaded and I see the results in insights and if tests fail in tests tab.

@yannCI

UPDATE: I can see now that this ‘Download previous test results’ step appears in workflows triggered by a push to a branch. But never in a workflow triggered by a push of a tag. The jobs in these two workflows use exactly the same command and the same executor.

I guess it’s a bug or a feature request? When pushing a tag try downloading results from a job with the same name?