Adding custom data to `$CIRCLE_INTERNAL_TASK_DATA/circle-test-results/results.json`

[juanca]

Hello!

I had a crazy idea that I think is possible. I would like to partition my test job’s containers by individual tests.

My mono-repo has widely varying test files (seconds to tens of minutes). Partitioning my test job by files does not produce optimal results – a single container can take up to 30 minutes. I was able to tame the largest files by splitting them into smaller individual files. However, this is not sustainable and produces a pattern that is not enforceable.

I was hoping I could just configure my CI containers to run specific tests. I’ve gone as far as producing XML files (JUnit compatible) with the appropriate information – e.g. filepath, line number, and timing data.

At the moment, I cannot seem to figure out how to ensure these additional pieces of data are copied into $CIRCLE_INTERNAL_TASK_DATA/circle-test-results/results.json. I am inferring that it is a computed file from all my test results given its internal structure.

For example, my XML file has a testcase that looks like the following:

<testcase classname="spec.selenium.some_spec" name="SomeSpec has a spec" 
file="./spec/selenium/some_spec.rb" line_number="57" time="12.326568"></testcase>

But the aggregated JSON results file looks like the following:

{
  "tests": [
    {
      "classname": "spec.selenium.some_spec",
      "file": "spec/selenium/some_spec.rb",
      "name": "SomeSpec has a spec",
      "result": "success",
      "run_time": 12.326568,
      "message": null,
      "source": "unknown",
      "source_type": "unknown"
    },
  ],
  "files": [
    "selenium.xml",
  ],
  "exceptions": []
}

1. Is there any documentation about this particular file?
2. Is there a way to persist additional data in this aggregated file?
3. Is there a way to persist additional files other than the aggregated file?

Any advice would be appreciated!

Noteworthy documentation snippet:

Under Splitting by Timings Data

If you need to manually store and retrieve timing data, use the store_artifacts step.

This might help me store custom timing data between builds but it seems like an incorrect piece of documentation – is there a restore_artifacts step?

[juanca]

Is there another avenue for feedback from CircleCI? Does this deserve a direct email to support?

Update: There is a different section of CircleCI for feature requests. I added this post as an idea @ https://circleci.com/ideas/?idea=CCI-I-706