Test results accumulating for rebuilds


#1

Hey,

first of all - kudos for the great work with 2.0, I’m loving the layer caching!

We’re using the test-results-store step in our config (docker executor, remote docker setup, reusable environment) and it seems the test results are accumulating for rebuilds over the same commit.

The step output includes a path that looks like this:
s3://circle-production-customer-artifacts/picard/hash-1/hash-2-0-build/raw-test-results

Hash-1 doesn’t seem to change over time. Hash-2 is identical to the one in CIRCLE_ENV, and only seems to change with the next commit. Thus if I re-run the same build, I can see the test results from the past build, too.

Is this intended or there something I’m missing/configuring wrong?

Thanks!
Nada


#2

This sounds like intended functionality. Is it causing any problems?


#3

It’s just the test results accumulate so technically they don’t have to be related to that particular build but multiple builds.

Two scenarios come to mind:

  • flaky tests - if a build fails and then succeeds after re-running, you’ll see the results for both builds at your most recent build
  • development branch - amended commits and force pushing

I searched some more and noticed this feature request: Path and environment variables for test-results-store. I think we could use this to separate the build results further…


#4

My assumption on the hash-2 changing with a new commit was incorrect, I’ve just pushed a new commit to my branch and the CIRCLE_ENV var changed to one of its historic values, thus showing irrelevant results.


#5

Can you link me to where you’re seeing that? I can open up a bug report internally.


#6

My apologies, the test results were actually accumulating because I persisted the data container volumes on the host (primary container), like so:

volumes:

  • /tmp/junit:/appdata/junit

Once I removed the host part of the volume definition, the old data stopped appearing in my test results.

Interestingly enough, prior to running any build/tests, I was calling docker container prune and docker volume prune which both claimed to reallocate some free space but still left files dangling…

Anyway, sorry to cause panic!


#7

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.