Build failures seem to wipe out test timings for subsequent builds


We use the circleci tests splitter to split by timings, and it seems that whenever we have a failed build, the build immediately after it does a very poor job splitting files among containers.

This only seems to the first build after a failed build. Even if that (second) build fails though, the next build seems to get the correct splits. E.g.

Build 1 <-- fails
Build 2 <-- will be balanced very badly
Build 3 . <-- balanced much better (even if #2 fails)

Build examples (in case helpful, I’m sure you can look up my company/project)
#48375 (failed)
#48376 (badly balanced, also failed)
#48380 (balanced much better)

Is this related to, or a duplicate of, this question?

No, they’re separate issues. I filed both of these.

That one has to do with under-reporting test times when splitting by file.

This one is about what build circle bases timing decisions off of.

1 Like