Detecting & Reporting on Nondeterministic Tests


#1

I’d like to be able to search or list for tests from SHAs that have multiple builds, some succeeding and some failing.

Sometimes these nondeterministic tests do not immediately manifest themselves. They would only be visible after multiple repeated runs. Such a feature would list and identify problematic long-lived tests that occasionally fail without explanation. Although the failures not be fully reproducible, at least this leaves a trail to start investigating.


#2