Hello,
I wanted to bring to attention a UX problem that had caused a lot of unintentional process failures on our side and our team
When tests are being reported on Github checks, the underlying link (https://circleci.com/gh/[ORG]/[Config]/[build_ID] brings you to your workflow stage and autoscrolls to the last failing step of the FIRST container
Clicking the github checks brings you there by auto-scrolling after opening the page:
Test suites often report the total number of failures, and you might be tempted to just scroll a bit higher, review the failing tests, and because of flakiness or coordination with other PRs, assume everything is okay-ish.
At this point you might be tempted to merge despite what you think are few failing or flaky specs(*) . BUT you might be missing the fact that your build is parallelized, and that there may be a lot of other failing containers, accounting for a lot more failing tests. Which was the case for the above PR :
[when scrolling to the top : ]
This is a UX problem that leads developers into unintentionally merging a lot of failures. I have 2 suggestions :
-
DO NOT autoscroll when there are more than one failing container
-
Keep autoscrolling, but somehow float, or hint the developer than there are many more failures than the current (for example the tab list could be in a fixed/sticky CSS position)
Thank you for your consideration regarding this.
I have also found a related topic Jump-to-failed-step ends up hiding parsed test results but which did not seem so critical (likely because of no parralelization)
(*) we are trying to merge only green builds as best software engineering practices recommends, but sometimes you just dont want to rerun the full suite just for a flaky test or because someone else broken something that will be fixed in another PR