Builds failing without any actual test failures

The feedback is “Your build ran 5085 tests in RSpec with 0 failures”, yet the suite fails.

This has happened 3 times this morning.
As a control, I re-ran a build that passed yesterday. This morning it fails.

This build passed yesterday: https://circleci.com/gh/Goldbely/uss-goldbely/19231

And here it fails, while showing 0 failures: https://circleci.com/gh/Goldbely/uss-goldbely/19240

I’ve also tried running the build with 2x the concurrency, lest this be a RAM issue. Still failed. https://circleci.com/gh/Goldbely/uss-goldbely/19245

I’ve also run the full command for a single container on my local machine to confirm that it passes.

Running the test suite via SSH also shows that all tests pass.

My current hunch is that this issue stems from the “returned exit code 1” I see at the bottom of the container. This error prints to STDOUT after the suite of tests finishes:

/home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:ininitialize’: A JSON text must at least contain two octets! (JSON::ParserErro
r)
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in new' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:inparse’
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/codecov-0.1.1/lib/codecov.rb:187:in format' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/result.rb:46:informat!'
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/configuration.rb:159:in block in at_exit' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/defaults.rb:60:incall’
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/defaults.rb:60:in block in <top (required)>'

We’ve fixed the issue by updating the CodeCov gem to version 0.1.4.

It’s still a mystery as to why this caused failures to start happening yesterday.

I’d like to thank myself and my co-worker Ajay for solving this. Nice work.

1 Like

well done!

seriously, great work filing the issue and documenting what you were finding!

1 Like