Builds failing without any actual test failures


#1

The feedback is “Your build ran 5085 tests in RSpec with 0 failures”, yet the suite fails.

This has happened 3 times this morning.
As a control, I re-ran a build that passed yesterday. This morning it fails.


#2

This build passed yesterday: https://circleci.com/gh/Goldbely/uss-goldbely/19231

And here it fails, while showing 0 failures: https://circleci.com/gh/Goldbely/uss-goldbely/19240


#3

I’ve also tried running the build with 2x the concurrency, lest this be a RAM issue. Still failed. https://circleci.com/gh/Goldbely/uss-goldbely/19245

I’ve also run the full command for a single container on my local machine to confirm that it passes.


#4

Running the test suite via SSH also shows that all tests pass.


#5

My current hunch is that this issue stems from the “returned exit code 1” I see at the bottom of the container. This error prints to STDOUT after the suite of tests finishes:

/home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:ininitialize’: A JSON text must at least contain two octets! (JSON::ParserErro
r)
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:in new' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:155:inparse’
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/codecov-0.1.1/lib/codecov.rb:187:in format' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/result.rb:46:informat!'
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/configuration.rb:159:in block in at_exit' from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/defaults.rb:60:incall’
from /home/ubuntu/uss-goldbely/vendor/bundle/ruby/2.2.0/gems/simplecov-0.10.0/lib/simplecov/defaults.rb:60:in block in <top (required)>'


#6

We’ve fixed the issue by updating the CodeCov gem to version 0.1.4.

It’s still a mystery as to why this caused failures to start happening yesterday.

I’d like to thank myself and my co-worker Ajay for solving this. Nice work.


#7

well done!

seriously, great work filing the issue and documenting what you were finding!


#8