I have a test suite which is currently running perfectly with Ruby 2.5.3.
I have a branch to update the version of Ruby to 2.6.1. In this branch all of the tests pass successfully, and the test output is identical to that of the 2.5.3 branches. However, after completing the tests with Ruby 2.6.1, things just hang and eventually time out after 10 minutes.
The end of the RSpec tests in the log look like this:
Finished in 7 minutes 37 seconds (files took 3.28 seconds to load)
513 examples, 0 failures, 2 pending
RSpec is shutting down and will print the summary report... Interrupt again to force quit.
Too long with no output (exceeded 10m0s)
I’ve run the tests manually on the CircleCI image over SSH and get the same behaviour with the tests not exiting once completed. Running them on my local machine does not provide the same behaviour though, they complete normally here.
After upgrading to Ruby 2.6.x is there something else I should be changing on the CircleCI config in order to get these to work properly?
I’ve no experience with Ruby, so just chucking ideas in the pot. Can you write the simplest possible test (true === true) in RSpec and replicate this issue in a separate repo? If that hangs then I’d regard it as an RSpec bug.
Are you doing any headless browser testing in RSpec that would require some complicated/slow shutdown?
Well, here’s the thing. If I run a subset of the tests on the command line they pass and complete correctly. If I run the entire test suite on the command line it hangs after the tests finish.
So, something is causing it to hang, and I very much doubt it’s the fault of RSpec. The strange thing is that it doesn’t happen when using Ruby 2.5.3.
Not that I’m aware of. The most detailed output is the documentation formatter, but there aren’t any RSpec errors reported. As I said, the test suite completes successfully, but RSpec hangs and doesn’t shut down.
I’ve just run the test suite again with SSH enabled, and managed to get it to complete successfully by killing the following process:
The content of that file was the HTML from the last system test executed. Nothing strange content wise. It just seems that on the final test, capybara just hangs around waiting for more instructions.
I’m not the best at ruby either, but I know I’ve seen related issues on javascript. Are you telling selenium to shut down the browser after the test suites? If that’s not happening I can see rspec possibly getting stuck waiting for it to return.
OK - so I managed to get VNC working and watched everything running on the server. For some reason one of my tests was spitting out a screenshot, which was holding the browser window open and causing the tests to time out.
Very strange that this wasn’t a problem on Ruby 2.5.3 to be honest, as that is showing the same behaviour but just not hanging at the end.
In any case … I’ve rewritten the troublesome test and all is now good in Ruby 2.6.1.
