RSpec tests time out after upgrading to ruby 2.6.x

I have a test suite which is currently running perfectly with Ruby 2.5.3.

I have a branch to update the version of Ruby to 2.6.1. In this branch all of the tests pass successfully, and the test output is identical to that of the 2.5.3 branches. However, after completing the tests with Ruby 2.6.1, things just hang and eventually time out after 10 minutes.

The end of the RSpec tests in the log look like this:

Finished in 7 minutes 37 seconds (files took 3.28 seconds to load)
513 examples, 0 failures, 2 pending

RSpec is shutting down and will print the summary report... Interrupt again to force quit.
Too long with no output (exceeded 10m0s)

I’ve run the tests manually on the CircleCI image over SSH and get the same behaviour with the tests not exiting once completed. Running them on my local machine does not provide the same behaviour though, they complete normally here.

After upgrading to Ruby 2.6.x is there something else I should be changing on the CircleCI config in order to get these to work properly?

Is your version of RSpec exactly the same, or was that upgraded too?

Yep - identical. No changes other than the version of Ruby.

Image wise, I’ve switched from circleci/ruby:2.5.3-node-browsers to circleci/ruby:2.6.1-node-browsers

I’ve no experience with Ruby, so just chucking ideas in the pot. Can you write the simplest possible test (true === true) in RSpec and replicate this issue in a separate repo? If that hangs then I’d regard it as an RSpec bug.

Are you doing any headless browser testing in RSpec that would require some complicated/slow shutdown?

Well, here’s the thing. If I run a subset of the tests on the command line they pass and complete correctly. If I run the entire test suite on the command line it hangs after the tests finish.

So, something is causing it to hang, and I very much doubt it’s the fault of RSpec. The strange thing is that it doesn’t happen when using Ruby 2.5.3.

Here’s an earlier question :slight_smile:

Yes, using chrome, capybara/selenium

Does RSpec have a verbose/debug mode that can give you a more accurate idea of what it is doing?

Not that I’m aware of. The most detailed output is the documentation formatter, but there aren’t any RSpec errors reported. As I said, the test suite completes successfully, but RSpec hangs and doesn’t shut down.

I’ve just run the test suite again with SSH enabled, and managed to get it to complete successfully by killing the following process:

/bin/sh /usr/bin/xdg-open /home/circleci/<project_name>/tmp/capybara/capybara-201902250901525185555611.html

The content of that file was the HTML from the last system test executed. Nothing strange content wise. It just seems that on the final test, capybara just hangs around waiting for more instructions.

I’m not the best at ruby either, but I know I’ve seen related issues on javascript. Are you telling selenium to shut down the browser after the test suites? If that’s not happening I can see rspec possibly getting stuck waiting for it to return.

Another thing you can try is using VNC and SSH to watch the tests and see exactly what’s happening

No - I’m not specifically telling it to shut down. I’ve never had to do that before. These tests are working fine with Ruby 2.5.3.

I recommend the SSH/VNC path then. Something seem to have changed between 2.5 and 2.6, but I don’t know would it would be.

I’m not aware of any changes between those two images besides the ruby version, you can view the Dockerfiles used to create them here

1 Like

OK - so I managed to get VNC working and watched everything running on the server. For some reason one of my tests was spitting out a screenshot, which was holding the browser window open and causing the tests to time out.

Very strange that this wasn’t a problem on Ruby 2.5.3 to be honest, as that is showing the same behaviour but just not hanging at the end.

In any case … I’ve rewritten the troublesome test and all is now good in Ruby 2.6.1.

Thank you both for your help!


This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.