Rails system tests failing intermittently

rails
ruby

#1

Is anyone else having constant trouble with intermittent system test failures? I’ve tried setting Capybara.default_max_wait_time = 15 #seconds which seems like a long time to wait, but it still happens.


#2

I don’t use Rails, but in general, intermittent failures are a problem with the tests rather than the CI. CI is just a VM with some tools in it, and so is unlikely to affect something at the Ruby level.

Is Capybara a browser test system with a headless browser? That should probably generate some logs - can we see them? Unless your app is particularly slow, 15 seconds is definitely too long to wait.


#3

Yes, Capybara drives headless Chrome via selenium. I never get these sorts of failures locally (macOS), only on Circle. They are things like

Failure/Error: expect(page).to have_content membership.number
expected to find text “9597DD55” in “”

Capybara generates a screenshot but it’s just blank so it seems like the page isn’t rendering for any reason.


#4

You might need to sit down with a SSH session to Circle (and a large mug of coffee) for a debugging session :smiley_cat: (I don’t know how much one can do that with Ruby, but hacking with PHPUnit and a console editor is possible, and sometimes required!)

Can you get any part of your app to render, or are all of your app endpoints failing? Can you get to your app’s web server logs to see if it is receiving the request and failing? Perhaps the web app is crashing.


#5

Thanks, I’ll see what I can do about increasing logging verbosity. The weird thing is that I have a bunch of system tests and it’s just 1 or 2 that fail at a time (and different ones every time, or sometimes they all pass).


#6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.