New version of Chrome causing problems in tests

chrome
headless
#1

Hi. Our CircleCI builds began failing inexplicably a couple of days ago, saying “Other element would receive the click” on certain automated browser tests on headless Chrome. After some debugging I discovered that this only happens on the latest stable version of Google Chrome (which apparently came out a couple of days ago). I am running the latest version of chromedriver as well.

I’d like to downgrade to the version of Chrome that last worked, but Google does not host earlier versions of Chrome.

Can anyone help me figure out what to do?

#2

Are you able to use Chromium? If so, there is a widget in this answer to get the correct Git hash from the official repo.

(For reference, that was found using this search).

#3

Thanks. This helped me debug further and it seems not to be a problem with the Chrome version.

It’s really a very strange issue. Although the failures happen locally as well, rarely, the strangest thing is that the tests seem to consistently fail in the 3rd and 4th instances in the CircleCI build, while passing in the 1st and 2nd instances.

Any thoughts on what could be happening to cause this?

#4

what could be happening to cause this?

It’s hard to advise. Remote browser tests are some of the hardest things to debug, and thus some of the hardest thing for volunteers to help with (short of pulling a repo and digging in for some hours themselves).

Moreover, I don’t think the meaning of this is clear:

the tests seem to consistently fail in the 3rd and 4th instances in the CircleCI build, while passing in the 1st and 2nd instances

Are “instances” parallel builders? I wonder if more detail (and code) is needed here.

If I were debugging this, I would:

  • Take a screenshot on failure and examine it, in case it shows that, for example, a failure to load a CSS file causes elements to incorrectly overlap (which might conceivably result in the error you have specified)
  • Rebuild the tests to run in Docker, so you can run them locally exactly as they are run remotely
#5

Yes, sorry if I had the wrong lingo. They are parallel builders. I’ve attached a screenshot if that helps clarify what’s going on.

Those are good suggestions - I’ve spent considerable time debugging the tests with SSH sessions, screenshots, etc. and improved them a great deal. I’ve also tried multiple back-versions of Chrome and Chromedriver. However, the 5 same tests consistently fail on only the 3rd and 4th instances (build passes on the 1st and 2nd).

It’s really bizarre to me that only the 3rd and 4th builders have this issue (also, this started pretty recently). Shouldn’t all the instances be identical? Is there something about the way the VMs are provisioned that would only affect the 3rd and 4th instances?

Edit: Update: I just did a run and this time the 2nd and 3rd builders failed, so it’s not specific to the 3rd and 4th slots, but it is always two that fail on the same 5 tests.

#6

Here’s my config:

#7

Well, I feel dumb. I didn’t realize that the tests are divided between the builders.

This is still an ongoing issue for me, but it seems less like it’s because of Chrome and more like an issue with my tests now. I got the help I needed to downgrade Chrome, so thanks for that!

2 Likes