Front end tests failing in circleci but passing locally

Build Environment
,
1

Hello!

I have some testcafe tests that run against my frontend and make various assertions. All of them pass in my local environment, however 1 of them continues to fail in circleci. The 1 that fails did pass yesterday (when I was trying to merge) but now that I’ve merged the branch into master, and circleci tries to rebuild, it’s failing again.

Here is my config.yml file for reference:

## Customize the test machine
machine:
  python:
    version: 3.6.1
  node:
    version: stable
  npm:
    version: stable

## Customize dependencies
dependencies:
  pre:
    - curl -L -o google-chrome.deb https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
    - sudo dpkg -i google-chrome.deb
    - sudo sed -i 's|HERE/chrome\"|HERE/chrome\" --disable-setuid-sandbox|g' /opt/google/chrome/google-chrome
    - rm google-chrome.deb
    - pip install -r requirements.txt
    - npm install -g testcafe

## Customize test commands
test:
  override:
    - coverage run --source='.' manage.py test
    - testcafe chrome app_1/tests_frontend.js --app "waitress-serve --port=8000 forestroad.wsgi:application"
    - testcafe chrome app_2/tests_frontend.js --app "waitress-serve --port=8000 forestroad.wsgi:application"

I’ve tried rebuilding without the cache, I’ve tried upgrading chrome in the ci container (as seen above) but nothing seems to work.

Here is the test that’s failing. It’s supposed to open the menu bar and close it and assert that the class that sets the menu bar as visible is no longer present.

 ✖ Menu Behavior - Close

   1) The element that matches the specified selector is not visible.

      Browser: Chrome 59.0.3071 / Linux 0.0.0

         310 |
         311 |test('Menu Behavior - Close', async t => {
         312 |    await t
         313 |        .click(Selector('nav').child(0).child(0).child(0))
         314 |        // Close the menu
       > 315 |        .click(Selector('#menu').child(1))
         316 |        .expect(
         317 |            Selector('body').hasClass('is-menu-visible')
         318 |        ).eql(false)
         319 |});
         320 |

This works fine locally so I’m not sure what’s going on in the build environment…

2

Update: I’ve just encountered another scenario where the build and tests against my branch work fine, but the tests fail after I’ve merged.