Running browser tests

karma
chrome
angular
browsers
circle.yml

#1

Hello,
Could you guys provide us more documentation about running browser tests? We started using Angular some weeks ago and it uses Karma + Chrome/PhantomJS to run tests.

I’ve found this topic:

But in this topic an Ubuntu image is used, which does not come with Git and Node, so we need to install them and it will make the build much slower.

There are also some comments talking about using selenium/standalone-chrome, but i didn’t find any documentation about it.


#2

Hey @iagomelanias, can you please share here one more bit of how you got it working with image suggested by @tom?

I mean if I have some base image of mine (node-slim based), do I need to add any other steps to get npm test to run?

  - image: some-custom-ci-image:1
  - image: selenium/standalone-chrome:3.1.0

It looks like selenium/standalone-chrome should start selenium on standard port 4444, but how do I tell npm test to look for it at that port? I’m getting an error of npm searching for google chrome to start:

$ npm test
...
Built project successfully. Stored in "/root/app/tmp/core_object-tests_dist-BJYeyChg.tmp".
not ok 1 Error
    ---
        message: >
            Launcher Chrome not found. Not installed?
    ...
...
Launcher Chrome not found. Not installed?

Do I miss any additional steps or arguments to npm test?


#3

Answering myself: built custom image which is based off node and adds bits of standalone-chrome images into a base one and got tests running simply via xvfb-run (we use Ember):

      - run:
          name: Run Tests
          shell: /bin/bash --login
          command: xvfb-run ember test --silent -r xunit > /tmp/reports/ember-test.xml


#4

Hi @alexander,
I didn’t get it working still. I’m going to open a ticket to get some help.

Maintain a custom image seems to not be a good idea since we can use different node versions for different apps and upgrade the node version used in the builds would be so much painful than it should be.

If i get a better solution, i will post here.


#5

Hey there all. I have browser tests working in chrome and firefox. Just thought I would share my Dockerfile and config. Please note we are intentionally using an old version of firefox. I would encourage you to replace that build step with one to install the appropriate version of Firefox for your application.

Edit: also, as noted below by Eric, you will need to pass the --no-sandbox flag to chrome either the way he suggests or at runtime.

Dockerfile:

FROM ubuntu:16.10

# Install some dependencies
RUN apt-get update -y
RUN apt install -y dirmngr wget git curl apt-utils

# Install Java
RUN apt-get install -y default-jre

# Install node 6.x
RUN curl -sL https://deb.nodesource.com/setup_6.x | bash -
RUN apt-get install -y nodejs

# Install Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - 
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get update
RUN apt-get install -y google-chrome-stable

# Install Firefox
RUN apt-get purge -y firefox
RUN wget https://downloads.sourceforge.net/project/ubuntuzilla/mozilla/apt/pool/main/f/firefox-mozilla-build/firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb
RUN dpkg -i firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb

# Install Xvfb
RUN apt-get install -y xvfb

CMD ['echo', 'Done!']

circle.yml:

version: 2
jobs:
  build:
    docker:
      - image: mydockerimage:latest
        env:
          - DISPLAY=:99
          - CHROME_BIN=/usr/bin/google-chrome
    steps:
      # some steps here to install dependencies
      - run:
          name: Run tests
          shell: /bin/bash --login
          # obviously your test command will be different but just prefix it with xvfb-run
          command: xvfb-run grunt test

Karma jasmine tests under 2.0
#6

@iagomelanias Maintaining a custom image is not a big deal in our case. Just use different images for applications that need different dependencies. Updating the node version and pushing a new tag to dockerhub would only take a few minutes, and how often do you need to do that? In our case not very often. The performance benefit of using an image with your dependencies pre-installed is huge compared to installing node as part of every single build (our build time went from 15+ min on circle 1.0 to 5-6 min on circle 2.0 just from having all dependencies pre-installed in the docker image rather than set up every build from scratch)


#7

As I’ve helped some customers get onto 2.0, I’ve seen Chrome tests fail to start. We disable Chrome-sandboxing in 1.0, and I’ve found it sometimes necessary in 2.0. This is what I have in a Dockerfile. This is on a ruby:2.3.1 Docker image, which has a Debian base, and installs google-chrome via apt-get. You may need to adjust this command if you manually install Chrome or have a different OS for your Docker image.

# Disable sandboxing - it conflicts with unprivileged containers
RUN sed -i 's|HERE/chrome"|HERE/chrome" --disable-setuid-sandbox --no-sandbox|g' \
           "/opt/google/chrome/google-chrome"

#8

Ah yes! Forgot that part. We pass the --no-sandbox flag to chrome in our config at runtime. Either method works. You are right that it didn’t run properly without it.


#9

I’m having this issue while using the --no-sandbox flag.

This is my command to run karma:
karma start karma-webpack.conf.js --single-run --no-auto-watch --no-sandbox --browsers ChromeHeadless

And I get the following in CircleCI logs:

02 08 2017 16:24:42.033:INFO [karma]: Karma v1.6.0 server started at http://0.0.0.0:4444/
02 08 2017 16:24:42.035:INFO [launcher]: Launching browser ChromeHeadless with unlimited concurrency
02 08 2017 16:24:42.066:INFO [launcher]: Starting browser ChromeHeadless
02 08 2017 16:25:42.126:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
02 08 2017 16:25:44.129:WARN [launcher]: ChromeHeadless was not killed in 2000 ms, sending SIGKILL.
02 08 2017 16:25:46.132:WARN [launcher]: ChromeHeadless was not killed by SIGKILL in 2000 ms, continuing.

I receive these errors regardless if I use Chrome or ChromeHeadless

My .circleci/config.yml file:

version: 2

jobs:
    build:
        working_directory: ~/nexgen
        # The primary container is an instance of the first list image listed. Your build commands run in this container.
        docker:
            - image: circleci/node:6
              environment:
                  CHROME_BIN: "/usr/bin/google-chrome"
        steps:
            - checkout
            - run:
                name: Install yarn
                command: sudo npm install -g yarn
            - restore_cache:
                key: dependency-cache-{{ checksum "package.json" }}
            - run:
                name: Install node_modules with yarn
                command: yarn
            - save_cache:
                key: dependency-cache-{{ checksum "package.json" }}
                paths:
                    - ./node_modules
            - run:
                name: Run unit tests with karma
                command: yarn test-single-headless
            - store_test_results:
                path: test-results.xml

UPDATE:
Fixed this by simply updating my image to include browsers:

docker:
            - image: circleci/node:6-browsers
              environment:
                  CHROME_BIN: "/usr/bin/google-chrome"

#10