Behat test passes locally but fails in CI. Both using same docker environment

acha5066 · June 12, 2018, 2:36pm

Hello.

I’m having a strange issue. I am using docker locally to run behat tests with selenium. These tests pass when I run the following command on my own machine:

docker exec -it my_drupal8_project_php bash -c 'cd behat && bin/behat'

If a test fails, it will take a screenshot. I’m a new user so can only post one image but the screenshot shows a real browser with CSS styling and the autocomplete indicating that javascript is working.

I am using the setup_remote_docker flag in circle ci so that I can replicate my local environment exactly in the CI environment.

That’s why I’m not sure why this is happening. When a build runs, a test that passes locally fails on the CI environment. The screenshot also looks completely different. No CSS and javascript is not working.

Here is my config.yml

    # PHP CircleCI 2.0 configuration file
version: 2
jobs:
  build:
    docker:
      - image: tmaier/docker-compose:18.02
    steps:
      - checkout
      - setup_remote_docker
      - run: docker-compose up -d

      # Install composer deps
      - run:
          name: "Install composer stuff"
          command: |
            docker cp . my_drupal8_project_php:/var/www/html
            docker exec -it my_drupal8_project_php ls
            docker exec -it my_drupal8_project_php whoami
            docker exec -it my_drupal8_project_php sudo chown -R wodby:wodby .
            docker exec -it my_drupal8_project_php composer install -n --prefer-dist
            docker exec -it my_drupal8_project_php bash -c 'cd behat && composer install --prefer-dist -n'

      # Install composer deps
      - run:
          name: "Install configuration"
          command: |
            docker exec -it my_drupal8_project_php bash -c 'drush cim -y'

      # set the path to the Drupal standard
#      - run: docker exec -it my_drupal8_project_php vendor/bin/phpcs --config-set installed_paths vendor/drupal/coder/coder_sniffer

      # set the standard to Drupal
#      - run: docker exec -it my_drupal8_project_php vendor/bin/phpcs vendor/bin/phpcs --config-set default_standard Drupal

      # Compile the javascript assets
#      - run:
#          name: "Compile javascript"
#          command: |
#            docker exec -it my_drupal8_project_php bash -c 'cd web/modules/custom/match/js && npm install'
#            docker exec -it my_drupal8_project_php bash -c 'cd web/modules/custom/match/js && npm run build'

      - run:
          name: Run behat tests
          command: |
            docker exec -it my_drupal8_project_php bash -c 'cd behat && bin/behat'

      - run:
          command: docker cp my_drupal8_project_php:/var/www/html/behat/screenshots/. ./screenshots/
          when: on_fail
      # Store screenshots for failed behat tests.
      - store_artifacts:
          path: ./screenshots
          destination: screenshots

      # run php codesniffer tests. see https://www.drupal.org/project/coder/issues/2867601
      - run: docker exec -it my_drupal8_project_php vendor/bin/phpcs --standard=Drupal --ignore="*.md,*.css,*.yml,*.txt,*.info" web/modules/custom

My docker-compose.yml

    version: "2"

services:
  mariadb:
    image: wodby/mariadb:$MARIADB_TAG
    container_name: "${PROJECT_NAME}_mariadb"
    stop_grace_period: 30s
    environment:
      MYSQL_ROOT_PASSWORD: $DB_ROOT_PASSWORD
      MYSQL_DATABASE: $DB_NAME
      MYSQL_USER: $DB_USER
      MYSQL_PASSWORD: $DB_PASSWORD
    volumes:
      - ./mariadb-init:/docker-entrypoint-initdb.d # Place init .sql file(s) here.
#      - /path/to/mariadb/data/on/host:/var/lib/mysql # I want to manage volumes manually.

  php:
    image: andrew/teamy:7.1-dev-4.4.2
    container_name: "${PROJECT_NAME}_php"
    environment:
      PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
      DB_HOST: $DB_HOST
      DB_USER: $DB_USER
      DB_PASSWORD: $DB_PASSWORD
      DB_NAME: $DB_NAME
      DB_DRIVER: $DB_DRIVER
      IS_DOCKER: $IS_DOCKER
# Read instructions at https://wodby.com/stacks/drupal/docs/local/xdebug/
      PHP_XDEBUG: 1
      PHP_XDEBUG_DEFAULT_ENABLE: 1
      PHP_XDEBUG_REMOTE_CONNECT_BACK: 0
      PHP_IDE_CONFIG: serverName=PHPSTORM
      PHP_XDEBUG_REMOTE_HOST: host.docker.internal # Docker 18.03+ & Linux/Mac/Win
#      PHP_XDEBUG_REMOTE_HOST: 172.17.0.1 # Linux, Docker < 18.03
#      PHP_XDEBUG_REMOTE_HOST: 10.254.254.254 # macOS, Docker < 18.03
#      PHP_XDEBUG_REMOTE_HOST: 10.0.75.1 # Windows, Docker < 18.03
    volumes:
      - ./:/var/www/html
## For macOS users (https://wodby.com/stacks/drupal/docs/local/docker-for-mac/)
#      - ./:/var/www/html:cached # User-guided caching
#      - docker-sync:/var/www/html # Docker-sync
## For Xdebug profiler files
#      - files:/mnt/files

  selenium-hub:
    image: selenium/hub
    ports:
      - "4444:4444"

  chrome:
    image: selenium/node-chrome-debug
    ports:
    - "5900:5900"
    links:
      - selenium-hub:hub
    volumes:
      - /dev/shm:/dev/shm # Mitigates the Chromium issue described at https://code.google.com/p/chromium/issues/detail?id=519952
    environment:
        HUB_PORT_4444_TCP_ADDR: selenium-hub
        HUB_PORT_4444_TCP_PORT: 4444

#  chrome:
#    image: yukinying/chrome-headless:63.0.3230.2
#    container_name: "${PROJECT_NAME}_chrome_headless"
#    depends_on:
#      - apache
#    ports:
#      - 0.0.0.0:9222:9222

#  nginx:
#    image: wodby/drupal-nginx:$NGINX_TAG
#    container_name: "${PROJECT_NAME}_nginx"
#    depends_on:
#      - php
#    environment:
#      NGINX_STATIC_CONTENT_OPEN_FILE_CACHE: "off"
#      NGINX_ERROR_LOG_LEVEL: debug
#      NGINX_BACKEND_HOST: php
#      NGINX_SERVER_ROOT: /var/www/html/web
##      NGINX_DRUPAL_FILE_PROXY_URL: http://example.com
#    volumes:
#      - ../.:/var/www/html
## For macOS users (https://wodby.com/stacks/drupal/docs/local/docker-for-mac/)
##      - ./:/var/www/html:cached # User-guided caching
##      - docker-sync:/var/www/html # Docker-sync
#    labels:
#      - 'traefik.backend=nginx'
#      - 'traefik.port=80'
#      - 'traefik.frontend.rule=Host:${PROJECT_BASE_URL}'

  mailhog:
    image: mailhog/mailhog
    container_name: "${PROJECT_NAME}_mailhog"
    labels:
      - 'traefik.backend=mailhog'
      - 'traefik.port=8025'
      - 'traefik.frontend.rule=Host:mailhog.${PROJECT_BASE_URL}'

#  postgres:
#    image: wodby/postgres:$POSTGRES_TAG
#    container_name: "${PROJECT_NAME}_postgres"
#    stop_grace_period: 30s
#    environment:
#      POSTGRES_PASSWORD: $DB_PASSWORD
#      POSTGRES_DB: $DB_NAME
#      POSTGRES_USER: $DB_USER
#    volumes:
#      - ./postgres-init:/docker-entrypoint-initdb.d # Place init file(s) here.
#      - /path/to/postgres/data/on/host:/var/lib/postgresql/data # I want to manage volumes manually.

  apache:
    image: wodby/php-apache:$APACHE_TAG
    container_name: "${PROJECT_NAME}_apache"
    depends_on:
      - php
    environment:
      APACHE_LOG_LEVEL: debug
      APACHE_BACKEND_HOST: php
      APACHE_SERVER_ROOT: /var/www/html/web
      IS_DOCKER: $IS_DOCKER
    volumes:
      - ./:/var/www/html
# For macOS users (https://wodby.com/stacks/drupal/docs/local/docker-for-mac/)
#      - ./:/var/www/html:cached # User-guided caching
#      - docker-sync:/var/www/html # Docker-sync
    labels:
      - 'traefik.backend=apache'
      - 'traefik.port=80'
      - 'traefik.frontend.rule=Host:${PROJECT_BASE_URL}'

  nvm:
    image: livingdocs/nvm
    container_name: "${PROJECT_NAME}_nvm"

#  varnish:
#    image: wodby/drupal-varnish:$VARNISH_TAG
#    container_name: "${PROJECT_NAME}_varnish"
#    depends_on:
#      - nginx
#    environment:
#      VARNISH_SECRET: secret
#      VARNISH_BACKEND_HOST: nginx
#      VARNISH_BACKEND_PORT: 80
#    labels:
#      - 'traefik.backend=varnish'
#      - 'traefik.port=6081'
#      - 'traefik.frontend.rule=Host:varnish.${PROJECT_BASE_URL}'

#  redis:
#    container_name: "${PROJECT_NAME}_redis"
#    image: wodby/redis:$REDIS_TAG

#  adminer:
#    container_name: "${PROJECT_NAME}_adminer"
#    image: wodby/adminer:$ADMINER_TAG
#    environment:
#      ADMINER_SALT: adminer-salt
#    labels:
#      - 'traefik.backend=adminer'
#      - 'traefik.port=9000'
#      - 'traefik.frontend.rule=Host:adminer.${PROJECT_BASE_URL}'

#  pma:
#    image: phpmyadmin/phpmyadmin
#    container_name: "${PROJECT_NAME}_pma"
#    environment:
#      PMA_HOST: $DB_HOST
#      PMA_USER: $DB_USER
#      PMA_PASSWORD: $DB_PASSWORD
#      PHP_UPLOAD_MAX_FILESIZE: 1G
#      PHP_MAX_INPUT_VARS: 1G
#    labels:
#      - 'traefik.backend=pma'
#      - 'traefik.port=80'
#      - 'traefik.frontend.rule=Host:pma.${PROJECT_BASE_URL}'

#  solr:
#    image: wodby/drupal-solr:$SOLR_TAG
#    container_name: "${PROJECT_NAME}_solr"
#    environment:
#      SOLR_HEAP: 1024m
#    labels:
#      - 'traefik.backend=solr'
#      - 'traefik.port=8983'
#      - 'traefik.frontend.rule=Host:solr.${PROJECT_BASE_URL}'

#  nodejs:
#    image: wodby/drupal-node:$DRUPAL_NODE_TAG
#    container_name: "${PROJECT_NAME}_drupal_nodejs"
#    environment:
#       NODE_SERVICE_KEY: node-service-key
#    labels:
#      - 'traefik.backend=nodejs'
#      - 'traefik.port=8080'
#      - 'traefik.frontend.rule=Host:nodejs.${PROJECT_BASE_URL}'
#    volumes:
#      - ./path/to/your/single-page-app:/app
#    command: sh -c 'npm install && npm run start'

#  memcached:
#    container_name: "${PROJECT_NAME}_memcached"
#    image: wodby/memcached:$MEMCACHED_TAG

#  rsyslog:
#    container_name: "${PROJECT_NAME}_rsyslog"
#    image: wodby/rsyslog:$RSYSLOG_TAG

#  athenapdf:
#    image: arachnysdocker/athenapdf-service:$ATHENAPDF_TAG
#    container_name: "${PROJECT_NAME}_athenapdf"
#    environment:
#      WEAVER_AUTH_KEY: weaver-auth-key
#      WEAVER_ATHENA_CMD: "athenapdf -S"
#      WEAVER_MAX_WORKERS: 10
#      WEAVER_MAX_CONVERSION_QUEUE: 50
#      WEAVER_WORKER_TIMEOUT: 90
#      WEAVER_CONVERSION_FALLBACK: "false"

#  node:
#    image: wodby/node:$NODE_TAG
#    container_name: "${PROJECT_NAME}_node"
#    working_dir: /app
#    labels:
#      - 'traefik.backend=node'
#      - 'traefik.port=3000'
#      - 'traefik.frontend.rule=Host:front.${PROJECT_BASE_URL}'
#    expose:
#      - "3000"
#    volumes:
#      - ./path/to/your/single-page-app:/app
#    command: sh -c 'npm install && npm run start'

#  blackfire:
#    image: blackfire/blackfire
#    container_name: "${PROJECT_NAME}_blackfire"
#    environment:
#      BLACKFIRE_SERVER_ID: XXXXX
#      BLACKFIRE_SERVER_TOKEN: YYYYY

#  webgrind:
#    image: wodby/webgrind:$WEBGRIND_TAG
#    container_name: "${PROJECT_NAME}_webgrind"
#    environment:
#      WEBGRIND_PROFILER_DIR: /mnt/files/xdebug/profiler
#    labels:
#      - 'traefik.backend=webgrind'
#      - 'traefik.port=8080'
#      - 'traefik.frontend.rule=Host:webgrind.${PROJECT_BASE_URL}'
#    volumes:
#      - files:/mnt/files

#  elasticsearch:
#    image: wodby/elasticsearch:$ELASTICSEARCH_TAG
#    environment:
#      ES_JAVA_OPTS: "-Xms500m -Xmx500m"
#    ulimits:
#      memlock:
#        soft: -1
#        hard: -1

#  kibana:
#    image: wodby/kibana:$KIBANA_TAG
#    depends_on:
#      - elasticsearch
#    labels:
#      - 'traefik.backend=kibana'
#      - 'traefik.port=5601'
#      - 'traefik.frontend.rule=Host:kibana.php.docker.localhost'

  portainer:
    image: portainer/portainer
    container_name: "${PROJECT_NAME}_portainer"
    command: --no-auth -H unix:///var/run/docker.sock
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    labels:
      - 'traefik.backend=portainer'
      - 'traefik.port=9000'
      - 'traefik.frontend.rule=Host:portainer.${PROJECT_BASE_URL}'

  traefik:
    image: traefik
    container_name: "${PROJECT_NAME}_traefik"
    command: -c /dev/null --web --docker --logLevel=INFO
    ports:
      - '8000:80'
#     - '8080:8080' # Dashboard
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

#volumes:
## Docker-sync for macOS users
#  docker-sync:
#    external: true
## For Xdebug profiler
#  files:

halfer · June 12, 2018, 2:49pm

That’s probably your answer right there - your web app is not working in the test env. Debug that, using the CircleCI SSH mode.

acha5066 · June 12, 2018, 5:21pm

Yep. You’re right. In my case (Drupal 8) the CSS and JS aggregation setting needed to be disabled to get it working. Although, I’m not sure that is a sustainable solution long term… Anway, my tests are working again. Thanks @halfer.

halfer · June 12, 2018, 8:56pm

Great!

All apps should have environment-specific settings so that you can turn things on/off in your tests. For example, you don’t want to fetch CSS/JS assets from a CDN here - it is much better to fetch them locally. You want to connect to a local database, not a hosted one, etc.

I don’t know what flexibility Drupal offers in this regard, but it is important to achieve this in a clean and seamless way, so that testing can be done without awkward or risky patching.

system · September 10, 2018, 8:56pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Testing locally with MAMP works but cannot get expected results on CircleCI Build Environment	4	669	August 6, 2019
Can i run behat tests with docker-compose exec? Build Environment	1	1547	June 18, 2018
Need help with CircleCI 2.0 Build Environment	1	1234	June 18, 2018
Test fail randomly in CI but work fine locally Running Tests nodejs , docker , react	0	2423	June 2, 2021
Build failed but it's green (used to be red few days ago) Build Environment docker	1	1233	November 16, 2019

Behat test passes locally but fails in CI. Both using same docker environment

Related topics