"MySQL server has gone away", but works when I ssh into the machine

Build Environment
, ,
1

Hello everyone,

I’m having some issues getting my unit tests to connect to the MySQL database. I’ve included my current config.yml below. I am running docker-compose, then installing dockerize so I can wait for port 3306 to be open (which should mean that my database is ready to establish connections), but then when the final step runs, it fails with the message “MySQL server has gone away”. I’ve tried increasing the wait timeout and the maximum allowed packet size from the my.cnf file, but no luck so far.

I think the issue must be that the MySQL server isn’t ready when the tests run, because when I rerun the job with ssh, and connect to the machine, I am able to run my unit tests without this error. I thought this was supposed to be the problem that dockerize solved though?

Any help is appreciated, and please let me know if something requires clarification, thank you.

version: 2
jobs:
  build:
    machine: true
    steps:
      - checkout

      - run:
          name: Initialize containers with docker-compose.yml
          command: |
            chmod -R g+w .
            docker-compose up -d
            
      - run:
          name: install dockerize
          command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          environment:
            DOCKERIZE_VERSION: v0.6.1

      - run:
          name: wait for db
          command: dockerize -wait tcp://localhost:3306 -timeout 1m

      - run:
          name: run unit tests
          command: |
            docker exec project_cashout-php_1 /bin/bash -c "vendor/bin/phpunit"
2

I am bumping this thread because I have yet to receive a response

3

Is it possible you could share the logs of the first docker-compose up?

I see you are running this with -d to daemonize it, but you can actually run that in the foreground and make use of the background attribute for the run step. This will send the stdout for the docker-compose containers out to the CircleCI UI and might help us get some adding logging from the MySQL container.

      - run:
          name: Initialize containers with docker-compose.yml
          background: true
          command: |
            chmod -R g+w .
            docker-compose up

Note that we removed the -d as well as setting background: true.

4

I made the changes you suggested, the following logs were printed from the mysql container:

project-mysql_1  | 2021-05-19 14:02:10+00:00 [Note] [Entrypoint]: Database files initialized
project-mysql_1  | 2021-05-19 14:02:10+00:00 [Note] [Entrypoint]: Starting temporary server
project-mysql_1  | 2021-05-19 14:02:10+00:00 [Note] [Entrypoint]: Waiting for server startup
project-mysql_1  | 2021-05-19T14:02:10.650973Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
project-mysql_1  | 2021-05-19T14:02:10.652760Z 0 [Note] mysqld (mysqld 5.7.34) starting as process 76 ...
project-mysql_1  | 2021-05-19T14:02:10.655814Z 0 [Note] InnoDB: PUNCH HOLE support available
project-mysql_1  | 2021-05-19T14:02:10.655837Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
project-mysql_1  | 2021-05-19T14:02:10.655841Z 0 [Note] InnoDB: Uses event mutexes
project-mysql_1  | 2021-05-19T14:02:10.655845Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
project-mysql_1  | 2021-05-19T14:02:10.655848Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
project-mysql_1  | 2021-05-19T14:02:10.655851Z 0 [Note] InnoDB: Using Linux native AIO
project-mysql_1  | 2021-05-19T14:02:10.656178Z 0 [Note] InnoDB: Number of pools: 1
project-mysql_1  | 2021-05-19T14:02:10.656307Z 0 [Note] InnoDB: Using CPU crc32 instructions
project-mysql_1  | 2021-05-19T14:02:10.658238Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
project-mysql_1  | 2021-05-19T14:02:10.668576Z 0 [Note] InnoDB: Completed initialization of buffer pool
project-mysql_1  | 2021-05-19T14:02:10.670761Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
project-mysql_1  | 2021-05-19T14:02:10.683800Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
project-mysql_1  | 2021-05-19T14:02:10.693559Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
project-mysql_1  | 2021-05-19T14:02:10.693623Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
project-mysql_1  | 2021-05-19T14:02:10.716921Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
project-mysql_1  | 2021-05-19T14:02:10.717730Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
project-mysql_1  | 2021-05-19T14:02:10.717746Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
project-mysql_1  | 2021-05-19T14:02:10.718731Z 0 [Note] InnoDB: Waiting for purge to start
project-mysql_1  | 2021-05-19T14:02:10.768964Z 0 [Note] InnoDB: 5.7.34 started; log sequence number 2747334
project-mysql_1  | 2021-05-19T14:02:10.769402Z 0 [Note] Plugin 'FEDERATED' is disabled.
project-mysql_1  | 2021-05-19T14:02:10.776136Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
project-mysql_1  | 2021-05-19T14:02:10.776159Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
project-mysql_1  | 2021-05-19T14:02:10.776278Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
project-mysql_1  | 2021-05-19T14:02:10.777983Z 0 [Note] InnoDB: Buffer pool(s) load completed at 210519 14:02:10
project-mysql_1  | 2021-05-19T14:02:10.778489Z 0 [Warning] CA certificate ca.pem is self signed.
project-mysql_1  | 2021-05-19T14:02:10.778524Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
project-mysql_1  | 2021-05-19T14:02:10.782129Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
project-mysql_1  | 2021-05-19T14:02:10.787747Z 0 [Note] Event Scheduler: Loaded 0 events
project-mysql_1  | 2021-05-19T14:02:10.787913Z 0 [Note] mysqld: ready for connections.
project-mysql_1  | Version: '5.7.34'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server (GPL)
project-mysql_1  | 2021-05-19 14:02:11+00:00 [Note] [Entrypoint]: Temporary server started.
project-mysql_1  | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
project-mysql_1  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.

I don’t see anything that might indicate what the problem is, but I’m still getting the same error from the tests: PDOException: PDO::__construct(): MySQL server has gone away

5

I agree with the background: true that @mike suggested. I think the other issue you’re facing is tcp://localhost:3306 MySQL is running on 127.0.0.1:3306 of container A while I think you’re trying to connect to 127.0.0.1:3306 of container B (project_cashout-php_1). This would explain the “When I connect via ssh it works”, You’re executing the mysql connection in the “local” context rather than within the project_cashout-php_1 container

That would also by why @mike suggested docker-compose up The docker-compose.yaml would include links eg

mysql:
    image: mysql:latest
    ports:
       - 3306:3306
project:
    image: project_cashout-php_1:latest
    links: 
       - mysql

The other thing to think about / consider is running the docker container as part of the build / test
eg

docker:
  - image:  your_existing_image:version
  - image: circleci/mysql:5.6-ram
    environment:
      MYSQL_ROOT_PASSWORD: rootpass
      MYSQL_DATABASE: testdb
      MYSQL_HOST: '0.0.0.0'
      MYSQL_ROOT_HOST: '%'
      MYSQL_USER: citest
      MYSQL_PASSWORD: cipass

I’d have used 127.0.0.1 in the above MYSQL_HOST however it would have the linking / context issue.

6

Including the links like you suggested worked! Thank you

7

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.