My CircleCI builds are failing intermittently with the following error:
If I rebuild without cache for the individual builds, they succeed. However it fails pretty frequently when new builds are created and it is frustrating to have to manually re-build every time. I’ve also included my config below.
Failing command: bundle exec rake db:create
Exit code: 1
Output:
rake aborted!
LoadError: libmysqlclient.so.18: cannot open shared object file: No such file or directory - /home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/mysql2.so
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies.rb:292:in `require'
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies.rb:292:in `block in require'
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies.rb:258:in `load_dependency'
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies.rb:292:in `require'
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2.rb:31:in `<top (required)>'
/home/circleci/parting-pro/config/application.rb:7:in `<top (required)>'
/home/circleci/parting-pro/Rakefile:4:in `require_relative'
/home/circleci/parting-pro/Rakefile:4:in `<top (required)>'
/home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
(See full trace by running task with --trace)
Exited with code 1
# Ruby CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-ruby/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/ruby:2.4.4-node-browsers
environment:
RAILS_ENV=test
- image: library/mysql:5.7.19
environment:
MYSQL_ALLOW_EMPTY_PASSWORD=true
MYSQL_ROOT_HOST=%
- image: redis
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/removed
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "Gemfile.lock" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run:
name: install dependencies
command: |
bundle install --jobs=4 --retry=3 --path vendor/bundle
- save_cache:
paths:
- ./vendor/bundle
key: v1-dependencies-{{ checksum "Gemfile.lock" }}
- restore_cache:
keys:
- v1-yarn-dependencies-{{ checksum "yarn.lock" }}
- v1-yarn-dependencies-
- run:
name: install package.json dependencies
command: |
yarn install
- save_cache:
paths:
- node_modules
key: v1-yarn-dependencies-{{ checksum "yarn.lock" }}
# Database setup
- run: bundle exec rake db:create
- run: bundle exec rake db:schema:load
# run tests!
- run:
name: run tests
command: |
mkdir /tmp/test-results
TEST_FILES="$(circleci tests glob "spec/**/*_spec.rb")"
bundle exec rspec --format progress \
--out /tmp/test-results/rspec.xml \
--format progress \
$TEST_FILES
# collect reports
- store_test_results:
path: /tmp/test-results
- store_artifacts:
path: /tmp/test-results
destination: test-results
Prior to the Rake command, could you list the directory involved to see what is in it? You’d just need a step of:
ls -l /home/circleci/parting-pro/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/
If that fails too (because the directory does not exist) then back up one or more levels, and use -R. Hopefully that will help find out what state this folder is in prior to the Rake command needing it.
Still no solution, but here are some additional details I discovered. Will try the temporary uninstall/reinstall as a workaround.
On a fresh build + SSH into box, here is the PATH and ldd lookup of mysql2.so. Note that libmariadbclient.so.18 is part of the shared library lookup. This is an example of the initial build that succeeds.
$PATH is /usr/local/bundle/bin:/usr/local/bundle/gems/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Note that libmysqlclient.so.18 cannot be found, which I believe is the source of the failure. Unfortunately I have not been able to debug any further. Any help is appreciated.