Dockerfile inconsistent build vs local

We are using docker and ECS/ECR on AWS to deploy.

I have a shell script that CircleCI is meant to execute. If I run it locally it runs fine and the image deploys smoothly to Amazon. However if I do the same thing via CircleCI the docker container exits and has this error

/var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:191:in rescue in load_spec_files': git://github.com/railsconfig/config.git (at master) is not yet checked out. Runbundle installfirst. (Bundler::GitError) from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:188:inload_spec_files’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/path.rb:92:in local_specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:159:inspecs’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/lazy_specification.rb:53:in __materialize__' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:88:inblock in materialize’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:85:in map!' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:85:inmaterialize’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:140:in specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:185:inspecs_for’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:174:in requested_specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/environment.rb:18:inrequested_specs’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:13:in setup' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler.rb:127:insetup’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/setup.rb:18:in <top (required)>' from /usr/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:inrequire’
from /usr/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require’

Below is our docker file and the shell script - the only thing removed are some sensitive variables. Docker should consistently build across any machine so I am at a loss as to why this is occurring.

Note - if I SSH into the box and manually run the deploy_stage.sh file it RUNS FINE. It is only erroring when CircleCI is executing on deploy script via circle.yml

Dockerfile

FROM phusion/passenger-ruby22:0.9.17
ENV HOME /root

RUN apt-get update -qq
#&& apt-get install -y memcached
RUN apt-get install -y libqtwebkit-dev qt4-qmake
#libxslt-dev libxml2-dev nano
ENV RAILS_ROOT /home/app/app-web
ENV RAILS_ENV staging
# Expose Nginx HTTP service
EXPOSE 80
EXPOSE 443
EXPOSE 3000
# Start Nginx / Passenger
RUN rm -f /etc/service/nginx/down

# Remove the default site
RUN rm /etc/nginx/sites-enabled/default
# Add the nginx site and config
COPY _docker-config/nginx.conf /etc/nginx/sites-enabled/test_app.conf
COPY _docker-config/rails-env.conf /etc/nginx/main.d/rails-env.conf


# Install bundle of gems
WORKDIR /tmp
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
#RUN bundle install --without test production staging
RUN bundle install
# Add the Rails app
ADD . /home/app/app-web
RUN chown -R app:app /home/app/app-web
WORKDIR /home/app/app-web
#RUN rake assets:precompile --trace
CMD ["/usr/local/bin/bundle", "exec", "passenger", "start"]
#RUN set -a && . /home/app/app-web/.env.staging && bundle exec rake assets:precompile --trace
#RUN bundle exec
#rake assets:precompile --trace

Shell Script

    #!/usr/bin/env bash

    set -e
    set -u
    set -o pipefail
    CIRCLE_SHA12=staging
    # more bash-friendly output for jq
    JQ="jq --raw-output --exit-status"
    #docker build -t app-web:staging .
    sudo pip install awscli==1.9.17
    curl -L -o ~/bin/jq https://github.com/stedolan/jq/releases/download/jq-1.5rc1/jq-linux-x86_64-static && chmod +x ~/bin/jq
    curl -L https://github.com/docker/compose/releases/download/1.5.0/docker-compose-`uname -s`-`uname -m` > ../bin/docker-compose && chmod +x ../bin/docker-compose
    aws ecr get-login --region us-east-1 > ./aws_login.sh
    chmod u+x aws_login.sh
    chmod g+x aws_login.sh
    ./aws_login.sh
    rm aws_login.sh
    docker build --rm=false -t app/app-web:$CIRCLE_SHA12 -f Dockerfile.staging . | cat # workaround progress weirdness
    #docker login -u $AWS_LOGIN_USER -p $AWS_LOGIN_PASS -e $AWS_LOGIN_URL
    docker tag -f app/app-web:$CIRCLE_SHA12 XXXXX.dkr.ecr.us-east-1.amazonaws.com/app-web:staging
    echo "Start Image Deploy"
    date #Lets time this
    docker push XXXXX.dkr.ecr.us-east-1.amazonaws.com/app-web:staging | cat # workaround progress weirdness
    echo "Image Deployed"
    date #Lets see the end timee

Relevant circle.yml snippet

deployment:
        branch: dev-aws2
        commands:
            - ./deploy_staging.sh

Are you selecting a Ruby version in your circle.yml file? If not, could you please do so? (There is an example here.) This will make sure that the Ruby environment is set up properly.

Also, is bundle install step run in your build? In which section of the build does it get executed? Thanks.

bundle install is run in the build and ruby 2 is selected in circle.yml

The fact that the script works fine via ssh indicates that timing is involved with the error somehow. I still don’t understand how timing is involved, but can you add sleep 60 right before running the script?

To close this out there was a red-herring in the debugging.

We were trying to add the app files without repulling from git. As a result we were adding the modfiied version of the files that CircleCI uses for tests (For those unaware CircleCI modifies the database.yml and a few other bits) the bundle install CircleCI ran also threw off our build.

To resolve once tests pass in our deployment phase we cd /tmp and re clone the repo from the appropriate branch and then run the docker build command there.

1 Like