Dockerfile inconsistent build vs local

Build Environment
1

We are using docker and ECS/ECR on AWS to deploy.

I have a shell script that CircleCI is meant to execute. If I run it locally it runs fine and the image deploys smoothly to Amazon. However if I do the same thing via CircleCI the docker container exits and has this error

/var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:191:in rescue in load_spec_files': git://github.com/railsconfig/config.git (at master) is not yet checked out. Runbundle installfirst. (Bundler::GitError) from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:188:inload_spec_files’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/path.rb:92:in local_specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/source/git.rb:159:inspecs’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/lazy_specification.rb:53:in __materialize__' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:88:inblock in materialize’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:85:in map!' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:85:inmaterialize’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:140:in specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:185:inspecs_for’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/definition.rb:174:in requested_specs' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/environment.rb:18:inrequested_specs’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:13:in setup' from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler.rb:127:insetup’
from /var/lib/gems/2.2.0/gems/bundler-1.10.6/lib/bundler/setup.rb:18:in <top (required)>' from /usr/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:inrequire’
from /usr/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require’

Below is our docker file and the shell script - the only thing removed are some sensitive variables. Docker should consistently build across any machine so I am at a loss as to why this is occurring.

Note - if I SSH into the box and manually run the deploy_stage.sh file it RUNS FINE. It is only erroring when CircleCI is executing on deploy script via circle.yml

Dockerfile

FROM phusion/passenger-ruby22:0.9.17
ENV HOME /root

RUN apt-get update -qq
#&& apt-get install -y memcached
RUN apt-get install -y libqtwebkit-dev qt4-qmake
#libxslt-dev libxml2-dev nano
ENV RAILS_ROOT /home/app/app-web
ENV RAILS_ENV staging
# Expose Nginx HTTP service
EXPOSE 80
EXPOSE 443
EXPOSE 3000
# Start Nginx / Passenger
RUN rm -f /etc/service/nginx/down

# Remove the default site
RUN rm /etc/nginx/sites-enabled/default
# Add the nginx site and config
COPY _docker-config/nginx.conf /etc/nginx/sites-enabled/test_app.conf
COPY _docker-config/rails-env.conf /etc/nginx/main.d/rails-env.conf


# Install bundle of gems
WORKDIR /tmp
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
#RUN bundle install --without test production staging
RUN bundle install
# Add the Rails app
ADD . /home/app/app-web
RUN chown -R app:app /home/app/app-web
WORKDIR /home/app/app-web
#RUN rake assets:precompile --trace
CMD ["/usr/local/bin/bundle", "exec", "passenger", "start"]
#RUN set -a && . /home/app/app-web/.env.staging && bundle exec rake assets:precompile --trace
#RUN bundle exec
#rake assets:precompile --trace

Shell Script

    #!/usr/bin/env bash

    set -e
    set -u
    set -o pipefail
    CIRCLE_SHA12=staging
    # more bash-friendly output for jq
    JQ="jq --raw-output --exit-status"
    #docker build -t app-web:staging .
    sudo pip install awscli==1.9.17
    curl -L -o ~/bin/jq https://github.com/stedolan/jq/releases/download/jq-1.5rc1/jq-linux-x86_64-static && chmod +x ~/bin/jq
    curl -L https://github.com/docker/compose/releases/download/1.5.0/docker-compose-`uname -s`-`uname -m` > ../bin/docker-compose && chmod +x ../bin/docker-compose
    aws ecr get-login --region us-east-1 > ./aws_login.sh
    chmod u+x aws_login.sh
    chmod g+x aws_login.sh
    ./aws_login.sh
    rm aws_login.sh
    docker build --rm=false -t app/app-web:$CIRCLE_SHA12 -f Dockerfile.staging . | cat # workaround progress weirdness
    #docker login -u $AWS_LOGIN_USER -p $AWS_LOGIN_PASS -e $AWS_LOGIN_URL
    docker tag -f app/app-web:$CIRCLE_SHA12 XXXXX.dkr.ecr.us-east-1.amazonaws.com/app-web:staging
    echo "Start Image Deploy"
    date #Lets time this
    docker push XXXXX.dkr.ecr.us-east-1.amazonaws.com/app-web:staging | cat # workaround progress weirdness
    echo "Image Deployed"
    date #Lets see the end timee

Relevant circle.yml snippet

deployment:
        branch: dev-aws2
        commands:
            - ./deploy_staging.sh
2

Are you selecting a Ruby version in your circle.yml file? If not, could you please do so? (There is an example here.) This will make sure that the Ruby environment is set up properly.

3

Also, is bundle install step run in your build? In which section of the build does it get executed? Thanks.

4

bundle install is run in the build and ruby 2 is selected in circle.yml

5

The fact that the script works fine via ssh indicates that timing is involved with the error somehow. I still don’t understand how timing is involved, but can you add sleep 60 right before running the script?

6

To close this out there was a red-herring in the debugging.

We were trying to add the app files without repulling from git. As a result we were adding the modfiied version of the files that CircleCI uses for tests (For those unaware CircleCI modifies the database.yml and a few other bits) the bundle install CircleCI ran also threw off our build.

To resolve once tests pass in our deployment phase we cd /tmp and re clone the repo from the appropriate branch and then run the docker build command there.

1 Like