java.io.IOException: Unable to open file for writing


#1

Hi,

I have been trying to get my CircleCI 2.0 setup working, but I’m bumping into a weird problem.

First let me explain my setup. I have an environment defined with docker compose. I have still chosen machine as an executor because I need to be able to connect to the containers from the host (to run the integration tests). So my setup is following:

  1. Checkout
  2. Start containers
  3. Run tests with maven wrapper

Here is the configuration:

version: 2
jobs:
  build:
    machine: true
    working_directory: ~/Project
    steps:
      - checkout
      - run:
          name: Start container
          command: |
            set -x
            docker-compose up -d
      - run:
          name: Run tests
          command: |
            mkdir -p ~/.m2/
            cp settings.xml ~/.m2/settings.xml
            cd Integrations/Tests
            ./mvnw verify -X

In the last run phase I’m bumping into following error:

Caused by: org.apache.maven.plugin.MojoExecutionException: Error copying artifact from /home/circleci/.m2/repository/fi/http-api-mock/1.1.0/http-api-mock-1.1.0.war to /home/circleci/Project/Integrations/Tests/target/http-api-mock.war
	at org.apache.maven.plugin.dependency.AbstractDependencyMojo.copyFile(AbstractDependencyMojo.java:197)
	at org.apache.maven.plugin.dependency.fromConfiguration.CopyMojo.copyArtifact(CopyMojo.java:94)
	at org.apache.maven.plugin.dependency.fromConfiguration.CopyMojo.execute(CopyMojo.java:72)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	... 26 more
Caused by: java.io.IOException: Unable to open file /home/circleci/Project/Integrations/Tests/target/http-api-mock.war for writing.
	at org.codehaus.plexus.util.FileUtils.copyStreamToFile(FileUtils.java:1134)
	at org.codehaus.plexus.util.FileUtils.copyFile(FileUtils.java:1048)
	at org.apache.maven.plugin.dependency.AbstractDependencyMojo.copyFile(AbstractDependencyMojo.java:192)
	... 30 more

So it seems maven doesn’t have rights to write the file but doesn’t seem right. Test -folder doesn’t have target -folder beforehand (I checked) so it is generated on the go. And circleci user should have all the rights and the maven wrapper is run with that user.

Am I missing something obvious?


#2

Ah the problem seems to be that the Test directory is shared with one of the docker containers through a volume and the docker container creates the target folder with root priviliges. So then in test running phase we cannot access the directory with circleci user…

How could I solve this? Ask docker compose to run containers with circleci user? How do I do that?


#3

Okay wow forget my previous comment. The docker containers are not creating the target folder with root permissions. It is the maven wrapper…I did this for debug purposes:

  -run:
      name: Run tests
      command: |
        set +e
        mkdir -p ~/.m2/
        cp settings.xml ~/.m2/settings.xml
        cd Integrations/Tests
        ls -la
        ./mvnw verify -X
        ls -la

And in the logs it showed following.

before ./mvnw verify

total 40
drwxrwxr-x 4 circleci circleci 4096 Apr 13 10:42 .
drwxrwxr-x 8 circleci circleci 4096 Apr 13 10:42 …
drwxrwxr-x 3 circleci circleci 4096 Apr 13 10:42 .mvn
-rwxrwxr-x 1 circleci circleci 7272 Apr 13 10:42 mvnw
-rw-rw-r-- 1 circleci circleci 5271 Apr 13 10:42 mvnw.cmd
-rw-rw-r-- 1 circleci circleci 4766 Apr 13 10:42 pom.xml
drwxrwxr-x 4 circleci circleci 4096 Apr 13 10:42 src

After:

total 44
drwxrwxr-x 5 circleci circleci 4096 Apr 13 10:55 .
drwxrwxr-x 8 circleci circleci 4096 Apr 13 10:42 …
drwxrwxr-x 3 circleci circleci 4096 Apr 13 10:42 .mvn
-rwxrwxr-x 1 circleci circleci 7272 Apr 13 10:42 mvnw
-rw-rw-r-- 1 circleci circleci 5271 Apr 13 10:42 mvnw.cmd
-rw-rw-r-- 1 circleci circleci 4766 Apr 13 10:42 pom.xml
drwxrwxr-x 4 circleci circleci 4096 Apr 13 10:42 src
drwxr-xr-x 5 root root 4096 Apr 13 10:55 target

So why the heck is the target created with root permissions…


#4

Could you share your docker-compose config?

A lot of default docker images use the root user. You can configure the user that is being used but its hard to troubleshoot without seeing what image you are using.


#5

My I have two containers in my setup and both are custom images. Here is my compose configuration:

version: '2'

networks:
  esb:

services:
  esb:
    container_name: esb
    build:
      context: .
      dockerfile: ./docker/esb/esb.dockerfile
    volumes:
      - ./:/data/Project
    networks:
      esb:
        aliases:
          - esb
    expose:
      - "9443"
      - "8280"
    ports:
      - "8280:8280"
      - "9443:9443"
  mock-server:
    container_name: http-api-mock
    build:
      context: .
      dockerfile: ./docker/http-api-mock/http-api-mock.dockerfile
    volumes:
      - ./:/data/Project
    networks:
      esb:
        aliases:
          - mock-server
    expose:
      - "8888"
    ports:
      - "8888:8888"

#6

Are you using the root user on those images?


#7

Yes the user used in the Dockerfiles is root. I still wonder how the target folder is created with root rights since is not created by docker. The maven wrapper command is run outside docker.


#8

Yeah my debugging confirms it. Docker root user is the problem. I will have to make changes to my Dockerfiles.


#9