Interacting with secondary images

Build Environment
1

Hi all, I’m wondering if it is supposed to be possible to interact with a secondary image (in a Docker executor) beyond what ports might have been exposed for it. I have a peculiar situation that requires me to be able to get files into a secondary image, and then run a command in that image before I can interact with the service it provides. In an ordinary container situation that would equate to something like:
docker run --name service_image somerepo:image
docker cp my/localdir/ service_image:/target_dir
docker exec -it service_image command_that_needs_target_dir

If I can’t figure out how to do that in my secondary image then I can’t perform a prereq step, and therefore can’t use the service it provides. And I should mention that I also can’t go the route of starting my own container with ‘docker run’ based on the desired image. I know I can copy the files and run the command that way, but as far as I understand it, I can’t then publish a port from it and connect to it from steps running in my primary image. I feel stuck between a rock and a hard place.

So that’s the TLDR version. The peculiar situation is that I’m trying to use Solr Cloud from the solr:8 image, and then use the Solr admin API at the exposed port to create a new core based on a config set that is local to my checked out code. I’ve done this many times. But this time, annoyingly enough, my Solr config has a feature enabled which can’t be uploaded as “untrusted”, i.e. it has to be uploaded with authentication. So, I’m trying to get basic auth working for Solr cloud, which requires using the Zookeeper client to copy in an auth file, which requires the auth file to be local etc etc etc. Alternatively, I could run Solr in the image as standalone and use the CLI to create a core from a configset. But its a different permutation of the same problem. I would have to be able to get into the image and run a command, which then requires the config to be in a local directory that I would have to copy there. So I’m stuck in this dumb roadblock just because I can’t get my custom Solr config/schema to a Solr instance provided by a common image before running tests. Seems like a simple requirement.

2

Hi @randalldfloyd! Just for clarification, are you using multiple images as listed in this documentation?

Any small config example you can provide can help!

3

I actually meant to add a snippet. Yes its super simple using the Docker executor, a primary image and a secondary dependency image:

    jobs:
      build:
        docker:
        - image: circleci/ruby:2.6.6-stretch-node-browsers
        - image: solr:8
          command: bin/solr -cloud -noprompt -f -p 8985

And so again the issue is that I need to get a file into my Solr server and run a command in there before invoking an API endpoint at port 8985. So the question is, is it possible to use docker commands to interact with the secondary container started via image: solr:8, or am I overlooking something simpler that gives my steps access to the contents of that container?

4

There is not currently a way to run docker commands on the secondary containers. The main way you can communicate with them is via the network on their exposed ports. There is a blog article that talks about some workarounds and explains the situation fairly well. But basically you need to do something tricky like run a resource inside the Solr container that listens on a port and can take in that file.

With that that said, have you considered using a machine executor instead? The machine executors come with Docker and Docker Compose preinstalled. You could drop a docker-compose.yml file in your .circleci folder right next to you config.yml

With the machine executors you are basically just in a straightforward VM and can do things like mount files on the local file system from your repository directly into that Solr container in your Docker Compose config.

While the Docker instances have a some advantages, there tends to not be a massive difference in startup time and for projects that don’t fit neatly into a Docker instance, they can be a solid solution.

1 Like
5

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.