All project SSH Keys being attempted

I have a few ssh keys in my Project Settings. All have a hostname defined that is the IP address of the servers that each key is associated with. One of my job steps is defined like:

      - run:
          name: Pull image
          command: ssh -oStrictHostKeyChecking=no -v ${USER}@${IP_ADDRESS}

The ssh command succeeds when it uses the correct key for the server, but it’s iterating through all of the keys, not just the one associated with the IP address of the server its logging into.

How does CircleCI use the hostname associated with each SSH key? Is it not able to find the correct key because I’m using an environment variable for the IP address, or is it because I’m using an IP address instead of a hostname?