Bash environment / command run failure


#1

Something recently changed in the way test environment containers are started and commands are run in them that is breaking our builds.

We have container image defined here: https://github.com/gliderlabs/infra.gl/tree/master/ci

It defines a file /root/.bashrc, which if I recall wasn’t enough to get it to be sourced, so I set the environment BASH_ENV to point to it in the Dockerfile. It had been working for weeks. Now we have a project using it with a run step like this:

- run:
  name: Push Image
  command: ecr-push gliderlabs/cmd $(infra-query -r ".modules[0].outputs.repos.value.cmd")

And it now fails with this output for the step:

#!/bin/bash -eo pipefail
ecr-push gliderlabs/cmd $(infra-query -r ".modules[0].outputs.repos.value.cmd")
/bin/bash: infra-query: command not found
/bin/bash: ecr-push: command not found
Exited with code 127

I can SSH in and infra-query and ecr-push are available. Though oddly, BASH_ENV is set to a file that does not exist. I’m not sure if it’s always done that. If bash is run again from this context, the bashrc is not sourced and infra-query and ecr-push are unavailable. So perhaps you changed how run steps are executed. Or perhaps you started overriding BASH_ENV. Either way, it broke a reasonable setup. Please advise.


#2

That’s possible. Can you link me to one of the builds where you’re seeing this?


#3

https://circleci.com/gh/gliderlabs/cmd/31


#4

Thank you! I will open a bug ticket.

In the meantime, can you run a new build with an environment section defining BASH_ENV?


#5

Unless rebuilding an old build uses the environment variables set at the time, it looks like that didn’t make a difference: https://circleci.com/gh/gliderlabs/cmd/33


#6

That config doesn’t have an environment section. For example:

version: 2
jobs:
  build:
    environment:
      BASH_ENV: "~/.bashrc"

It should overwrite whatever we’re doing to break it right now.


Injecting custom environment variables when running locally via CircleCI CLI
#7

Engineers got back to me- they pushed a fix! Can you try a new commit without that env section? Rebuilding might even work.

They extend their apologies for this incident. We are working on supporting BASH_ENV properly but we weren’t accounting for instances where you’re already supporting it.


#8

Ah, I thought you meant set environment in Settings. Anyway, unset it and it works now from rebuild. Thanks.


#9

Awesome! If you run into anything else, definitely don’t hesitate to reach out.


#10