Using Environment Variables inside Circle.yml

Any news on this?

You can already achieve this with these lines in your circle.yml:

    test:
      post:
        - >
          curl "https://some-service.com?token=${TOKEN}"

I assume this approach works everywhere… as I’m just executing a shell command.
We use this for triggering a jenkins deployment.

    deployment:
      staging:
        branch: develop
        commands:
          - >
            curl "https://${JENKINS_USER}:${JENKINS_PASSWORD}@.../job/.../buildWithParameters?token=${JENKINS_TOKEN}

Works like a charm :slight_smile:

Hi,

Has anyone got this working for webhooks?

I tried this:

checkout:
post:
- sed -i “s/KEY/$API_KEY/g” circle.yml

notify:
webhooks:
# A list of hook hashes, containing the URL field
- url: https://my.webook.com?key=KEY

I can see it updates the circle.yml file correctly - but my webhook still gets called with KEY rather than the value… :frowning:

Environment variables aren’t parsed for the webhooks section.

This greatly diminishes the usefulness of the webhooks, as there’s no way to verify the payload (since it’s impossible to add a secret token as a URL parameter). No secret token means that the web hook can’t trust any of the data received in the payload, as there’s no way to verify that the request actually came from CircleCI. This means the payload is essentially useless and any webhook is going to have to hit CircleCI’s build API to get the data.

Webhooks are still useful, but they must always ignore the payload other than the build number, especially in open-source projects where the circle.yml file is publicly accessible

1 Like

I also would like environment variables in webhooks but an alternative way to verify that the request came from CircleCI is to do a GET request and see if the build payload matches up (because webhook notify’s payload should be identical to the Build API)

an alternative way to verify that the request came from CircleCI is to do a GET request and see if the build payload matches up

That’s what I ended up doing - I only use the build number from the payload and ignore everything else. I do a GET to their API and then treat that data as the source of truth. It’s annoying to do that though, and it’d be unnecessary if CircleCI just included some sort of private token in the payload.

+1 on environment variable support for webhook URLs… this is useful for both API tokens and passing parameters to the webhook

1 Like

+1 for having environment variables. I did try to solve a problem with notifications (Formatted Slack message after successful build) through a webhook, just to realise this will not provide me the $BRANCH_NAME variable.

+1 on environment variable support for webhook URLs

1 Like

Any news on this? I’d like to be able to use environment variables in the path section of store_artifacts as well. E.g.:

- store_artifacts:
          path: $ARTIFACT_DIR
          destination: ./

With the release of CircleCI 2.0, can we expect the webhooks section to support one day the environment variables or will it never?

Without being able to use environment variables, I’m really don’t know how this feature can be useful.

Otherwise, I there a place where we can manually make a cURL POST at the really end of a build?

I don’t know.

First step is to make sure we have this as a proper feature request. Does this one make sense? If you, " :heart:'ing" the top post will help gauge interest in it.

I do agree that webhooks should have environment variable support. That’s just common sense to me.

I’m also looking for a way to get this support. I’d previously been using the webhook element but want to avoid inlining the API key.

I’ve tried using the notification tag with a command element:

notify:
  # The default webhook integration doesn't expand environment variables in the webhook url definition.
  # We workaround this by querying the build info via. the REST API then packaging this result as a notification
  # payload and pushing to OpsGenie.   See https://circleci.com/docs/1.0/configuration/#notify for documentation.
  command: |
    BUILD_INFO=$(curl https://circleci.com/api/v1.1/project/github/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME/$CIRCLE_BUILD_NUM?circle-token=$CIRCLE_LOCK_API_TOKEN)
    curl -d "{\"payload\": $BUILD_INFO}" -H "Content-Type: application/json" -X POST https://api.opsgenie.com/v1/json/circleci?apiKey=$OPS_GENIE_API_KEY

I’ve manually SSH’d into the Circle pod for the build and the commands work fine when run directly. However they don’t appear to be called when run as part of the regular build file. i.e. CI seems to ignore the command sub-element. Am I missing something or is this WAI?

The inability to parse an environment variable in the notify section is disappointing - my project(s) are all open source but my team’s private chat room is not; I really don’t want to post our chat room’s API token into the config.yml file. An env var would be the perfect way to obscure it!

I absolutely agree with everyone here, not having the option of using environment variables inside the config makes using CircleCI pretty difficult right now. What is even more frustrating is your lack of communication regarding this. There are several threads where people discuss this issue only to receive a one-line answer that doesn’t solve anything.

edit: typo

Alright, calm down. I mean this kindly, but passive-aggressive complaints are not going to enthuse CircleCI staff to tend immediately to your specific requirement. :smiley_cat: Spend time stroking the happy cat instead.

Indeed there are, but I think you meant “threads”.

On a wider note - and I don’t mean you specifically here - I have several times on this board seen rather bitter and unpleasant commentary directed towards Circle for not having feature X, or not having commented about it sufficiently to satisfy the commenter. I wonder whether some folks making the harshest complaints have not worked in a busy, commercial software environment. I suspect furthermore than some of the moaners are looking to use the free tier anyway, and they forget that engineering time is pretty expensive, and needs to be used carefully.

Let’s remember that employees are people too, and for those of us who are software engineers, let’s remember the last time that a hostile or poorly-worded customer complaint ruined our day.

Balancing every customer desire and getting stuff done is really hard, and inevitably some folks will be disappointed. The trick is trying to satisfy enough customers in order to pay staff and keep the lights on.

1 Like

I completely agree with you regarding the harsh commentary in places like this. It doesn’t help anyone and may ruin someone else’s day. However, I am not sure where in my short text you saw passive-aggressiveness or the fact that I expect you to cater to my specific needs. All I wrote was that I feel frustrated when I see posts that are almost two years old and the only valuable answer by CircleCI employees is “We don’t have this feature right now, we will at some point.”

2 Likes

I acknowledge that as a perfectly fair point, which perhaps underscores how unreliable communication is in transmitting intent and tone! For what it’s worth, I acknowledge that my interpretation could be at fault, and “proving it” is probably going to be a fool’s errand - we interpret things as we interpret them, and there is probably more unreliable emotional alchemy that goes into that process than science.

If I had to try to pinpoint the problem, I’d say that - to some degree at least - we are not our emotional condition. Thus, if I say to someone that “I am angry at you and that’s your fault”, then in the ways such an accusation is codified, it can be hard to work out that perhaps the best response is that I need to achieve better control of my anger.

Of course, some anger is justified, and sometimes complaints are too. However, one solution for a complainant is to work out what can be said to achieve the desired end goal. For example, in this case, something like “Would a detailed use-case doc help CircleCI determine if this would be a popular feature?” That would be more likely to elicit employee replies. Perhaps a private email would be better too, with stuff like “I know you guys have a lot of competing demands, but I think this feature would not be too hard to achieve, if it was done in way X”.

Finally (and apols for the broad off-topic!), I think the theme of how we communicate with each other is very interesting in the context of (future) commercial relationships. The economic system we work within is broadly coercive: if I pay money or offer to, I might reserve the right to take a strident/brusque/abusive tone with the folks I am paying. That’s a very strange power we have inadvertently given to money, and we often do that subconsciously. And even though I am of these opinions, I still have to stop myself from doing it, because it is quite normalised (within Western societies at least).