Echo/Evaluate external variable to slack orb

Build Environment
,
1

Hi all,

I have an issue with CircleCi config.yaml version 2.1 and passing/evaluating variables. I have a script that creates a summary of a file and stores it as variable in $BASH_ENV. This variable needs to be evaluated/echoed in CircleCi’s slack orb as json custom message but unfortunately this throws a parsing error. I tried with setting a simple “hello” message but jq still cannot parse it. If I echo the variable in a different step it shows the contents of this variable correctly.

In my main job I am using a custom private image that the commands runs in, without affecting $BASH_ENV or any environment variables.

Let me know if someone can help or if I am doing something wrong on that matter.

Best Regards and thank you

config.yml

version: 2.1

orbs:
  slack: circleci/slack@4.3.0

commands:
  custom_slack_message:
    steps:
      - run:
          command: |
              ./scripts/store_variable.sh
      - slack/notify:
          custom: |
            {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": $MESSAGE }}]}
          event: always
          channel: "a-channel"

scripts/store_variable.sh

#!/bin/bash
# GET THE PLAN FILE
FILE="./.terraform/plan/tfplan"

# GET THE DIFF
plan_diff=$(terraform show $FILE | grep -E "^\s*[#~+-]")

echo "export MESSAGE=\"Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`\"" >> $BASH_ENV; source $BASH_ENV;
2

Hi @christos ,

Just to rule out possible causes, could you please:

  • Add double-quotes around $MESSAGE in the custom parameter:
{"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "$MESSAGE" }}]}

 

  • Replace a pair of double-quotes with single-quotes in the command that sets the MESSAGE variable:
echo 'export MESSAGE=\"Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`\" ' >> $BASH_ENV; source $BASH_ENV;
3

@christos,

Just to suggest an alternative syntax (double-quote+single-quote+double-quote):

{"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "'"$MESSAGE"'" }}]}

Let me know if this helps.

4

Hi @yannCI,

First of all, I want to thank you for your time to propose suggestions and possible solutions.

I tried all of them but unfortunately nothing worked.

  • The first suggestion was the only that actually send a slack message but unfortunately the message was empty when the variable inside the script wasn’t.

  • The second suggestion failed inside the script and then failed also inside the slack/notify with the same error. Because of that I didn’t try any combination with the 1st or 3rd suggestion. The error I received was:

    /tmp/.bash_env-609d3d727362977939a6d087-0-build: line 1: export: `terraform:’: not a valid identifier
    /tmp/.bash_env-609d3d727362977939a6d087-0-build: line 1: export: ```````"’: not a valid identifier

  • Your third suggestion failed during the slack/notify with the error:

    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 584: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 585: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 586: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 587: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 588: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 589: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 590: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 591: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 592: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 593: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 594: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 595: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 596: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 597: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 598: -: command not found
    /tmp/.bash_env-609d43ce7362977939a6e1cf-0-build: line 599: -: command not found

Do you have any other suggestions?

Thank you

5

Hi @christos,

Thanks for trying the suggestions out and sharing the outcome.

The errors you’re getting confirm that the issue stems from the position of the quotes and the way they are escaped.

I was able to set the MESSAGE variable with the following syntax:

echo ' export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" ' >> $BASH_ENV; source $BASH_ENV;

To confirm all variables are properly set, could you please modify the first run step as follows:

      - run:
          command: |
              ./scripts/store_variable.sh
              echo $FILE
              echo $plan_diff
              echo $MESSAGE

 
And then use either of the below syntaxes for the custom parameter:

  • “double-quote+single-quote+double-quote”
    {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "'"$MESSAGE"'" }}]}
  • “double-quotes”
    {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "$MESSAGE" }}]}
6

Hi @yannCI,

Thank you for the further explanations and suggestions. Could you please elaborate more how you made your last suggestion to work with the echo of the variables. For me outside of the script it doesn’t echoing the variables. I tried with two options:

  • First I echoed inside the script file store_variable.sh:
echo ' export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" ' >> $BASH_ENV; source $BASH_ENV;
echo $FILE
echo $plan_diff
echo $MESSAGE

This echoed all the variables correctly

  • Secondly I tried your suggestion to echo the variables in the command:
  - run:
       command: |
          ./scripts/store_variable.sh
          echo $FILE
          echo $plan_diff
          echo $MESSAGE

This didn’t echo anything like the variables weren’t set at all. For the $FILE and $plan_diff variables logically this is correct as their scope is inside the script and not outside of them. However the $MESSAGE as it got exported inside the $BASH_ENV should have values.

After these two extra tries I am thinking if I am doing something of how I am exporting the variable inside the $BASH_ENV but I would like to know how you made it run on your end.

Thank you

7

Hi @christos,

Since every run step is a new shell, the command:

export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`"

will be executed in a shell that doesn’t know of the plan_diff variable, hence the incomplete value you’re seeing for the MESSAGE variable (Changes to be applied from terraform: ``````)

And if you replace:
plan_diff=$(terraform show $FILE | grep -E "^\s*[#~+-]")
with
echo 'export plan_diff=$(terraform show $FILE | grep -E "^\s*[#~+-]")' >> $BASH_ENV

 
then you’ll also need to replace:
FILE="./.terraform/plan/tfplan"
with
echo ' export FILE="./.terraform/plan/tfplan" ' >> $BASH_ENV
for the same reason as above.

 
To make it work I exported all variables to BASH_ENV:

#!/bin/bash
# GET THE PLAN FILE
echo ' export FILE="./.terraform/plan/tfplan" ' >> $BASH_ENV

# GET THE DIFF
echo 'export plan_diff=$(echo "totototo is tititi")' >> $BASH_ENV

echo 'export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`"' >> $BASH_ENV

(Note: You only need to add source $BASH_ENV in the above step if you want to echo the variables within this step. It is not needed for the variables to be interpolated in subsequent steps because in every step, CircleCI uses bash to source BASH_ENV)

Let me know if this works for you.

8

Hi @yannCI,

Thank you for your suggestions and the possible solutions. Your suggestions, unfortunately, didn’t work for me. I should have made it more clear from my previous response that when I am echoing the variables inside the command they were completely empty, like undefined variables, and not like this:

Changes to be applied from terraform: ``````

However, in your last note regarding source $BASH_ENV it made me think if every step uses bash to source $BASH_ENV. So I tried to add it after the execution of the script and I managed to get the variables inside the config.yaml and echo the variables after I added source $BASH_ENV in the run step example code:

steps:
  - run:
      command: |
          ./scripts/store_variable.sh
          source $BASH_ENV
          echo $MESSAGE

Only with this I could manage to echo the $MESSAGE and have the correct value inside command’s run step. I am not quite sure why you could make it run and I couldn’t or what are the incosistencies between our testing environments but I found it really weird.

Unfortunately though, when now I have my concrete variable jq in slack/notify method is complaining with the error:

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 593, column 43

By using a fake string works and posts the message to my slack channel correctly eg using the below for plan_diff:
echo 'export plan_diff=$(echo "totototo is tititi")' >> $BASH_ENV

I tried to escape the string with jq -Rs . and it escapes the whole string but slack/notify still fails it.

echo ' export MESSAGE=$(echo "Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" | jq -Rs .) ' >> $BASH_ENV

Any ideas would be appreciated on the matter.

Regards,
Christos

9

Hi @christos,

Thanks for getting back to me!

The reason the MESSAGE variable appeared empty (rather than showing “Changes to be applied from terraform: ``````”)

in the step:

  - run:
       command: |
          ./scripts/store_variable.sh
          echo $FILE
          echo $plan_diff
          echo $MESSAGE

is that the script:

echo ' export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" ' >> $BASH_ENV; source $BASH_ENV;
echo $FILE
echo $plan_diff
echo $MESSAGE

sets MESSAGE for the current shell by using BASH_ENV, but because CircleCI sources BASH_ENV at the start of every step, the value won’t be interpolated within the same step unless you manually source BASH_ENV` (as you did in your latest configuration).

 
Regarding the jq parsing error, have you tried the “double-quote+single-quote+double-quote” notation:

{"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "'"$MESSAGE"'" }}]}

 
If not, could you please share the latest custom parameter you specified in the slack/notify step?

10

Hi @yannCI,

I have tried all your suggestions regarding the custom parameter in slack/notify.
I tried with double quotes or double-single-double quotes and all are failing. Please find below what I have tried:
1st:

  - slack/notify:
      custom: |
        {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "$MESSAGE" }}]}
      event: always

2nd:

  - slack/notify:
      custom: |
        {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "'"$MESSAGE"'" }}]}
      event: always

3rd:

  - slack/notify:
      custom: |
        {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": $MESSAGE }}]}
      event: always

Let me know if you need anything further.

11

Hi @christos,

In order to further troubleshoot, and try to find out what the issue stems from, could you please execute all commands within the config.yml rather than using the external script store_variable.sh?

Your test config.yml would look like this:

version: 2.1

orbs:
  slack: circleci/slack@4.3.0

commands:
  custom_slack_message:
    steps:
      - run:
          command: |
              echo ' export FILE="./.terraform/plan/tfplan"  ' >> $BASH_ENV
              echo ' export plan_diff=$(echo "totototo is tititi") ' >> $BASH_ENV
              echo ' export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" ' >> $BASH_ENV

      - slack/notify:
          custom: |
            {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": $MESSAGE }}]}
          event: always
          channel: "a-channel"

Let me know what the outcome is.

12

Hi @yannCI ,

I’ve applied your suggestions and it is failing. Please find below an image from the pipeline:

image
image904×1180 124 KB

13

Hi @christos,

Thanks for getting back to me.

My bad! The missing double-quotes around $MESSAGE in the JSON array are causing this error.

Also, I see that I keep pasting my sample test string totototo is tititi :laughing: , which you no longer need to use.

So my previous suggestion should’ve read:

version: 2.1

orbs:
  slack: circleci/slack@4.3.0

commands:
  custom_slack_message:
    steps:
      - run:
          command: |
              echo ' export FILE="./.terraform/plan/tfplan"  ' >> $BASH_ENV
              echo ' export plan_diff=$(terraform show $FILE | grep -E "^\s*[#~+-]") ' >> $BASH_ENV
              echo ' export MESSAGE="Changes to be applied from terraform: \`\`\`$plan_diff\`\`\`" ' >> $BASH_ENV

      - slack/notify:
          custom: |
            {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "$MESSAGE" }}]}
          event: always
          channel: "a-channel"
14

Hi @yannCI ,

I’ve updated with the new test and below you will find the results. In short the results are exactly the same except of the error throwing from jq which is different.

Please check the image below:

image
image904×1220 139 KB

The full error from jq in Slack-Sending Notification is:

Posting Status
Checking For JQ + CURL: Debian
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 685, column 61

Exited with code exit status 4

Also, you may notice that the plan file is not actually in .terraform/plan/tfplan but under the path .terraform/plan/dev/tfplan I made the appropriate changes in the rest of the script so you can ignore it.

Thank you.

15

:wave: Hi @christos!

Sorry about the delayed reply.

Could you please share the content of the plan_diff variable? I suspect some additional character escaping is needed.

16

Hi @yannCI !

Apologies for the late reply the past weeks were really stressful.

Please find below the content of the plan_diff variable below:

  ~ update in-place
  # module.a-module.kubectl_manifest.this[0] will be updated in-place
  ~ resource "kubectl_manifest" "this" {
        id                      = "/apis/apps/v1/namespaces/my-namespace/deployments/a"
      ~ yaml_body               = (sensitive value)
      ~ yaml_body_parsed        = <<-EOT
              namespace: my-namespace
                  - command:
                    - /bin/sh
                    - -c
                    - node index.js
                    - name: NAMESPACE
                    - name: NODE_NAME
                    - name: POD_IP
                    - name: TEST_DB_ENDPOINT
                      value: test-db-endpoint
                    - name: ENV_VARIABLE_1
                      value: http://test.my-namespace/test/test/
                    - name: ENV_VARIABLE_2
                      value: test-env-2
                    - name: ENV_VARIABLE_3
                      value: test-env.test-env-3
                    - name: ENV_VARIABLE_4
                    - name: ENV_VARIABLE_5
                    - name: ENV_VARIABLE_6
                    - name: ENV_VARIABLE_7
                      value: test-env.test-env-3
          -         image: image.repo.com/a:test-release-1
          +         image: image.repo.com/a:test-release-2
                    - containerPort: 80
Plan: 0 to add, 1 to change, 0 to destroy.

The above is currently failing in the step Slack - Sending Notification with the error:

Posting Status
Checking For JQ + CURL: Debian
parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 9, column 14

Let me know if you get the same error if you can identify what is needed to be escaped more.

Thank you

17

I’m having the same issue as well. Did anybody figure out how to fix it?

18

I am currently having the same issue. Been on this for days now. This shouldn’t be too difficult at all

19

Hi @christos,

Sorry about such a delayed update; this was a rather baffling issue, and I couldn’t get my head around it… until now :slight_smile:

I was able to reproduce the issue using the content of the plan_diff variable you shared.
As suspected the issue is caused by the multiline content, as well as, “problematic” characters which interfere with the request being sent to Slack’s API (in this case, double-quotes).

If you paste that content in the Slack block kit builder with the same template you’re using in your slack/notify step, you’ll see that it generates errors; these errors disappear once the newline characters and double-quotes are removed or escaped.

So the idea was to escape those characters programmatically. It took quite a bit of wrestling with sed and awk, but I was finally able to send that exact content via the slack/notify command and to have it displayed in the Slack message with its original layout.

Here’s how I did it (adapted to your project):

version: 2.1

orbs:
  slack: circleci/slack@4.3.0

commands:
  custom_slack_message:
    steps:
      - run:
          name: Populate MESSAGE variable
          command: |
              FILE="./.terraform/plan/tfplan"
              terraform show $FILE | grep -E "^\s*[#~+-]" | awk '{printf "%s\\n", $0}'|sed 's/\"/\\"/g' > message.txt
              echo 'export MESSAGE="Changes to be applied from terraform: \`\`\`$(cat message.txt)\`\`\`" ' >> $BASH_ENV

      - slack/notify:
          custom: |
            {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": "$MESSAGE" }}]}

 
Or with an additional script:

version: 2.1

orbs:
  slack: circleci/slack@4.3.0

commands:
  custom_slack_message:
    steps:
      - run:
          command: |
              chmod +x ./scripts/store_variable.sh
              ./scripts/store_variable.sh
      - slack/notify:
          custom: |
            {"blocks": [{"type": "section", "text": {"type": "mrkdwn", "text": $MESSAGE }}]}
          event: always
          channel: "a-channel"

where store_variable.sh contains:

#!/bin/bash
# GET THE PLAN FILE
FILE="./.terraform/plan/tfplan"

# GET THE DIFF AND POPULATE MESSAGE
terraform show $FILE | grep -E "^\s*[#~+-]" | awk '{printf "%s\\n", $0}'|sed 's/\"/\\"/g' > message.txt
echo 'export MESSAGE="Changes to be applied from terraform: \`\`\`$(cat message.txt)\`\`\`" ' >> $BASH_ENV

 
Let me know if this works for you.

@Durga, @barywhyte,

The above solution is quite tailored for @christos’ case; you will likely need to adapt it so it fits your respective cases.
The “trick” is to escape the problematic characters. Testing the message in the Slack block kit builder will help you identify which characters are causing the problem.

If you’re having trouble implementing the suggested solution, or if you’re still unable to send the message via the slack/notify command after implementing the solution, please create a brand new thread for your specific use-case.

20

@yannCI,

Thank you very much for these details. In a way it moved me forward beyond where I initially got stuck. Before now, I was using alpine docker instance to run this pipeline. Strangely, with this docker instance, and no matter what I do, including changing the default shell from ash to bash, I could not find a way to echo an environmental variable saved in $BASH_ENV in one run step to the next. Strangely, the moment I changed from alpine to circleci/node:14.17-browsers, and without making any further changes, things worked differently. I find this totally strange. I just wanted to point this out.

Now back to the main thing, the solution above hasn’t worked for and you made that pretty clear in your text. However, the situation @christos describe in almost the same as mine. I was able to save in $BASH_ENV

This is my CCI pipeline:

#!/bin/bash -eo pipefail
terraform plan -out /tmp/tf.plan


terraform show /tmp/tf.plan | grep -E "\#"  > message.txt

echo 'export MESSAGE="Changes to be applied from terraform: \`\`\`$(cat message.txt)\`\`\`" ' >> $BASH_ENV
source $BASH_ENV

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

google_container_cluster.gke-cluster: Refreshing state... [id=**********]
google_container_node_pool.preemptible_node_pool: Refreshing state... [id=*************/**********/default-pool]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  # google_container_node_pool.preemptible_node_pool will be updated in-place
  ~ resource "google_container_node_pool" "preemptible_node_pool" {
        cluster             = "**********"
        id                  = "*************/**********/default-pool"
        initial_node_count  = 2
        instance_group_urls = [
            "https://www.googleapis.com/compute/v1/projects/*********/zones/*************/instanceGroupManagers/gke-**********-default-pool-ce6d4d29-grp",
        ]
        location            = "*************"
        name                = "default-pool"
      ~ node_count          = 2 -> 3
        node_locations      = [
            "*************",
        ]
        project             = "*********"
        version             = "1.20.9-gke.1001"

        management {
            auto_repair  = true
            auto_upgrade = true
        }

        node_config {
            disk_size_gb      = 100
            disk_type         = "pd-standard"
            guest_accelerator = []
            image_type        = "COS_CONTAINERD"
            labels            = {}
            local_ssd_count   = 0
            machine_type      = "n1-standard-1"
            metadata          = {
                "disable-legacy-endpoints" = "true"
            }
            oauth_scopes      = [
                "https://www.googleapis.com/auth/devstorage.read_only",
                "https://www.googleapis.com/auth/logging.write",
                "https://www.googleapis.com/auth/monitoring",
            ]
            preemptible       = true
            service_account   = "default"
            tags              = []
            taint             = []

            shielded_instance_config {
                enable_integrity_monitoring = true
                enable_secure_boot          = false
            }
        }

        upgrade_settings {
            max_surge       = 1
            max_unavailable = 0
        }
    }

  # google_storage_bucket.static-site-********* will be created
  + resource "google_storage_bucket" "static-site-*********" {
      + bucket_policy_only          = (known after apply)
      + force_destroy               = false
      + id                          = (known after apply)
      + location                    = "US"
      + name                        = "static-site-*********"
      + project                     = (known after apply)
      + self_link                   = (known after apply)
      + storage_class               = "STANDARD"
      + uniform_bucket_level_access = true
      + url                         = (known after apply)

      + website {
          + main_page_suffix = "index.html"
          + not_found_page   = "404.html"
        }
    }

Plan: 1 to add, 1 to change, 0 to destroy.

------------------------------------------------------------------------

This plan was saved to: /tmp/tf.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "/tmp/tf.plan"

CircleCI received exit code 0

I was able to successfully save this result in $BASH_ENV and echoed it out in the next run step

printf "Echo Detailed TFPLAN\n"
echo ${MESSAGE}

Echo Detailed TFPLAN
Changes to be applied from terraform: ``` # google_container_node_pool.preemptible_node_pool will be updated in-place  # google_storage_bucket.static-site-********* will be created```
CircleCI received exit code 0

However, inside the slack/notify step, this failed again as usual.

Screenshot 2021-10-04 at 09.18.06
Screenshot 2021-10-04 at 09.18.062310×464 49.9 KB

Screenshot 2021-10-04 at 09.20.01
Screenshot 2021-10-04 at 09.20.012350×1574 273 KB

What more step can I take to get the $MESSAGE formatting correct for slack/notify?

"text": "```$MESSAGE```"

Let me say that line
terraform show /tmp/tf.plan | grep -E "^\s*[#~+-]" | awk '{printf "%s\\n", $0}'|sed 's/\"/\\"/g' > message.txt only produced empty message.txt in my case. But I am happy with terraform show /tmp/tf.plan | grep -E "\#" > message.txt as this is what I need basically.

Thank you for your painstaking help and looking forward to your response