Leveraging CircleCI API to include build logs in Slack notifications

Leveraging CircleCI API to include build logs in Slack notifications

The Slack Orb allows you to send notifications, and customize the message sent. In most cases, this is used to indicate a build status, such as if it failed.

While the above is helpful, it still requires you to navigate back to the build to get information about why it failed.

Wouldn’t it be nice if you could include the last few lines of the log output with your Slack message? This could help easily identify the issue, such as the exit code reason or specific error messages. Maybe something like this…

The following guide will provide step by step details on setting this up!

Requirements

  1. This guide utilizes Orbs, so you will need to be on at least version 2.1.
  2. The API calls and data manipulation are done in bash so your image will need to support that (including utilizing jq).
  3. While this should work out of the box, you may need to adjust slightly to account for how your steps output values.

Setup the Slack orb

Information on setting up the Slack orb.

Once the above is done you will need to ensure you include it within your config.yml file with the following:

orbs:
  slack: circleci/slack@4.2.1

Configure Environment Variables

Both the Slack orb and the run steps contained in this guide require some environment variables to be set. These can be done at the project level, or, if you plan to use it across projects – set these up in a context.

Variable Name Description
CIRCLE_API_TOKEN This will be a personal API token.
SLACK_ACCESS_TOKEN The OAuth token acquired through the Slack setup steps.
SLACK_DEFAULT_CHANNEL If no channel ID is specified, the Slack orb will attempt to post here.

The SLACK_ACCESS_TOKEN and SLACK_DEFAULT_CHANNEL should already be set from following the “Setup the Slack orb” step above.

Add the run step that collects and prepares log data

The snippet below should be able to be added as one of the last steps in the job you want to be notified in. While you can copy and paste it, you may want to adjust some parts.

  1. Remove or adjust jq install lines accordingly (first two steps under command). If you already have it installed on the image, remove them. If utilizing an alpine image, adjust to apk.
  2. On line 9 of the command step, adjust the amount of rows from the log you want returned. It’s set to 10, tail -n 10, so adjust 10 accordingly.
- run:
    when: on_fail
    name: Generate last log lines for slack message
    command: |
      sudo apt-get update
      sudo apt-get install jq
      API_URL=$(echo $CIRCLE_BUILD_URL | cut -d/ -f4-7)
      FAILED_STEP=$(curl "https://circleci.com/api/v1.1/project/${API_URL}?circle-token=${CIRCLE_API_TOKEN}" | jq '.steps | .[] | flatten | map(select(.status? == "failed")) | .[] | {allocation_id, step}')

      ALLOCATION_ID=$(echo "${FAILED_STEP}" | jq -r '.allocation_id')
      STEP=$(echo "${FAILED_STEP}" | jq -r '.step')
      curl -o slack_response.txt "https://circleci.com/api/v1.1/project/${API_URL}/output/${STEP}/0?file=true&allocation-id=${ALLOCATION_ID}&circle-token=$CIRCLE_API_TOKEN"
      sed -i 's/\r$//g' slack_response.txt
      SLACK_MESSAGE=$(cat slack_response.txt | tail -n 10 | sed '$!s/$/\\n/' | tr -d '\n')
      echo $SLACK_MESSAGE
      echo "export SLACK_MESSAGE='${SLACK_MESSAGE}'" >> $BASH_ENV

Invoke the Slack orb and utilize the message generated

The below step should go right after the above step. You can adjust as needed, but you will likely want to keep the following block:

{
  "type": "section",
  "text": {
    "type": "mrkdwn",
    "text": "```${SLACK_MESSAGE}```"
    }
}
           

As the above ensures the log message is included in the Slack message. Add this below the “Generate last log lines for slack message” step that you added in the last section.

- slack/notify:
    event: fail
    custom: |
      {
        "blocks": [
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "```${SLACK_MESSAGE}```"
            }
          },
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "Build failed, above is last few lines of the log"
            },
            "accessory": {
              "type": "button",
              "text": {
                "type": "plain_text",
                "text": "Click Me",
                "emoji": true
              },
              "value": "click_me_123",
              "url": "${CIRCLE_BUILD_URL}",
              "action_id": "button-action"
            }
          }
        ]
      }

:tada: That’s it! You should now receive the last few log lines of the failed step as part of your Slack notification.

Resources

This guide stemmed from a previous solution I provided a customer, which was a bash script that did something similar.

If helpful to see how this would look in a config.yml see the following gist:

4 Likes

@nbialostosky Thanks for posting this, very useful and something I have had on my to-do list to add to our deployments for a while.

One minor note is that the example config.yml on your github needs indents added for lines Lines 19-33 for it to be accepted by the circleci yaml parser. Correct indent in this post though.

Thanks for calling this out! I have updated the gist accordingly, and should be valid now!

1 Like

Thanks for this! I’ll be implementing this today, but I have a followup question: Is it possible to calculate the URL of a log file artifact for the failed job? The URL subdomain is a mix of the CIRCLE_BUILD_NUM and some other number. https://{Build Num}-{???}-gh.circle-artifacts.com/0/{file name}. My goal would be to include a button in the Slack message to directly download the log file.

Hi @jakeatoms, great question!

You can actually just make a secondary API call to the artifacts endpoint, I wrote a support article on this:

So you would add this run step after your artifact upload step and then include it in the Slack custom message you send. Hope that helps clarify!

1 Like