CircleCI CLI Orb Pack Beta

Hello everyone!

We have been working hard to make orb development a pleasant experience, and part of this means figuring out the best way to package orbs into their final form. To address this, we’ve created a new command for the CircleCI CLI: orb pack. This behaves similar to the existing command used for packing orbs, config pack, but with some orb-specific behaviour, We’re looking for orb developers to test this new command and help us catch edge cases we normally wouldn’t be able to.

Packing external scripts

First, we’ve introduced the ability to import other files into your packed orbs with the << include(filename) >> syntax. This can be scripts, JSON files, any sort of text you want to keep separate from the YAML for syntax highlighting (or other reasons)! These values will be replaced with the file contents when the orb is packed.

# src/@orb.yml
version: 2.1

# ...

commands:
  foo:
    steps:
      - run: << include(scripts/hello-world.sh) >>
# src/scripts/hello-world.sh
echo "Hello, world!"

Result:

version: 2.1

# ...

commands:
  foo:
    steps:
      - run: echo "Hello, world!"

Stricter project structure

Second, we’re standardizing the form that “destructured orbs” take (the layout of the project’s directory). Orbs that need to be packed must have an @orb.yml file, and will only pack the commands, examples, executors and jobs directories. So other directories, such as scripts from the example, are ignored, and you won’t end up with empty entries in your configuration, such as scripts: {}.

How do I start using it?

The command is present in the latest version of the CLI (v0.1.8731) (but it’s hidden).

While we’ve done our best to cover as many edge cases as possible, we’re limited by our sample size. We’re looking for more orbs to test against, preferably ones that will be able to take advantage of the features introduced. Give it a go, and please report any and all issues you encounter, either here or on GitHub.

We’re also working on the Slack orb to allow it to be more flexible, and utilises this new orb pack command: https://github.com/CircleCI-Public/slack-orb/tree/Major-block-kit-redesign

2 Likes

How do we pass parameters into scripts that are imported via the include syntax?

1 Like

Hello @jrnail23,
We’ll have more documentation coming soon. The way we are considering now might look like this.

example run statement

- run:
    name: Hello World
    environment:
      MY_PARAM_1: <<parameters.param-1>>
      MY_PARAM_2: <<parameters.param-3>>
    command: << include(scripts/hello-world.sh) >>

One nice benefit here is now your scripts will not contain the parameter syntax which is not valid bash scripting. Instead, you can expect a set of known environment variables.

Alternatively, you should be able to in your BASH script (or another language) and capture the parameter similarly.

example script
MY_PARAM_1="<<parameters.param-1>>"
echo “$MY_PARAM_1”

This will be inserted into the final orb.yml file as a literal string, So the <<parameters.param-1>> will be replaced at run time as they are today. The downside I see to this method is you will not as easily be able to locally execute the script.

Very curious to hear your thoughts!

1 Like

The first way is better, for the reasons you mentioned (local execution)… I think you’d definitely want to keep your actual script free of CircleCI syntax. While I’d probably prefer some kind of explicit parameterization, the environment variable approach seems like a good compromise, TBH.

I’m looking forward to giving this a try!

2 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.

Seems like this is not true anymore. https://github.com/CircleCI-Public/circleci-cli/pull/444 escapes the << in the script files.

Is the preferred way to use environment now?

1 Like

This is true. I will cross out that alternative method. Please use the first method.

Here is how we work with the env_var_name parameter, which I believe is the only tricky instance.

For strings, it is fairly straight forward:

- run:
    name: Hello World
    environment:
      MY_PARAM_1: <<parameters.param-1>>

The script in this instance should expect an environment variable MY_PARAM_1 loaded with some value.

Here’s a snippet from something we are working on with the shellcheck orb

  - run:
      name: Run Shellcheck
      environment:
        SC_PARAM_EXCLUDE: <<parameters.exclude>>
Set_SHELLCHECK_EXCLUDE_PARAM() {
    if [ -n "$SC_PARAM_EXCLUDE" ]; then
        SHELLCHECK_EXCLUDE_PARAM=" --exclude $SC_PARAM_EXCLUDE"
    else
        SHELLCHECK_EXCLUDE_PARAM=""
    fi
}

The string type parameter exclude is assigned to the env var SC_PARAM_EXCLUDE and we reference that within our script, rather than the <<parameter>> syntax which is exclusive to CircleCI Configuration YAML. This keeps our shell scripts completely portable, and separate from our config.

What I ended up doing for env var type was a little smaller any reason why what you have above is safer or better?

PARAM_USER=$(printf '%s\n' "${!PARAM_USER_ENV_VAR}")

https://github.com/roopakv/orbs/blob/master/scripts/trigger-pipeline.sh is where i use it