Config.yaml multiline folding syntax

Feedback & Bug Reports
1 
      - run: >
          curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY
            -d team=bit
            -d service=boost-bit-workflows
            -d buildId=$CIRCLE_BUILD_NUM
            -d "links[0].title=CircleCI build"
            -d "links[0].url=$CIRCLE_BUILD_URL"
            -d "links[1].title=Diff url"
            -d "links[1].url=$CIRCLE_COMPARE_URL"
            -d "notifySlackChannel=boost-bit-squad"

Results in this error message:

#!/bin/bash -eo pipefail
curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY
  -d team=bit
  -d service=boost-bit-workflows
  -d buildId=$CIRCLE_BUILD_NUM
  -d "links[0].title=CircleCI build"
  -d "links[0].url=$CIRCLE_BUILD_URL"
  -d "links[1].title=Diff url"
  -d "links[1].url=$CIRCLE_COMPARE_URL"
  -d "notifySlackChannel=boost-bit-squad"
/bin/bash: line 1: -d: command not found
Exited with code 127

But it should make it one line,

According to the 1.0 spec:
http://yaml.org/spec/1.0/#id2566944

1 Like
2

The line breaks are still seen by the CLI. You need to append \ to every non-last line.

3 
$ pbpaste  > config.yml
$ irb
irb(main):001:0> require 'yaml'
=> true
irb(main):008:0> print YAML.load(File.read('config.yml'))[0]['run']
curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY
  -d team=bit
  -d service=boost-bit-workflows
  -d buildId=$CIRCLE_BUILD_NUM
  -d "links[0].title=CircleCI build"
  -d "links[0].url=$CIRCLE_BUILD_URL"
  -d "links[1].title=Diff url"
  -d "links[1].url=$CIRCLE_COMPARE_URL"
  -d "notifySlackChannel=boost-bit-squad"=> nil
irb(main):009:0>

Ruby’s YAML parser behaves the same as the Java-based one that we are using internally.

I think it’s an indentation issue in the input string.

4

Oddly, if you indent the string like this is behaves you expect:

      - run: >
          curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY
          -d team=bit
          -d service=boost-bit-workflows
          -d buildId=$CIRCLE_BUILD_NUM
          -d "links[0].title=CircleCI build"
          -d "links[0].url=$CIRCLE_BUILD_URL"
          -d "links[1].title=Diff url"
          -d "links[1].url=$CIRCLE_COMPARE_URL"
          -d "notifySlackChannel=boost-bit-squad"

Yay for YAML :roll_eyes:

1 Like
5

Ooh, I didn’t know you could do that. Interesting!

Either way, use this parser to see how your file will be understood.

6

Wow! That was a speedy reply 10x a lot guys. What parser is actually used by circleci? I thought I’ve seen Clojure errors so I assumed it was that, but I guess its probably just a Java lib.

7

This does seem as correctly meeting YAML 1.0 specification. Not a bug.

Notice the indents:

test: >
  foo
  --bar
  --buzz

— gives { "test": "foo --bar --buzz\n" } as expected, but

test: >
  foo
    --bar
    --buzz

— gives { "test": "foo\n --bar\n --buzz\n" }.

This is correct on the grounds of 4.6.6.1, emphasis mine:

The combined effect of the processing rules above is that each “paragraph” is interpreted as a single line, empty lines are used to represent a line feed, and “more indented” lines are preserved.

Also see Example 4.24 where the spec gives understandable motivation behind this:

indented text: >
    This is a folded paragraph
    followed by a list:
     * first entry
     * second entry
    Followed by [...]

above is equal to: |
    This is a folded paragraph followed by a list:
     * first entry
     * second entry
    Followed by [...]

TL;DR you must not indent subsequent lines in a folded block scalar. This would work as you expect:

     - run: >
          curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY
          -d team=bit
          -d service=boost-bit-workflows
          -d buildId=$CIRCLE_BUILD_NUM

or this (using literal block scalar and bash-level line continuations):

     - run: |
          curl https://shipit.ovotech.org.uk/deployments?apikey=$SHIPIT_API_KEY \
            -d team=bit \
            -d service=boost-bit-workflows \
            -d buildId=$CIRCLE_BUILD_NUM \
            [...]