YAML parser confused by colons in shell commands

Feedback & Bug Reports
1

The following snippet doesn’t parse as expected:

dependencies:
  override:
   - echo 'extra-include-dirs: [foo, bar]' > config.yaml

The issue is that your parser thinks that “echo 'extra-include-dirs” is the name of a key in a map. In this particular case, it turns out removing the space between the “:” character and the “[” above is a workaround to this particular parser bug.

1 Like
2

Hi @mboes,

I don’t believe this is a bug with our parser as it’s just the nature of how YAML works in general by recognizing a colon and a space as a key-value separator. You should be able to work around this by making your circle.yml look like this instead:

dependencies:
  override:
   - |
     echo 'extra-include-dirs: [foo, bar]' > config.yaml

For more information, see “2.3 Scalars” in the YAML spec, or try your YAML out at http://www.yamllint.com/.

Cheers,
Frank

1 Like
3

Hi folks, @frank, @mboes!

I’m facing the same issue, yaml parsing since to be broken. I also believe it’s a bug!

--- 
dependencies: 
  cache_directories: 
    - ~/.cache/pip
  override: 
    - 
      ? | 
        docker pull stackstorm/buildpack:centos7

This is not just useless complex key, which produces error:

Action failed: Configure the build
modifiers to 'buildpack:centos7' must be a map

Action failed: Configure the build

But it’s a feature which I’m using to run commands in parallel by help of small wrapper bash function which I called step. So the following test code works like charm without all extra pollution [ $CIRCLE_NODE_INDEX = smth ] etc, all the logic to fire action on a specific node is wrapped. Yaml at the same time stays clean, but fully descriptive:

test:
  override:
    - ? |
        step tox -e lint; next
        step tox -e py27
      : parallel: true

All the above examples are absolutely valid yamls, you can check on it in www.yamllint.com! While test example works and doesn’t produce any error, dependency stage with docker pull command fails, exactly because of parse failure broken and can’t parse line containing :.

– Denis

4

The complex key identifier ? uses a colon to know when the complex key has ended.

In the docker pull example you showed above, there is no need to use both a complex key identifier and a scalar identifier. The scalar by itself should suffice. It should look something like this:

dependencies: 
  cache_directories: 
    - ~/.cache/pip
  override: 
    - | 
      docker pull stackstorm/buildpack:centos7

-Frank

5

Yup @frank, I agree.

That’s indeed sufficient, however the following form is valid according to yamllint

--- 
dependencies: 
  cache_directories: 
    - ~/.cache/pip
  override: 
    - 
      ? "docker pull stackstorm/buildpack:centos7"
      : ~

And it couldn’t be parsed correctly by cicrcle yaml parser, I don’t think it’s a matter of high importance, but nevertheless parsing fails. It’s probably caused due to that parser expects only modifiers hash, but actually there could be : ~ also. Anyway, it doesn’t correlate with the original matter of the issue, where just simply incorrect syntax has been provided. :slightly_smiling:

1 Like
6

Just hit myself, and I also think this is a bug in the YAML parsing. According to the spec you can put anything but control characters, backslashes, and double quotes inside double-quoted string scalars.

http://www.yamllint.com/ is also quite happy with

--- 
foo: "bar: baz"

(whereas dropping the quotes produces an error)