Trouble with persistent workspaces

Build Environment
1

Hey there,
I’m the sysadmin with my company who has been tasked with configuring our circleci integration to copy the contents of our build to an aws bucket ONLY if the changes that are made are coming from our develop branch. I’ve never worked with any integrations like this so it was a bit of a learning curve but after reading some documentation, it seemed like something I would be able to do.

We already had a yml file so I just had to make changes to it to accomplish this task. We were previously running everything in one job - build then copy to s3 but the problem was it was copying all changes to our develop bucket, regardless of what branch you were working on. What I did was separate the build and copy steps into two separate jobs and used workflows to restrict the copy job to just the develop branch.
However, because the build directory was no longer in the same job, the copy step failed. So i looked into persistent workspaces (and executors) as a solution but continue to get a configuration error and I’m not sure what the error is pointing to specifically. It looks like I’m missing a “path” key for the persist_to_workspace field but looks like there’s more to it than that.

Here’s the error:
ERROR IN CONFIG FILE:

[#/jobs] 5 schema violations found

Jobs is a map

1. [#/jobs/build] only 1 subschema matches out of 2

|   1. [#/jobs/build/steps/6] 0 subschemas matched instead of one

|   |   1. [#/jobs/build/steps/6] Input not a valid enum value

|   |   |   Steps without arguments can be called as strings

|   |   |     enum:

|   |   |     - checkout

|   |   |     - setup_remote_docker

|   |   |     - add_ssh_keys

|   |   2. [#/jobs/build/steps/6/persist_to_workspace] only 1 subschema matches out of 2

|   |   |   1. [#/jobs/build/steps/6/persist_to_workspace] 0 subschemas matched instead of one

|   |   |   |   1. [#/jobs/build/steps/6/persist_to_workspace] required key [path] not found

|   |   |   |   2. [#/jobs/build/steps/6/persist_to_workspace] required key [paths] not found

2. [#/jobs/copy-to-s3] only 1 subschema matches out of 2

|   1. [#/jobs/copy-to-s3/steps/0] 0 subschemas matched instead of one

|   |   1. [#/jobs/copy-to-s3/steps/0] extraneous key [at] is not permitted

|   |   |   Permitted keys:

|   |   |     - persist_to_workspace

|   |   |     - save_cache

|   |   |     - run

|   |   |     - checkout

|   |   |     - attach_workspace

|   |   |     - store_test_results

|   |   |     - restore_cache

|   |   |     - store_artifacts

|   |   |     - add_ssh_keys

|   |   |     - deploy

|   |   |     - setup_remote_docker

|   |   |   Passed keys:

|   |   |     []

|   |   2. [#/jobs/copy-to-s3/steps/0] Input not a valid enum value

|   |   |   Steps without arguments can be called as strings

|   |   |     enum:

|   |   |     - checkout

|   |   |     - setup_remote_docker

|   |   |     - add_ssh_keys

and the config

version: 2.1
executors:
  environment:
    docker:
      - image: circleci/node:10.14-browsers
    working_directory: ~/repo

jobs:
  build:
    executor: environment
    steps:
      - checkout
      # Make workspace directory
      - run: mkdir -p ~/repo/workspace
      # Setup for AWS
      - run:
          name: Install depends
          command: |
            sudo apt-get update && sudo apt-get install --no-install-recommends -y \
              python3-pip \
              python3-dev \
              python3-setuptools \
              apt-transport-https \
              ca-certificates \
              zip \
              curl \
              gnupg2 \
              software-properties-common

      - run:
          name: Install awscli with pip
          command: |
            pip3 install --upgrade --user wheel
            pip3 install --upgrade --user awscli

      - run:
          name: npm install
          command: |
            npm install
            npm install @ng-bootstrap/ng-bootstrap
      # run production build
      - run:
          name: Production build
          command: |
                npm run build

      # Create persistent workspace
      - persist_to_workspace:
          root: ~/repo/workspace

  copy-to-s3:
    executor: environment
    steps:
      - attach_workspace:
        at: ~/repo/workspace

      - run:
          name: Copy build to S3
          command: |
                aws s3 cp ./folder/path/ s3://bucketname/ --recursive --acl public-read

workflows:
  version: 2
  build_and_copy:
    jobs:
      - build
      - copy-to-s3:
          requires:
            - build
          filters:
            branches:
              only: workingbranch

Any pointers would be appreciated. Thanks

2

I managed to fix the error I was getting by adding the paths: . key and fixing two indention errors…

However, it’s still failing on the copy job due to missing awscli… I guess I don’t fully understand what the persistent workspace does if it’s not storing the dependancies that were installed during the build job. I can add the steps to install awscli to each job but that seems like it’s unnecessary

edit: this has been a really good learning experience for me… besides the spacing errors causing me problems, I realized the contents of the build weren’t being added to the workspace because I created a separate folder for it and attached it there instead of at my working directories root folder. geez… anyway, iv’e gotten past the big hurdles. Now the issue I’m facing is an access denied error on the copy job itself. At least it’s trying to copy so, that’s progress!

edit 2: fixed the permissions issue in aws and everything is now working as expected. great success! the only thing that’s a little annoying is that I have to install awscli for each copy job but that’s ok for now. hopefully I can find a way to centralize that part of it in the future. and hopefully someone else will find my learning struggles helpful lol

key takeaways for me:

  1. always triple check your indentions/spacing…
  2. pay attention to your workspace mount points
  3. read the documentation