Path-filtering orb: get names of files which changed

Build Environment
1

My project has several expensive queries which reside in separate files within the src/sql folder of the code repo. For example:

.
├── .circleci
│   ├── config.yml
│   └── update_queries.yml
├── src
│   ├── sql
│         ├── long-running-query1.sql
│         ├── long-running-query2.sql
│         ├── long-running-query3.sql

My goal is to reduce build cost and only run the query(ies) whose file(s) changed.

Below is the path-filtering excerpt from the CircleCI config.yml file:

      - path-filtering/filter:
          name: check-updated-files
          mapping: |
            src/sql/.* run-redeploy-queries-job true
          base-revision: << pipeline.git.branch >>
          config-path: .circleci/update_queries.yml

Is there a way to access the exact names of the files which got matched by the mapping regex? I would ideally need to access those from the docker, which gets spun up by the matching workflow in update_queries.yml, so that I can trigger only that/those query(ies) which changed.

I would prefer a more general solution over the one where I would have a separate mapping for every file.

Any ideas or hints are welcome and thank you in advance.

2

Hello

You can set the mapping to specific files instead of the whole directory.

If you change your mapping to match as below it will only work if the specific file is updated

          mapping: |
            src/sql/long-running-query1.sql run-redeploy-queries-job true
            src/sql/long-running-query2.sql run-redeploy-queries-job true
            src/sql/long-running-query3.sql run-redeploy-queries-job true

If this does not work in the way you would like you could also move each into its own folder and define the mapping to each folder so that it will only trigger when the folder updates.

Kind Regards
Owen Oliver

3

Hi Owen, I appreciate your reply. However, it is not what I was hoping for.

What I had in mind is described in this feature request in the path-filtering orb’s repo:
get exact names of files which changed.

My motivation is as described in the request (copy/pasting it here):

That is because to support such a solution would be difficult. For example, suppose a new file were added or a name change were introduced to an existing file. Then the continue config file, pointed to by config-path, would need to be updated in either case. Furthermore, the need arises to document the update process and teach it to colleagues who would like to make a change, but have not yet seen the project. Finally, imagine you’ve dozens of files, changes in which need to be tracked. This means that the continue config file at config-path would grow in size significantly and require more effort to maintain.

Thanks!

1 Like