"infrastructure as code" and circleci limitations

Feedback & Bug Reports
1

I think most people here know of the idea of “infrastructure as code” and the aim of replacing all the historic manual tasks within the IT realm with scripts that can automate and control everything.

In general, circleci is part of the toolset to enable this transformation as it allows the automation of the whole build process, but its current inane inability to allow environment variables to be used within its own scripting language without using workarounds is very unfriendly and a real pain at times.

Earlier this week a firm example of the problems this lack of ability can cause showed up and so I can now write this with an example of a real impact to our processes, rather than just expressing a dislike due to the limitations this causes in other parts of the environment.

Background

Self-hosted runners failed due to some reason that is yet to be fully understood as the setup process failed when trying to update the local agent. Maybe the update process could be written better, but at the end of the day this is just an example of the fact that things will go wrong and everyone needs to have in place solutions that can cope based on the needs of their business.

In an “infrastructure as code” world

With the tools I have in place, to resolve this issue I should have been able to just change one value in my secrets store (think fancy .env file) and rerun the ci jobs. The secret store is able to sync changes into circleci jobs as environment variables, but circleci scripts do not use these for anything instead they have a totally different parameter system.

In the current “circleci” world

I had to go in and hand edit each config.yml to change the executor statement of each defined job and once the issue was resolved I had to then reverse the change - even as a small business this meant creating branches and pull requests in each project for each update - not a quick and painless process.

The platform needs a solution that does not involve using a shell script to dynamically write a circleci script or a curl call from one build job to another build job where environment variables can be passed as parameters and so accessed in the second build job.

2

Now for a hopefully more constructive post/request

Please can you provide a page per project within the web interface where we can provide the static mapping of named parameters to values in the same way that you do for the more dynamic Trigger Pipeline option that already exists.

As long as you evaluate the provided value as a possible known at runtime environment variable we end up with a solution that correctly allows dynamic external values to be passed to the standard config.yml file at runtime.

This would address many of the requests/questions posted here in the past and would allow all the issues I have so far come across when using self-hosted runners.

Once such an interface is in place and proven it could also be made visible via the API as all it is, is a simple matrix 3 by n matrix of values.