Environment variable set to the number of available CPU's

Looking at the list of injected environment variables I do not see one that provides information about the resource limits on the current container or VM.

I wanted to verify this is the case and ask if it would be possible to get one.

My reasoning is noted in the resource class documentation https://circleci.com/docs/2.0/configuration-reference/#resource_class

Note : Java, Erlang and any other languages that introspect the /proc directory for information about CPU count may require additional configuration to prevent them from slowing down when using the CircleCI 2.0 resource class feature. Programs with this issue may request 32 CPU cores and run slower than they would when requesting one core. Users of languages with this issue should pin their CPU count to their guaranteed CPU resources.

If there was an environment variable it would allow a way to set the number of schedulers/threads a runtime uses in a generic way, and even in an orb, instead of requiring each user to define the necessary configuration and change it depending on the resources they allocate to a job.

1 Like

I can confirm this. It is impossible to get the CPU count in the bash:

        cpuset=$(cat /sys/fs/cgroup/cpuset/cpuset.cpus)
        limits=(${cpuset//-/ })
        TOTAL_CPUS=$((${limits[1]}-${limits[0]}+1))
        echo "CPU count is ${TOTAL_CPUS}"
        export TOTAL_CPUS

I independently went through precisely the same thought process as the OP and am also unaware of a way to correctly introspect the number of available vCPUs, so I just wanted to second this feature request. I will add that the issue extends beyond languages to any tooling that introspects /proc (e.g., jest or, I suspect, msbuild -maxCpuCount). Furthermore, the symptoms resulting from this issue could vary from slow build times, to out-of-memory errors, to flaky builds, to exposing race conditions in buggy tools (which I suppose is a good thing), making it often rather opaque and troublesome to debug for those who aren’t already aware of it.

I can’t speak to the other executors, but I was able to determine the number of “vCPUs” available in my docker executor environment by inspecting cgroup configuration:

echo $(($(cat /sys/fs/cgroup/cpu/cpu.shares) / 1024))

Of course, it would be much nicer to have this info exposed through /proc or at least as an environment variable.

1 Like