Beginning around June 14, MacOS self-hosted runners are failing with 400-Bad Request

Starting last week, we’ve had a lot of issues with our small fleet of self hosted MacOS runners. First it appeared that an automatic upgrade failed, then once that issue was resolved we’re now seeing the issue of runners showing as active from within CircleCI, but the runner never processes jobs. The runner logs are full of the following error, which seems to be the root cause:

time=“2022-06-15T07:21:40-04:00” level=error msg=“error running task” error=“unexpected error claiming task: server supplied detail: hostname not valid (400-Bad Request)”

Versions
MacOS: arm64 on Monterey 12.3.1
Runners: 1.0.39853-df56b3e

Hello Nick

I just wanted to follow up on this post in case others run into the same issue.

This issue was due to an issue on CircleCI in regards to the hostname that was being used.

This issue has now been resolved but if anyone else runs into a similar issue please feel free to raise a support ticket linked here

Kind Regards
Owen Oliver

1 Like

I guess the question is what changes have you made as self-hosted runners are now reporting

Agent download unsuccessful.
error: allocation=“7OUPNF54” download failed: error downloading task agent version=“1.0.132505-9428cda7” os=“linux” arch=“amd64”: could not write file “/tmp/circleci-launch-agent3989761449/circleci-agent/1.0.132505-9428cda7/linux/amd64/circleci-agent.tmp”: context deadline exceeded (Client.Timeout or context cancellation while reading body)

The whole path is created but no file is retrieved.

Hello

There was an issue around this time where folder permissions were changed but they are not updated fully within the update causing it to fail as you have seen.

This has now been resolved and should no longer be causing an issue but if you are still having issues you can manually run

chown -R circleci:circleci /var/opt/circleci

This will allow the update to install

Kind Regards
Owen Oliver