Hello,
On August 3, 2022, our infrastructure teams implemented a CDN configuration change that was intended to give us better visibility and control over some of our network ingress points.
This was fully deployed yesterday, and we were alerted to an unexpected side-effect: certain requests to our API began to fail with an HTTP ‘403’ error. We identified the cause of the failures, and reverted the configuration. The requests that began to fail are all GET requests with a request body. Prior to this change, our system accepted the GET and ignored any additional payload. The updated CDN configuration that we deployed began to reject those requests as invalid, and our upstream provider does not allow us to change this behavior
We would like to re-enable this configuration on September 19. In order to enable a smooth transition, people may need to update their tools. Here are some ways to avoid running into this problem once we re-enable the CDN configuration:
Using Ruby
Based on review of logs while the configuration was enabled, the highest volume of failing requests appeared to originate from the circleci Ruby Gem. Thanks to a community member (who was also one of the initial people to alert us to the issue) this has now been updated and we recommend everyone update to version 2.1.0 or later.
Using Curl
When making API queries with Curl, be sure to avoid using the -d, --data, --data-ascii, --data-binary, --data-raw, and --data-urlencode options when making GET requests.
Other clients may also need to be updated to exclude sending a request body along with a GET request. However, as nearly all clients default to not sending a body with GET requests, we expect that only tools where this default has been overridden will be affected.