Issues fetching artifacts vi UI and API

Feedback & Bug Reports
, ,
1

Starting around March 17th at midnight UTC, I noticed some issues/inconsistencies when fetching build artifacts within a public project. Take for example https://app.circleci.com/pipelines/github/crystal-lang/crystal/11514/workflows/ec8923f6-0c3a-4da4-b28f-1dee01e0d29b/jobs/75119/steps. The workflow and job were both successful, however when looking at the artifacts tabs, it has an error: Error: [Network Error] Failed to fetch.

When trying to fetch the artifacts via the API via https://circleci.com/api/v1.1/project/github/crystal-lang/crystal/75119/artifacts I get a Build not found error, even tho visiting https://circleci.com/api/v1.1/project/github/crystal-lang/crystal/75119 shows that the build exists and was successful. However I do get the artifacts back when using the V2 API https://circleci.com/api/v2/project/github/crystal-lang/crystal/75119/artifacts, but not in all cases. for example when using curl (or an incognito window) both of them respond with Bad request.

$ curl https://circleci.com/api/v2/project/github/crystal-lang/crystal/75119/artifacts
{
    "message": "Bad request."
}
$ curl https://circleci.com/api/v1.1/project/github/crystal-lang/crystal/75119/artifacts
{
    "message": "Bad request."
}

Additionally, you can get the V2 API to return the data in an incognito window by:

  1. Open https://circleci.com/api/v2/project/github/crystal-lang/crystal/75119/artifacts in incognito window and see you get a Bad request.
  2. In another tab in the same window open https://circleci.com/api/v1.1/project/github/crystal-lang/crystal/75119
  3. Refresh the first tab and notice you now get the artifacts back

I also did validate that other public projects have the same problem, so it’s not isolated. E.g. https://app.circleci.com/pipelines/github/huggingface/transformers/60141/workflows/7a4c5f50-a2e8-40f7-93b7-6cbd9e234b44/jobs/736460/steps which I found at random. Updating the API requests to use that user/project and 736460 as the build number results in the same behavior.

Overall this behavior seems super weird and can’t imagine it being expected. I looked around and didn’t notice any existing threads about this, nor anything on https://status.circleci.com/, nor anything recent/relevant on Track CircleCI Updates - CircleCI. This particular V2 endpoint doesn’t seem to be mentioned in CircleCI API either.

Any ideas on the cause/solution for this?

ref: fix: Nightly build failing to fetch artifacts by marcdrury · Pull Request #21 · crystal-lang/install-crystal · GitHub

2

I’m seeing the same thing in our project. Out of a sudden we started getting HTTP 400 when trying to access artifacts via v2 API.

For example the artifacts for job 1278363: https://app.circleci.com/pipelines/github/ethereum/solidity/28794/workflows/24239abb-436c-4a0c-9bf4-ed80474192f8/jobs/1278363/artifacts

  • Trying in a browser session without cookies I get an error message:
    Error: [Network Error] NetworkError when attempting to fetch resource.
  • With curl I get HTTP 400:
    curl 'https://circleci.com/api/v2/project/gh/ethereum/solidity/1278363/artifacts'

    {
    "message": "Bad request."
}
  • I can access the artifacts in a browser session where I’m logged into CircleCI. Seems like the key element is the ring-session cookie (i.e. authentication).
  • With curl I can access it if I use an API token.

It looks like CircleCI has disabled unauthenticated access to the artifacts/ endpoint. Is that intentional? I haven’t seen that announced anywhere. If intentional, seems kinda pointless since the artifacts themselves are still accessible without any kind of authentication.

3

I just posted that I’m experiencing the same thing, with some details on my situation, but my post was caught in the spam filter (due to links I guess)? In any case, there are more people affected by this. The ability to make unauthenticated requests to some API endpoints seems broken.

EDIT: My post has finally shown up above.

4

An additional problem is that this endpoint does not work with project tokens. We wanted to use a read-only one as a workaround. It forces you to use personal tokens for some reason, which we do not want to use.

5

Thanks for reporting this. We have identified the root cause and will be rolling it back within the next hour.

6

The changes have been reverted, please let us know if you are still seeing any issues.

and a huge thank you for the detailed reports @Blacksmoke16 & @cameel. They helped us pinpoint the issue immediately.

7

:muscle: glad to hear! Everything seems to be back to normal. I re-ran some of the jobs that were failing due to this and they passed. Thanks for the quick response.