I’m working on a program for automatically downloading logs of CircleCI runs, and I’ve run into a snag that seems to be caused by a bug in the API.
As far as I can tell, records of CircleCI pipeline runs expire after 90 days, after which requests for an expired pipeline’s workflows via the v2 API return workflow objects with id and other fields set to null; my program currently ignores such pipelines. However, I’ve found one pipeline from almost a year ago whose workflow still has an ID, and so my program progresses to trying to download the logs, which involves getting information about the jobs from the v1.1 API, and at this point all requests result in a 404.
The actual requests & responses, in curl form, are:
Pipeline:
curl -sSL -H "Circle-Token: $CIRCLECI_CLI_TOKEN" https://circleci.com/api/v2/project/gh/bids-standard/bids-specification/pipeline/6927
Response:
{
"id" : "e9a9f3e3-2add-41d7-b379-92211d67a33f",
"errors" : [ ],
"project_slug" : "gh/bids-standard/bids-specification",
"updated_at" : "2024-02-21T17:15:48.685Z",
"number" : 6927,
"state" : "created",
"created_at" : "2024-02-21T17:15:48.685Z",
"trigger" : {
"received_at" : "2024-02-21T17:15:47.863Z",
"type" : "webhook",
"actor" : {
"login" : "effigies",
"avatar_url" : "https://avatars.githubusercontent.com/u/83442?v=4"
}
},
"vcs" : {
"origin_repository_url" : "https://github.com/effigies/bids-specification",
"target_repository_url" : "https://github.com/bids-standard/bids-specification",
"revision" : "7d8844e6783f0e79768e635760a647f8b3ae9c5e",
"review_id" : "1671",
"provider_name" : "GitHub",
"commit" : {
"body" : "",
"subject" : "FIX: Implement inheritance for all TSV/JSON files"
},
"branch" : "pull/1671"
}
}
Workflows:
curl -sSL -H "Circle-Token: $CIRCLECI_CLI_TOKEN" https://circleci.com/api/v2/pipeline/e9a9f3e3-2add-41d7-b379-92211d67a33f/workflow
Response:
{
"next_page_token" : null,
"items" : [ {
"pipeline_id" : "e9a9f3e3-2add-41d7-b379-92211d67a33f",
"id" : "dab04d2a-cb81-4206-ba1f-7cad1b3b733c",
"name" : "search_build",
"project_slug" : "gh/bids-standard/bids-specification",
"status" : "success",
"started_by" : "59d64852-c0b7-42d9-9987-23b80e014b0f",
"pipeline_number" : 6927,
"created_at" : "2024-02-21T17:15:48Z",
"stopped_at" : "2024-02-21T17:19:54Z"
} ]
}
Jobs (v2 API):
curl -sSL -H "Circle-Token: $CIRCLECI_CLI_TOKEN" https://circleci.com/api/v2/workflow/dab04d2a-cb81-4206-ba1f-7cad1b3b733c/job
Response:
{
"next_page_token" : null,
"items" : [ {
"job_number" : 20636,
"stopped_at" : "2024-02-21T17:19:54Z",
"started_at" : "2024-02-21T17:15:50Z",
"name" : "build_docs_pdf",
"project_slug" : "gh/bids-standard/bids-specification",
"type" : "build",
"requires" : { },
"status" : "success",
"id" : "f07e42e1-de98-4761-9877-b67a6ffbca04",
"dependencies" : [ ]
}, {
"job_number" : 20635,
"stopped_at" : "2024-02-21T17:17:45Z",
"started_at" : "2024-02-21T17:15:51Z",
"name" : "build_docs",
"project_slug" : "gh/bids-standard/bids-specification",
"type" : "build",
"requires" : { },
"status" : "success",
"id" : "0e57451b-dfc5-4f72-a434-206654fff020",
"dependencies" : [ ]
}, {
"job_number" : 20637,
"stopped_at" : "2024-02-21T17:19:43Z",
"started_at" : "2024-02-21T17:17:54Z",
"name" : "check_links",
"project_slug" : "gh/bids-standard/bids-specification",
"type" : "build",
"requires" : {
"0e57451b-dfc5-4f72-a434-206654fff020" : [ "success" ]
},
"status" : "success",
"id" : "0f5e0239-575d-4186-ba38-023eff11cb0e",
"dependencies" : [ "0e57451b-dfc5-4f72-a434-206654fff020" ]
} ]
}
Jobs (v1.1 API):
for number in 20636 20635 20637
do curl -sSL -H "Circle-Token: $CIRCLECI_CLI_TOKEN" https://circleci.com/api/v1.1/project/gh/bids-standard/bids-specification/$number
done
Responses:
{
"message": "Job not found"
}{
"message": "Job not found"
}{
"message": "Job not found"
}
Is this an indication on some sort of problem on CircleCI’s end? Alternatively, if this is expected behavior somehow, what criteria about the pipeline should my program use to determine that a 404 is expected rather than an error?