Merge artifact result

Hi,
I’m new with Circleci and I’m trying to find a way to merge my artifacts created during in job with several parallels.
For now, I’m seeing all artifacts created with each parallel but at the end of the job (and not in another job), I would like to merge the result from each artifacts.
I found a way by downloading each artifact content but it can be crazy to handle (especially if the created artifacts has nested folders and files…).
I tried to use persist_to_workspace but the only way to get access to it would be on another job.
Any idea?
Thanks :slight_smile:

Hi @geekorlife,

Can you please clarify what you mean by merge artifact results. Are you trying to store additional artifacts to the same file multiple times in a singular job? If so, you can add multiple store_artifacts steps within the same job. If you are trying to add artifacts to the file across jobs you can do this using workspaces. You can find more information in our docs here. Please feel free to share a config.yml file if you’d like.

I’m not sure if I’m having the same issue, but I think I do. My situation is that I have a workflow that runs three jobs. Each job creates an artifact and uses store_artifacts. This works in the sense that I see the artifacts on the job results pages. However the API to request artifacts works on builds, not jobs. And when I request the list of artifacts for the build, I get only the one from the first job.

By the way, with “the API to request artifacts” I mean this one (described here):

https://circleci.com/api/v1.1/project/:vcs-type/:username/:project/latest/artifacts

Hi @smedmen,

When you use latest with the artifacts endpoint ( https://circleci.com/api/v1.1/project/:vcs-type/:username/:project/latest/artifacts) CircleCI will return the lasts artifact. Your first job is likely the latest job to be built. It sounds like you want to retrieve all artifacts for the entire workflow.

You can persist data throughout a workflow using workspaces and run your store_artifacts step in your final/last job. You can use to requires key to make sure the job that is running your store artifacts step is last. If you do this, all of your artifacts will be stored in the same build. Therefore, you can use this endpoint (https://circleci.com/api/v1.1/project/:vcs-type/:username/:project/latest/artifacts) to retrieve all of your latest builds artifacts.

2 Likes

Right, that’s pretty much what I ended up doing except that I have a separate job for storing the artifacts, that depends on a number of other (potentially parallel) jobs that generate the artifacts. Thanks for confirming that’s the right way to do it, I thought I was missing something. :+1:

@smedmen Absolutely!

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.