# Uploading artifacts recursively is exceptionally slow **URL:** \ **Category:** Bug Reports\ **Tags:** artifacts\ **Created:** [July 20, 2017, 11:14pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558 "2017-07-20T23:14:05Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![QuLogic](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/qulogic/32/16137_2.png) [@QuLogic](https://discuss.circleci.com/u/QuLogic)\ **Post date:** [July 20, 2017, 11:14pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/1 "2017-07-20T23:14:05Z")
When CircleCI is told to upload a directory artifact, it is uploaded one file at a time, which ends up being extremely slow. Here is [a build with two modes of artifact uploading](https://circleci.com/gh/QuLogic/matplotlib/64#artifacts/containers/0) (with [this configuration](https://github.com/QuLogic/matplotlib/blob/circle-docs/.circleci/config.yml)): - various intermediate build steps that we generally don’t need to check often, so it’s uploaded as a tarball; and, - documentation that we generally _do_ want to check (if documentation was changed in the PR), so it’s preferable to upload individually for easy online browsing and it’s uploaded as a directory, recursively. The intermediate steps consist of 10693 files in 107 directories totalling 90M. The compressed tarball produced by CircleCI is 53.7 MB. From the build log, it took 0s to produce the tarball and 1s to upload it as an artifact. On a local build (I did not want to try to download everything individually), the documentation consists of 7285 files in 112 directories totalling 132M. From the build log, it took 5m53s to upload. In summary, the recursive upload was of 2.45 the size as the tarball, but it took _353 times_ longer to upload. That is extremely slow. I would guess it’s creating a new connection for each file, or trying to record in an artifact database the link to each file. It would be so much better for build times (both ours and just for general efficiency’s sake) if this could be batched somehow, e.g., tar|upload|untar or similar. ---
**Author:** ![ashanbrown](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ashanbrown/32/8944_2.png) [@ashanbrown](https://discuss.circleci.com/u/ashanbrown)\ **Post date:** [August 31, 2017, 5:25am UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/2 "2017-08-31T05:25:00Z")
I’m also seeing this . We generate a 16MB tarball that takes 1 second to upload this to s3. We upload both this tarball and the 830 files in it to the artifacts and the entire process takes 30 seconds. ---
**Author:** ![cans](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/cans/32/10377_2.png) [@cans](https://discuss.circleci.com/u/cans)\ **Post date:** [January 10, 2018, 11:16am UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/3 "2018-01-10T11:16:46Z")
Same here, 4 mins to upload coverage reports is quite unacceptable. Would you have ways to go around this ? Building an archive with the files isn’t good enough: it renders the files too cumbersome to access. Thanks. ---
**Author:** ![yoz](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/yoz/32/18000_2.png) [@yoz](https://discuss.circleci.com/u/yoz)\ **Post date:** [March 15, 2018, 9:23pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/4 "2018-03-15T21:23:18Z")
I agree, the “Collecting artifacts” step is taking 2.5 minutes out of our 18-minute build (almost 15% of the total time!) and it’s a significant factor that slows down our development cycle. “Collecting 5,788 build artifacts” (coverage reports) appears to result in 5788 calls to curl and 5788 separate connections. There’s also a lot of spam in the logs for this step: /usr/bin/curl: /usr/local/lib/libcurl.so.4: no version information available (required by /usr/bin/curl) ---
**Author:** ![system](https://global.discourse-cdn.com/circleci/original/3X/5/1/511608acda8e85906e15909b57bd0a46c1dbe291.png) [@system](https://discuss.circleci.com/u/system)\ **Post date:** [June 18, 2018, 11:34am UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/5 "2018-06-18T11:34:54Z")
---
**Author:** ![drazisil](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/drazisil/32/24029_2.png) [@drazisil](https://discuss.circleci.com/u/drazisil)\ **Post date:** [February 12, 2019, 7:50pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/6 "2019-02-12T19:50:24Z")
---
**Author:** ![drazisil](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/drazisil/32/24029_2.png) [@drazisil](https://discuss.circleci.com/u/drazisil)\ **Post date:** [February 12, 2019, 7:58pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/7 "2019-02-12T19:58:53Z")
Sorry for the radio silence here. This post is quite old, so we were not aware this was still an issue. If you are still seeing this can you please [open a ticket with a build URL](https://support.circleci.com/hc/en-us/requests/new) so we can investigate? ---
**Author:** ![system](https://global.discourse-cdn.com/circleci/original/3X/5/1/511608acda8e85906e15909b57bd0a46c1dbe291.png) [@system](https://discuss.circleci.com/u/system)\ **Post date:** [February 22, 2019, 8:05pm UTC](https://discuss.circleci.com/t/uploading-artifacts-recursively-is-exceptionally-slow/14558/8 "2019-02-22T20:05:00Z")
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.