CI failure due to - Too many uploads to this commit

Support
#1

Description

“Too many uploads to this commit.” error is received when trying to upload test logs for multiple services.

Commit SHAs

d6ab03c5770f83f0d9df6a9cc9e2dbe8bdd20eeb

CI/CD or Build URL

Using CircleCI on a private repository.

Uploader

Using latest version of bash uploader.

Codecov Output

==> git version 2.20.1 found
==> curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
==> Circle CI detected.
project root: /root/project
→ token set from env
Yaml found at: codecov.yml
→ Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
+ TestResults/47d12bd9-ed9d-4a5a-9f8c-bad424fbc990/coverage.cobertura.xml bytes=1273937
==> Appending adjustments
https://docs.codecov.io/docs/fixing-reports
→ No adjustments found
==> Gzipping contents
112K /tmp/codecov.TeAi8N.gz
==> Uploading reports
url: https://codecov.io
query: branch=<branch_name>&commit=d6ab03c5770f83f0d9df6a9cc9e2dbe8bdd20eeb&build=891366&build_url=&name=&tag=&slug=<our_org>%2F<our_repo>>&service=circleci&flags=<service_flag>&pr=2782&job=0&cmd_args=Z,f,F
→ Pinging Codecov
https://codecov.io/upload/v4?package=bash-20210129-7c25fce&token=secret&branch=<branch_name>&commit=d6ab03c5770f83f0d9df6a9cc9e2dbe8bdd20eeb&build=891366&build_url=&name=&tag=&slug=<our_org>%2F<our_repo>>&service=circleci&flags=<service_flag>&pr=2782&job=0&cmd_args=Z,f,F
[ErrorDetail(string=‘Too many uploads to this commit.’, code=‘invalid’)]
400

Expected Results

All services to upload their output without any issues.

Actual Results

CI fails due to Codecov failure exit status.

Too many uploads to this commit
#2

FYI, I already opened a ticket here for same issue with GitHub Actions Workflow : Too many uploads to this commit

#3

Hi @lcharette, we have rolled out a fix for this issue. Let me know if you are still experiencing it.

#4

Still there 20 minutes ago. For some runs, the message has changed to:

{'detail': ErrorDetail(string='Actions workflow run is stale', code='not_found')}

#5

Got the Actions workflow run is stale on one repo and the too many upload on another one, but all other repos (~6-8 of them) I re-run the build and they where fine.

See here :

#6

Still failing here. Most uploads fail with ‘Actions workflow run is stale’.

#7

@rollbear @lcharette

We have a 100-upload limit per commit. If you are hitting the too many upload issue, that is why. I don’t have a solution that works right now, but hopefully we can figure something out that will work for you.

If you are getting the Actions workflow run is stale, and your workflow doesn’t take more than 60 minutes, then please link. If it takes more than 60 minutes, let me know how long it takes. A workaround for now is to add the upload token.

#8

Excellent. Please look at Merge pull request #227 from Eppendorf-SW/fix/gcc_9_10_nonull_warning · rollbear/trompeloeil@714e8af · GitHub

You may note that there is a total of 68 jobs (not all of them upload coverage.) 68 < 100.
You may also note that the total run time of all 68 jobs is 11m50s. 11m50s < 1h.

The 1st and 7th failure were “too many uploads”. Those in between were “stale handle”. There were successful uploads before, in between, and after.

It also used to work flawlessly for quite some time, until you changed something on your end.

#9

@rollbear, I pushed a change here to help identify what’s going on that may also include a fix. Are you still seeing the Actions workflow run is stale issue?

#10

Retriggered the build, and it’s still going, but there are so many failures (15 so far.) Most are “too many uploads”, but I’ve seen at least one “stale”.

[edit:] it stopped at 15 failures.

#11

I don’t know if you’ve actually fixed this, or if I’ve been lucky today, but I’ve just had two consecutive builds pass without upload problems.

#12

@rollbear we’ve made some changes last night, but it is a little difficult to ensure the quality of the fix over the weekend (whether we are hitting all edge cases). If the fix is working for you, that’s great! I except it to continue to do so. I’ll mark this as the solution next week if it continues to be fine for you.