Codecov posts two comments (reports) on every PR

For every new PR, Codecov posts the same comment (report) twice. This happens every time.

@jerrod Can you please take a look at this? Thank you! :pray:

Same issue as these:


Screenshot:

Thanks Danny for the well written ticket and referencing past tickets. Digging into this for you.

@DannyNemer can you provide your GH username? If public, can you provide a link to the repo and an example PR?

@DannyNemer I think I found you. In this case you had a Github App connection AND a team bot set so both accounts were posting comments. I’ve reset the teambot for you and left the Github App. Can you check if that’s working?

Separately, I’ll clean up to the docs to clarify this in the team bot page.

Thank you, @jerrod. This issue has since worsened to CodeCov sometimes posting three comments on a PR. Today, this issue also started to occur on our second repo as well.

My GitHub username: DannyNemer
The organization’s name (that owns these repos): afresh-technologies

These are all private repositories, so I can not send you a private PR. But, let me know if you would like more screenshots.

Thank you! I’ll take a look.

Thanks @DannyNemer, started drafting an FAQ related to this: https://docs.codecov.io/docs/team-bot#section-frequently-asked-questions

Let’s see if this initial database refresh of team bot fixes your issue. If not, will need to submit as a bug.

Either way, we will do a better job of surfacing when this may be in issue in the UI.

@jerrod Everything seems to be fixed. I’ll let you know if this regresses (similar to how one of our repositories suddenly started experiencing this issue much later than another repository). Thank you!

@jerrod The bug is back. For context: this occured when locally rebasing a branch with master and then force-pusshing to the branch. Perhaps Codecov does not recognize them as part of the same branch because each begins with a different SHA? (Just speculating.)

Hey @DannyNemer, thanks for message, sorry still a bug.

So to clarify:

  1. You were seeing double (or triple) Codecov comments on all PRs
  2. I reset your team bot
  3. You are now only seeing double comments in use cases in which you are rebasing + force-pushing?

Or are you seeing the double on all PRs again? Do you use a rebase + force-push workflow on all PRs?

I can not discern (without taking time to test) whether or not this is unique to rebasing + force-pushing because we all on the team do it many times per PR (whenever updating with master).

Internal note: another data point here https://github.com/storyscript/cli/pull/227

@DannyNemer what’s the repo name this is coming from?

These errors occur in our private repositories.

Also, please ignore my previous comment about force-pushing - it is unrelated. The double-comments from Codecov occur on every PR immediately after opening the PR.

Hi @DannyNemer

Helping Jerrod here. Are you using a codecov.yml file to configure notifications, or the defaults? Can you double-check the settings for the repo and org on Codecov to make sure a team bot didn’t somehow get set again?

Nope

Where do I find this in settings?

https://codecov.io/account/gh/<org>/yaml near the bottom it will tell you if you are using a bot account

Hi @DannyNemer

Were you able to check if you had a bot account set?

Sorry for the late reply. Yes, it says we are using a bot account.

The two-post bug continues and occurs every for every PR. Sometimes, there are three Codecov posts on a PR (the third coming later in the age of the PR).

This double-post bug grew so annoying that we disabled Codecov for about a month. But, we are trying one more time. Our integration has few configurations; we merely enabled Codecov via the GitHub marketplace and added two private repos.

Our CI uses CircleCI’s Codecov Orb, and it uploads a code coverage report for each test job (each tagged differently). I.e., each CI run (for a PR) makes 4 uploads to CircleCI. I believe this is standard configuration. Could that be a culprit here?

We just had a PR where Codecov posted 4 times - that’s quite a bit of scrolling to get through

I hear you @DannyNemer, our new sprint starts Thursday and really doubling-down (awful pun not intended) on getting this fixed.

For what it’s worth, we dog food Codecov every day internally and we are also seeing this bug.

Thanks for contributing the CircleCI Orbs concept as possible cause, I’ve added as a thesis to prove/disprove (have already disproved team bot as culprit).

Jerrod