PRs not registering in codecov/PR status

Support
#1

Description

PRs aren’t showing up correctly on codecov, even though the bash upload from travis-ci completes fine, and the PR status is not being set.

Repository

https://github.com/ljharb/es-abstract/pull/95 (merged)
https://github.com/ljharb/es-abstract/pull/93 (unmerged)
https://github.com/ljharb/es-abstract/pull/94 (will be merged soon)

CI/CD

travis-ci

Uploader

bash uploader, it worked with no changes a few months back

Commit SHAs

8a05dc9aeba1d1f603d986dc6ac43f81f16d8c52

Codecov YAML

Please include the codecov.yml as code

Codecov Output

No output visible.

GitHub public repository issues
#2

I have the same issue, and it’s been like this for some time now. See e.g. this PR https://github.com/gap-system/gap/pull/4028 – we are using the Codecov app on GitHub (and have been for a long time).

That said, https://codecov.io/gh/gap-system/gap/pull/4028 exists – but it doesn’t load. That’s an issue I first reported a couple years ago, but I’ve given up hope for it to be resolved (I was told by Codecov staff that we produce too much coverage data, apparently the Codecov tooling can’t deal with that).

#3

@ljharb are you using the GitHub marketplace app? There seems to be a permissions issue, which is why you weren’t getting statuses.

#4

@fingolfin, digging into your issue though it is unrelated. Would you be able to create a separate thread?

#5

I’m also using the GitHub marketplace app (just reinstalled it to the organization today), and it doesn’t seem to be updating commit statuses. It is able to post the coverage comment just fine, but it is not able to add a failing commit status when coverage decreases.

Example PR: https://github.com/liatrio/github-azure-demo/pull/5

#6

Hi @mrparkers, you have some errors in your yaml

codecov:
  require_ci_to_pass: true

coverage:
  precision: 2
  round: down
  range: "25...100"
  status:
    project:
      default:
        target: auto
        threshold: 0%
        base: auto
        paths:
          - "src/client"
          - "src/Conduit"

parsers:
  gcov:
    branch_detection:
      conditional: yes
      loop: yes
      method: no
      macro: no

comment:
  layout: "reach,diff,flags,tree"
  behavior: default
  require_changes: false
  branches:
    - master
#7

Thank you for the quick response! I updated my yaml file with the fixes you suggested but I’m still not receiving failing commit statuses.

#8

@mrparkers, this is really strange. I’m seeing us send over status checks in our logs. Could you re-run your workflow?

#9

Hey @tom, I re-ran the workflow. I saw that the comment was edited, but I’m still not seeing the commit statuses.

#10

It doesn’t appear to work on PRs against my personal account (not an organization) either: https://github.com/mrparkers/github-azure-demo/pull/2

#11

@mrparkers, not sure what’s happening here, but I’m seeing the status checks now. Would you be able to run one more time?

#12

Another update (sorry for the spam), the status checks never showed up, but they appeared as valid options in the branch protection rules, so I made them required and they’re showing up now.

However, now my other statuses are broken :frowning:

This is starting to look like an issue on GitHub’s end.

I’ll go ahead and re-run the workflow.

1 Like
#13

@mrparkers, I’m seeing some pretty strange behavior. All the status checks are getting sent to a merge commit. https://codecov.io/gh/liatrio/github-azure-demo/pull/11/commits. Is this coming from you?

#14

@mrparkers linking to the GitHub checks, I see they believe it to be a merge commit as well: https://github.com/liatrio/github-azure-demo/pull/11/checks?check_run_id=718109863

image
image1608×1068 151 KB

#15

I’m also pretty confused. My other checks are coming from GitHub actions, which test / lint some JavaScript and C# code.

When I look at the branch protection rules for master, all of the checks (my GitHub actions and CodeCov) are there.

When I require that my GitHub actions checks pass, they show up in the PR, but the CodeCov checks do now.

When I require the CodeCov checks, they show up, but my GitHub actions checks do not.

When I require both, only the CodeCov checks show up.

I am not sure why those checks are being assigned to a merge commit, but regardless, this is really looking like an issue on GitHub’s end. No combination I try allows me to use all of the checks.

I appreciate your help, I’m going to attempt to escalate this with our GitHub support rep.

1 Like
#16

Yes that is really strange, running git log locally does not show this as a merge commit. The shas are different as well. I’ll attempt to look into this too.

1 Like
#17

This could be the culprit: https://github.com/actions/checkout/issues/237

EDIT: Hopefully the final update - I was able to fix this by using actions/checkout@v1. I was previously using actions/checkout@master. https://github.com/liatrio/github-azure-demo/pull/12

1 Like
#18

@tom sorry for missing this earlier. yes, i’m using the github app, but i haven’t changed anything about this repo or the app being installed on my user account. Must i completely uninstall and reinstall it to fix the issues, or is there a URL i can use to force regathering permissions?

#19

@ljharb, could you try logging out of codecov.io, clearing those cookies, and re-logging back in? That might be enough.

#20

I’ve solved it in all our repos in Github. We have about 54 repos, all using Codecov installed as a Github app in our org, and we also use Travis CI.

We used https://www.npmjs.com/package/codecov version 3.7.0 (latest) and that showed in the build log that the report was sent after a successful Travis CI build, and gave a link. However, the link showed there was an error (but couldn’t give anything more). The codecov status in our PRs remained unchanged (‘waiting’).

Switching the use of this npm module to the Bash uploader https://docs.codecov.io/docs/about-the-codecov-bash-uploader fixed everything. Now all codecov statuses are reported properly.

I was concerned that this would not work in our our Windows builds in our Travis CI build matrix because it was a bash script, but Travis CI uses git-bash on Windows so it’s all good.

1 Like