Coverage report not showing any "green" lines (executed code) on app.codecov.io but works on codecov.io

Description

A file coverage report on the new app.codecov.io is not not showing any “green” lines, i.e., not marking any lines as “code that is being covered”, despite there being hundreds; it works in another view of the same data on the old codecov.io

Commit SHAs

E.g. ff4358f9ab0d01e8e194b2651d62ee6bbd985fed

Repository

CI/CD or Build URL

Uploader

We are using the bash uploader, i.e., https://codecov.io/bash

Codecov Output

If you really need this, I guess we can extract it from the GitHub Action CI logs above (but there are like … 18 or so ?)

Expected Results

Code which is executed should be marked with green lines; code not executed with red lines; non-code stays white. As is the case on e.g. https://codecov.io/gh/gap-system/gap/src/ff4358f9ab0d01e8e194b2651d62ee6bbd985fed/src/io.c

Actual Results

Looking at Codecov there are no green lines, only a couple red ones – however, that code is really being executed: For starters in a few places like line 498 there are these little green circles with a hover tool tip saying “this line appeared in 4 coverage reports”, despite that line being shown as white. Also if I directly go to the same page but now for the commit, not the PR (i.e. Code coverage done right.) then I do get those missing green lines. Note how that URL is on codecov.io while the broken one is on app.codecov.io

Additional Information

None.

Hmmm @fingolfin, I think something strange happened through your CI.

These are the commits for that PR. You’ll notice the most recent commit is only showing 4 builds. It also is showing that this commits was deleted or rebased on. For reference, a previous commit had 8 uploads to Codecov.

Hmm, actually, the “most recent” commit was not most recent, so I ignored it, considering it yet another fluke in the codecov UI (to which I’ve become quite used over the years, so I don’t really think about them anymore). And yeah, something weird is going on, because a full CI will upload more like 15 build reports, see e.g. Code coverage done right.

But whether it is 4, 8 or 15 reports: isn’t it still weird that Codecov shows no code executed code lines? Something is really weird here.

BTW, I see many oddities with codecov all the time. E.g on another PR it reports a massive coverage drop that I can’t explain at all. I know that these things often are fixed by re-running the CI, though. I tried to see if Codecov there works better, but it just shows me a spin wheel “forever”

Anyway, I’ve just rebased that PR and triggered a full rebuild for it this way, let’s wait and see if that helps.

Latest commit (the automatic merge commit created by GitHub) is this one and it has 14 build results uploaded as I write this. But the PR specific file view of io.c appears to still be bogus, albeit in a different way (it now has both red and green lines, but they cover a seemingly random subset of lines). While the commit specific file view of io.c seems fine (well, modulo the issue I describe in another post).

I note that one is on the app. subdomain of codecov, the other on the top domain, so maybe this is a matter of the new versus old website code?

I just observed a similar problem on a completely different project, even using a different programming language (Julia). The CI report for this GitHub pull request reports increased coverage. Looking at the PR specific file coverage report for mpoly.jl (or fo I also only see

  • white lines
  • red lines
  • lines that changed from red to green
  • lines marked with a green circle but otherwise white (but only a few of them)

But no purely green lines, even though I know they must be there; and indeed, the lines with the green circles should be green (but also many more). This screen shot might illustrate the issue. Note that this is on app.codecov.io once again.

I then went to the commit status page on Codecov, and searched for the relevant file there and ended up at this commit specific file coverage report for mpoly.jl – note that it is on codecov.io, the old site. Here is a screenshot from the same function; it looks reasonable here.

Note how there are green lines; and more green circles.

My impression thus is quite strongly that this is a bug on the “new” site.