Following the recent security issue with the bash uploader, I wanted to switch to using the node package instead of fetching+executing the bash script like before.
And I was surprised that the result in codecov is notably different.
In our project we’re using jest with these options for the coverage system:
This looks pretty standard to me. As a result Jest generates 3 files:
- lcov-report/ (directory)
That’s where I noticed a difference, when looking at the CI log:
- the bash script uploads all 3
- the node script seems to upload only
Could that make a difference ? It’s not clear to me what different informations all these files are, until then I supposed they were containing the same information really.
Some more information:
- here is the PR switching to this node module: Use the codecov module instead of downloading a bash script in CI by julienw · Pull Request #3281 · firefox-devtools/profiler · GitHub
- This is the codecov report for this switch: Codecov (I believe this is available without being logged in)
Codecov’s UI doesn’t make it any easier to analyze…
For example, see this file: Codecov
+10% as coverage, but the 2 lines that are shown are red as “not covered”; they were already not covered previously.
But when looking closer:
- before: https://codecov.io/gh/firefox-devtools/profiler/src/be3a6378ef3eab3b8db1c5a8351f25b9a85e4330/src/components/shared/PanelSearch.js
- after: https://codecov.io/gh/firefox-devtools/profiler/src/1d036c0680d83a837e480eeae6211e2c9cdb4437/src/components/shared/PanelSearch.js
we see that 2 lines (42 and 43) were previously shown as “uncovered” and now they are just blank.
Another example is: Codecov. Again the UI is quite bad… let’s look at the files directly:
- before: https://codecov.io/gh/firefox-devtools/profiler/src/be3a6378ef3eab3b8db1c5a8351f25b9a85e4330/src/components/shared/FilterNavigatorBar.js
- after: https://codecov.io/gh/firefox-devtools/profiler/src/1d036c0680d83a837e480eeae6211e2c9cdb4437/src/components/shared/FilterNavigatorBar.js
We clearly see a lot of the case disjunctions in the JSX code are missing in the new coverage.
I believe the “previous” information is correct in these 2 cases but haven’t looked at every cases.
All that to say that the behavior is different. Which might be OK I guess I mostly need some guidance from you experts on how to get the same behavior as with the bash script. Happy to help with more data if needed!