Codecov randomly drops coverage

Description

Well, I have no idea why, but Codecov randomly drops coverage despite passing build on Travis. List of recent commits:

https://codecov.io/gh/dzikoysk/reposilite/branch/master/commits

Repository

CI/CD

Uploader

Repeated bash (it does not affect this issue, the problem was that codecov sometimes just didnt respond immediately)

Commit SHAs

7992f0f0f6448d93e66922d126319b5daef5269a

Codecov YAML

Codecov Output

A little dirty due to missing formatting of CLI colors

e[0K$ bash <(curl -s https://codecov.io/bash) || (sleep 5 && bash <(curl -s https://codecov.io/bash)) || (sleep 5 && bash <(curl -s https://codecov.io/bash)) || (sleep 5 && bash <(curl -s https://codecov.io/bash)) || (sleep 5 && bash <(curl -s https://codecov.io/bash))

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20200629-ffaf297


e[0;90m==>e[0m Travis CI detected.
    e[0;90mproject root:e[0m .
    e[0;90mYaml found at:e[0m codecov.yml
e[0;90m==>e[0m Running gcov in . e[0;90m(disable via -X gcov)e[0m
e[0;90m==>e[0m Python coveragepy not found
e[0;90m==>e[0m Searching for coverage reports in:
    e[0;32m+e[0m .
    e[0;90m->e[0m Found 1 reports
e[0;90m==>e[0m Detecting git/mercurial file structure
e[0;90m==>e[0m Appending build variables
    e[0;32m+e[0m TRAVIS_OS_NAME
    e[0;32m+e[0m openjdk8
e[0;90m==>e[0m Reading reports
    e[0;32m+e[0m ./reposilite-backend/target/site/jacoco/jacoco.xml e[0;90mbytes=201666e[0m
e[0;90m==>e[0m Appending adjustments
    e[0;36mhttps://docs.codecov.io/docs/fixing-reportse[0m
    e[0;90m->e[0m No adjustments found
e[0;90m==>e[0m Gzipping contents
e[0;90m==>e[0m Uploading reports
    e[0;90murl:e[0m https://codecov.io
    e[0;90mquery:e[0m branch=master&commit=7992f0f0f6448d93e66922d126319b5daef5269a&build=292.1&build_url=&name=&tag=&slug=dzikoysk%2Freposilite&service=travis&flags=&pr=false&job=355907982
e[0;90m->e[0m  Pinging Codecov
https://codecov.io/upload/v4?package=bash-20200629-ffaf297&token=secret&branch=master&commit=7992f0f0f6448d93e66922d126319b5daef5269a&build=292.1&build_url=&name=&tag=&slug=dzikoysk%2Freposilite&service=travis&flags=&pr=false&job=355907982
e[0;90m->e[0m  Uploading to
https://storage.googleapis.com/codecov/v4/raw/2020-06-30/EDB243A5518D3BC8C6A33D4BE0022526/7992f0f0f6448d93e66922d126319b5daef5269a/74b28494-172a-4d9d-a6b8-0d3d6b2df672.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20200630%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20200630T160456Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=88b79b945c45c9e85e8f9c0a0b746a632e11f721cfe5f23b0a34876f15efc381
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 13751    0     0  100 13751      0  52269 --:--:-- --:--:-- --:--:-- 52484
    e[0;32m->e[0m View reports at e[0;36mhttps://codecov.io/github/dzikoysk/reposilite/commit/7992f0f0f6448d93e66922d126319b5daef5269ae[0m
travis_time:end:1b689a96:start=1593533095527380243,finish=1593533096775554638,duration=1248174395,event=after_success
e[0Ktravis_fold:end:after_success
e[0K
Done. Your build exited with 0.

Hi @dzikoysk, doing a little digging, I compared your raw coverage reports for a covered commit and an uncovered commit

If you open up the builds and diff the xml, you’ll notice that they are showing up as uncovered in the 0% commit.

uncovered:

         <method name="&lt;init&gt;" desc="(Lorg/panda_lang/reposilite/Reposilite;)V" line="45">
            <counter type="INSTRUCTION" missed="22" covered="0" />
            <counter type="LINE" missed="7" covered="0" />
            <counter type="COMPLEXITY" missed="1" covered="0" />
            <counter type="METHOD" missed="1" covered="0" />
         </method>

covered:

<class name="org/panda_lang/reposilite/repository/DeployController" sourcefilename="DeployController.java">
         <method name="&lt;init&gt;" desc="(Lorg/panda_lang/reposilite/Reposilite;)V" line="45">
            <counter type="INSTRUCTION" missed="0" covered="22" />
            <counter type="LINE" missed="0" covered="7" />
            <counter type="COMPLEXITY" missed="0" covered="1" />
            <counter type="METHOD" missed="0" covered="1" />
         </method>

The only other difference I could find was the presense of xml tags like <sessioninfo id="localhost-7eefd00a" start="1593510762671" dump="1593510782093"/>

I think there might be something going wrong with the way you are collecting coverage reports.

1 Like

Thanks for a guide @tom :heart: I’ve reported this issue to JaCoCo team and it looks like I’ve experienced bug in jdk, I’m leaving it for the record:

I hope that it might be helpful for someone in the future.

1 Like

Wow incredible @dzikoysk. Glad that you were able to get it resolved!

1 Like