Coverage not accurately reported

adamtheturtle · August 18, 2020, 8:14pm

Description

I have a repository with >40 jobs in the main CI workflow.
I am using GitHub Actions.
I am using the fail_ci_if_error option.

Sometimes I see that there is missing coverage reported by Codecov for code that I know for sure is covered.

On a particular run ( Bump codecov and make CI fail if it fails by adamtheturtle · Pull Request #690 · VWS-Python/vws-python-mock · GitHub) the code coverage is reported as 99.25%.

The “missing” coverage is code which I am certain is covered - it even includes tests which I can see were run.

Repository

CI/CD

GitHub Actions.

Uploader

Commit SHAs

143c1ffd35af34b7c14fc3817ddf1e25f31b82f3

Codecov YAML

coverage:
  status:
    patch:
      default:
        # Require 100% test coverage.
        target: 100%

Codecov Output

From a particular job which I believe has not been counted by Codecov:

https://pipelines.actions.githubusercontent.com/24aMLzmFiVrnOAZbDJVsjvzPJ0pTXAUSmss8BIAcQXv2PpCi9Y/_apis/pipelines/1/runs/518/signedlogcontent/18?urlExpires=2020-08-18T20%3A13%3A45.0179966Z&urlSigningMethod=HMACV1&urlSignature=B1ikGK047KcthYnVF4yMpocUrwvIA9I5jsIrhIDZWBU%3D

2020-08-18T08:12:17.4591510Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[Real Vuforia-extra_headers0] PASSED
2020-08-18T08:12:17.8542687Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[Real Vuforia-extra_headers1] PASSED
2020-08-18T08:12:17.9022981Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[In Memory Mock Vuforia-extra_headers0] PASSED
2020-08-18T08:12:17.9163753Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[In Memory Mock Vuforia-extra_headers1] PASSED
2020-08-18T08:12:18.5692709Z tests/mock_vws/test_query.py::TestAcceptHeader::test_invalid[Real Vuforia] PASSED
2020-08-18T08:12:19.5470067Z tests/mock_vws/test_query.py::TestAcceptHeader::test_invalid[In Memory Mock Vuforia] PASSED
2020-08-18T08:12:19.5470422Z 
2020-08-18T08:12:19.5471172Z ----------- coverage: platform linux, python 3.8.5-final-0 -----------
2020-08-18T08:12:19.5471334Z Coverage XML written to file coverage.xml
2020-08-18T08:12:19.5471420Z 
2020-08-18T08:12:19.5471537Z ===Flaky Test Report===
2020-08-18T08:12:19.5471601Z 
2020-08-18T08:12:19.5471926Z test_valid[Real Vuforia-extra_headers0] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472256Z test_valid[Real Vuforia-extra_headers1] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472592Z test_valid[In Memory Mock Vuforia-extra_headers0] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472927Z test_valid[In Memory Mock Vuforia-extra_headers1] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473087Z test_invalid[Real Vuforia] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473239Z test_invalid[In Memory Mock Vuforia] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473324Z 
2020-08-18T08:12:19.5473443Z ===End Flaky Test Report===
2020-08-18T08:12:19.5473522Z 
2020-08-18T08:12:19.5473657Z ============================== 6 passed in 3.45s ===============================
2020-08-18T08:12:19.6089999Z ##[group]Run codecov/codecov-action@v1.0.13
2020-08-18T08:12:19.6090200Z with:
2020-08-18T08:12:19.6090326Z   fail_ci_if_error: true
2020-08-18T08:12:19.6090444Z env:
2020-08-18T08:12:19.6090571Z   pythonLocation: /opt/hostedtoolcache/Python/3.8.5/x64
2020-08-18T08:12:19.6090691Z ##[endgroup]
2020-08-18T08:12:20.9660431Z [command]bash codecov.sh -n  -F  -Z
2020-08-18T08:12:20.9755937Z 
2020-08-18T08:12:20.9756627Z   _____          _
2020-08-18T08:12:20.9757017Z  / ____|        | |
2020-08-18T08:12:20.9757474Z | |     ___   __| | ___  ___ _____   __
2020-08-18T08:12:20.9757943Z | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
2020-08-18T08:12:20.9758152Z | |___| (_) | (_| |  __/ (_| (_) \ V /
2020-08-18T08:12:20.9758286Z  \_____\___/ \__,_|\___|\___\___/ \_/
2020-08-18T08:12:20.9759376Z                               Bash-20200728-9fb7d93
2020-08-18T08:12:20.9759483Z 
2020-08-18T08:12:20.9759549Z 
2020-08-18T08:12:20.9773233Z e[0;90m==>e[0m GitHub Actions detected.
2020-08-18T08:12:20.9875385Z     e[0;90mproject root:e[0m .
2020-08-18T08:12:20.9980662Z     e[0;90mYaml found at:e[0m codecov.yaml
2020-08-18T08:12:21.0473837Z e[0;90m==>e[0m Running gcov in . e[0;90m(disable via -X gcov)e[0m
2020-08-18T08:12:21.0519429Z e[0;90m==>e[0m Searching for coverage reports in:
2020-08-18T08:12:21.0521115Z     e[0;32m+e[0m .
2020-08-18T08:12:21.0647069Z     e[0;90m->e[0m Found 2 reports
2020-08-18T08:12:21.0649805Z e[0;90m==>e[0m Detecting git/mercurial file structure
2020-08-18T08:12:21.0775170Z e[0;90m==>e[0m Reading reports
2020-08-18T08:12:21.0837834Z     e[0;32m+e[0m ./coverage.xml e[0;90mbytes=175783e[0m
2020-08-18T08:12:21.0933147Z     e[0;32m+e[0m ./codecov.yaml e[0;90mbytes=105e[0m
2020-08-18T08:12:21.0964327Z e[0;90m==>e[0m Appending adjustments
2020-08-18T08:12:21.0965193Z     e[0;36mhttps://docs.codecov.io/docs/fixing-reportse[0m
2020-08-18T08:12:21.1157149Z     e[0;90m->e[0m No adjustments found
2020-08-18T08:12:21.1157959Z e[0;90m==>e[0m Gzipping contents
2020-08-18T08:12:21.1355709Z e[0;90m==>e[0m Uploading reports
2020-08-18T08:12:21.1357239Z     e[0;90murl:e[0m https://codecov.io
2020-08-18T08:12:21.1358311Z     e[0;90mquery:e[0m branch=bump-codecov&commit=f285876ef885f1e85d8e897c84a8f9039815aff3&build=213293561&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F213293561&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=690&job=&cmd_args=n,F,Z
2020-08-18T08:12:21.1399529Z e[0;90m->e[0m  Pinging Codecov
2020-08-18T08:12:21.1400914Z https://codecov.io/upload/v4?package=bash-20200728-9fb7d93&token=secret&branch=bump-codecov&commit=f285876ef885f1e85d8e897c84a8f9039815aff3&build=213293561&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F213293561&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=690&job=&cmd_args=n,F,Z
2020-08-18T08:12:44.4056522Z e[0;90m->e[0m  Uploading to
2020-08-18T08:12:44.4058183Z https://storage.googleapis.com/codecov/v4/raw/2020-08-18/875BAD842DFC1254D7E8EE01DC26615F/f285876ef885f1e85d8e897c84a8f9039815aff3/3237d7b7-3f52-44bc-9e4b-78c21c0b8d21.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20200818%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20200818T081244Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=5c9b4d0317f3f73867751c738f30f0cad27750e3666810e0e075616d94ea9b64
2020-08-18T08:12:44.4186032Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2020-08-18T08:12:44.4186705Z                                  Dload  Upload   Total   Spent    Left  Speed
2020-08-18T08:12:44.4187055Z 
2020-08-18T08:12:44.6083483Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2020-08-18T08:12:44.6084948Z 100 13468    0     0  100 13468      0  69422 --:--:-- --:--:-- --:--:-- 69422
2020-08-18T08:12:44.6162467Z     e[0;32m->e[0m View reports at e[0;36mhttps://codecov.io/github/VWS-Python/vws-python-mock/commit/f285876ef885f1e85d8e897c84a8f9039815aff3e[0m
2020-08-18T08:12:44.6279235Z Post job cleanup.

Steps to Reproduce

I just use the codecov action on a workflow with many jobs and this happens sometimes.

Expected behavior:

I see 100% coverage.

Actual behavior:

I see 99.25% coverage.

Flakiness?

Sometimes.

Additional Information

N/A

tom · August 19, 2020, 2:35am

@adamtheturtle, I went through all 42 builds here. In particular, I was checking line 25 of mock_vws/_query_validators/accept_header_validators.py, but they all show it not being covered. This implies that the coverage reports being uploaded to Codecov do not run that particular line.

Due to the sheer amount of builds, I wasn’t able to match it up to the build in GitHub. What is the name of the test that runs that line?

adamtheturtle · August 19, 2020, 4:24pm

Hi @tom,

Wow and thank you for the investigation.
If you need me to dig into that line, I can, but I think that I have a clearer example:

Line 947 of test_query.py is shown as not covered for commit 143c1ffd35af34b7c14fc3817ddf1e25f31b82f3.

This line is the first line of tests/mock_vws/test_query.py::TestAcceptHeader::test_valid.

I believe that if this line is not run, that means the test is not run.

The GitHub Action job which runs that test is at Bump codecov and make CI fail if it fails by adamtheturtle · Pull Request #690 · VWS-Python/vws-python-mock · GitHub.
We can see from the logs that the test is run:

2020-08-18T08:12:17.4591510Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[Real Vuforia-extra_headers0] PASSED
2020-08-18T08:12:17.8542687Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[Real Vuforia-extra_headers1] PASSED
2020-08-18T08:12:17.9022981Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[In Memory Mock Vuforia-extra_headers0] PASSED
2020-08-18T08:12:17.9163753Z tests/mock_vws/test_query.py::TestAcceptHeader::test_valid[In Memory Mock Vuforia-extra_headers1] PASSED
2020-08-18T08:12:18.5692709Z tests/mock_vws/test_query.py::TestAcceptHeader::test_invalid[Real Vuforia] PASSED
2020-08-18T08:12:19.5470067Z tests/mock_vws/test_query.py::TestAcceptHeader::test_invalid[In Memory Mock Vuforia] PASSED
2020-08-18T08:12:19.5470422Z 
2020-08-18T08:12:19.5471172Z ----------- coverage: platform linux, python 3.8.5-final-0 -----------
2020-08-18T08:12:19.5471334Z Coverage XML written to file coverage.xml
2020-08-18T08:12:19.5471420Z 
2020-08-18T08:12:19.5471537Z ===Flaky Test Report===
2020-08-18T08:12:19.5471601Z 
2020-08-18T08:12:19.5471926Z test_valid[Real Vuforia-extra_headers0] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472256Z test_valid[Real Vuforia-extra_headers1] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472592Z test_valid[In Memory Mock Vuforia-extra_headers0] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5472927Z test_valid[In Memory Mock Vuforia-extra_headers1] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473087Z test_invalid[Real Vuforia] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473239Z test_invalid[In Memory Mock Vuforia] passed 1 out of the required 1 times. Success!
2020-08-18T08:12:19.5473324Z 
2020-08-18T08:12:19.5473443Z ===End Flaky Test Report===
2020-08-18T08:12:19.5473522Z 
2020-08-18T08:12:19.5473657Z ============================== 6 passed in 3.45s ===============================
2020-08-18T08:12:19.6089999Z ##[group]Run codecov/codecov-action@v1.0.13
2020-08-18T08:12:19.6090200Z with:
2020-08-18T08:12:19.6090326Z   fail_ci_if_error: true
2020-08-18T08:12:19.6090444Z env:
2020-08-18T08:12:19.6090571Z   pythonLocation: /opt/hostedtoolcache/Python/3.8.5/x64
2020-08-18T08:12:19.6090691Z ##[endgroup]
2020-08-18T08:12:20.9660431Z [command]bash codecov.sh -n  -F  -Z
2020-08-18T08:12:20.9755937Z 
2020-08-18T08:12:20.9756627Z   _____          _
2020-08-18T08:12:20.9757017Z  / ____|        | |
2020-08-18T08:12:20.9757474Z | |     ___   __| | ___  ___ _____   __
2020-08-18T08:12:20.9757943Z | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
2020-08-18T08:12:20.9758152Z | |___| (_) | (_| |  __/ (_| (_) \ V /
2020-08-18T08:12:20.9758286Z  \_____\___/ \__,_|\___|\___\___/ \_/
2020-08-18T08:12:20.9759376Z                               Bash-20200728-9fb7d93
2020-08-18T08:12:20.9759483Z 
2020-08-18T08:12:20.9759549Z 
2020-08-18T08:12:20.9773233Z e[0;90m==>e[0m GitHub Actions detected.
2020-08-18T08:12:20.9875385Z     e[0;90mproject root:e[0m .
2020-08-18T08:12:20.9980662Z     e[0;90mYaml found at:e[0m codecov.yaml
2020-08-18T08:12:21.0473837Z e[0;90m==>e[0m Running gcov in . e[0;90m(disable via -X gcov)e[0m
2020-08-18T08:12:21.0519429Z e[0;90m==>e[0m Searching for coverage reports in:
2020-08-18T08:12:21.0521115Z     e[0;32m+e[0m .
2020-08-18T08:12:21.0647069Z     e[0;90m->e[0m Found 2 reports
2020-08-18T08:12:21.0649805Z e[0;90m==>e[0m Detecting git/mercurial file structure
2020-08-18T08:12:21.0775170Z e[0;90m==>e[0m Reading reports
2020-08-18T08:12:21.0837834Z     e[0;32m+e[0m ./coverage.xml e[0;90mbytes=175783e[0m
2020-08-18T08:12:21.0933147Z     e[0;32m+e[0m ./codecov.yaml e[0;90mbytes=105e[0m
2020-08-18T08:12:21.0964327Z e[0;90m==>e[0m Appending adjustments
2020-08-18T08:12:21.0965193Z     e[0;36mhttps://docs.codecov.io/docs/fixing-reportse[0m
2020-08-18T08:12:21.1157149Z     e[0;90m->e[0m No adjustments found
2020-08-18T08:12:21.1157959Z e[0;90m==>e[0m Gzipping contents
2020-08-18T08:12:21.1355709Z e[0;90m==>e[0m Uploading reports
2020-08-18T08:12:21.1357239Z     e[0;90murl:e[0m https://codecov.io
2020-08-18T08:12:21.1358311Z     e[0;90mquery:e[0m branch=bump-codecov&commit=f285876ef885f1e85d8e897c84a8f9039815aff3&build=213293561&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F213293561&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=690&job=&cmd_args=n,F,Z
2020-08-18T08:12:21.1399529Z e[0;90m->e[0m  Pinging Codecov
2020-08-18T08:12:21.1400914Z https://codecov.io/upload/v4?package=bash-20200728-9fb7d93&token=secret&branch=bump-codecov&commit=f285876ef885f1e85d8e897c84a8f9039815aff3&build=213293561&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F213293561&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=690&job=&cmd_args=n,F,Z
2020-08-18T08:12:44.4056522Z e[0;90m->e[0m  Uploading to
2020-08-18T08:12:44.4058183Z https://storage.googleapis.com/codecov/v4/raw/2020-08-18/875BAD842DFC1254D7E8EE01DC26615F/f285876ef885f1e85d8e897c84a8f9039815aff3/3237d7b7-3f52-44bc-9e4b-78c21c0b8d21.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20200818%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20200818T081244Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=5c9b4d0317f3f73867751c738f30f0cad27750e3666810e0e075616d94ea9b64
2020-08-18T08:12:44.4186032Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2020-08-18T08:12:44.4186705Z                                  Dload  Upload   Total   Spent    Left  Speed
2020-08-18T08:12:44.4187055Z 
2020-08-18T08:12:44.6083483Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2020-08-18T08:12:44.6084948Z 100 13468    0     0  100 13468      0  69422 --:--:-- --:--:-- --:--:-- 69422
2020-08-18T08:12:44.6162467Z     e[0;32m->e[0m View reports at e[0;36mhttps://codecov.io/github/VWS-Python/vws-python-mock/commit/f285876ef885f1e85d8e897c84a8f9039815aff3e[0m

tom · August 31, 2020, 6:46pm

@adamtheturtle, hmmm… I went through all the reports there, and I’m seeing that lines in the method never get covered. However, I see for 16 of the reports, that the function definition gets called.

Are you able to output coverage.xml for those runs before they get to the Codecov step? Probably adding a step that does cat ./coverage.xml.

adamtheturtle · March 23, 2021, 4:22pm

Thanks @tom

I added the cat ./coverage.xml step and I have an example of this not working!

The affected PR is Add cache to pip install process by adamtheturtle · Pull Request #884 · VWS-Python/vws-python-mock · GitHub.

The Codecov report is at Codecov.
The Codecov report explains shows that many lines in test_docker.py are not covered. For example, it shows that line 90 is not covered.

The check which runs test_docker.py is at: Add cache to pip install process by adamtheturtle · Pull Request #884 · VWS-Python/vws-python-mock · GitHub

It is clear as someone who knows the code that the missing lines were run, and this is confirmed by the ./coverage.xml contents. See the “Show coverage file” stage.

The following section shows (if I understand correctly) that line 90 was hit.

<class name="test_docker.py" filename="mock_vws/test_docker.py" complexity="0" line-rate="0.9079" branch-rate="0.5">
	<methods/>
	<lines>
		<line number="5" hits="1"/>
		<line number="6" hits="1"/>
		<line number="7" hits="1"/>
		<line number="8" hits="1"/>
		<line number="9" hits="1"/>
		<line number="10" hits="1"/>
		<line number="12" hits="1"/>
		<line number="13" hits="1"/>
		<line number="14" hits="1"/>
		<line number="15" hits="1"/>
		<line number="16" hits="1"/>
		<line number="18" hits="1"/>
		<line number="21" hits="1"/>
		<line number="22" hits="1"/>
		<line number="26" hits="1"/>
		<line number="27" hits="1"/>
		<line number="28" hits="1"/>
		<line number="32" hits="0"/>
		<line number="35" hits="0"/>
		<line number="39" hits="1"/>
		<line number="40" hits="1"/>
		<line number="42" hits="1"/>
		<line number="45" hits="1"/>
		<line number="49" hits="1"/>
		<line number="57" hits="1"/>
		<line number="58" hits="1"/>
		<line number="60" hits="1"/>
		<line number="61" hits="1"/>
		<line number="62" hits="1"/>
		<line number="65" hits="1"/>
		<line number="66" hits="1"/>
		<line number="68" hits="1"/>
		<line number="69" hits="1"/>
		<line number="70" hits="1"/>
		<line number="71" hits="1"/>
		<line number="72" hits="1"/>
		<line number="74" hits="1"/>
		<line number="75" hits="1"/>
		<line number="80" hits="0"/>
		<line number="81" hits="0" branch="true" condition-coverage="0% (0/2)" missing-branches="exit,86"/>
		<line number="86" hits="0"/>
		<line number="87" hits="0"/>
		<line number="88" hits="0"/>
		<line number="90" hits="1"/>
		<line number="95" hits="1"/>
		<line number="100" hits="1"/>
		<line number="106" hits="1"/>
		<line number="107" hits="1"/>
		<line number="108" hits="1"/>
		<line number="110" hits="1"/>
		<line number="117" hits="1"/>
		<line number="125" hits="1"/>
		<line number="134" hits="1"/>
		<line number="135" hits="1"/>
		<line number="138" hits="1"/>
		<line number="139" hits="1"/>
		<line number="143" hits="1"/>
		<line number="144" hits="1"/>
		<line number="145" hits="1"/>
		<line number="146" hits="1"/>
		<line number="148" hits="1"/>
		<line number="149" hits="1"/>
		<line number="150" hits="1"/>
		<line number="151" hits="1"/>
		<line number="153" hits="1"/>
		<line number="156" hits="1"/>
		<line number="161" hits="1"/>
		<line number="163" hits="1"/>
		<line number="169" hits="1"/>
		<line number="177" hits="1"/>
		<line number="179" hits="1"/>
		<line number="185" hits="1"/>
		<line number="187" hits="1" branch="true" condition-coverage="100% (2/2)"/>
		<line number="188" hits="1"/>
		<line number="189" hits="1"/>
		<line number="191" hits="1"/>
	</lines>
</class>

This test skips on an if statement if we run on Windows.
As well as the linked Linux build, there is also a Windows build at Add cache to pip install process by adamtheturtle · Pull Request #884 · VWS-Python/vws-python-mock · GitHub.
I wonder - could a Windows build with hits="0" potentially override the Linux build for those lines?

tom · March 29, 2021, 5:41pm

@adamtheturtle hmmm, that’s not the coverage that I’m seeing on our side. If you go to the commit’s build tab, you can view the coverage uploaded to Codecov. All 4 builds show missed lines after line 90. Could you paste the Codecov upload output so we can see why this might be happening?

adamtheturtle · May 2, 2021, 5:49pm

Sure @tom - here you go.

2021-03-23T09:55:44.0526695Z ##[group]Run codecov/codecov-action@v1.0.13
2021-03-23T09:55:44.0527186Z with:
2021-03-23T09:55:44.0527562Z   fail_ci_if_error: true
2021-03-23T09:55:44.0527938Z env:
2021-03-23T09:55:44.0528465Z   pythonLocation: /opt/hostedtoolcache/Python/3.9.2/x64
2021-03-23T09:55:44.0529196Z   LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.2/x64/lib
2021-03-23T09:55:44.0529734Z ##[endgroup]
2021-03-23T09:55:44.2416845Z [command]bash codecov.sh -n  -F  -Z
2021-03-23T09:55:44.2511226Z 
2021-03-23T09:55:44.2511908Z   _____          _
2021-03-23T09:55:44.2512720Z  / ____|        | |
2021-03-23T09:55:44.2513224Z | |     ___   __| | ___  ___ _____   __
2021-03-23T09:55:44.2513586Z | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
2021-03-23T09:55:44.2513955Z | |___| (_) | (_| |  __/ (_| (_) \ V /
2021-03-23T09:55:44.2514342Z  \_____\___/ \__,_|\___|\___\___/ \_/
2021-03-23T09:55:44.2515296Z                               Bash-20210309-2b87ace
2021-03-23T09:55:44.2515616Z 
2021-03-23T09:55:44.2515855Z 
2021-03-23T09:55:44.2536300Z e[0;36m==>e[0m git version 2.30.2 found
2021-03-23T09:55:44.2645433Z e[0;36m==>e[0m curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
2021-03-23T09:55:44.2646893Z Release-Date: 2020-01-08
2021-03-23T09:55:44.2648011Z Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
2021-03-23T09:55:44.2649815Z Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
2021-03-23T09:55:44.2660213Z e[0;90m==>e[0m GitHub Actions detected.
2021-03-23T09:55:44.2660986Z     Env vars used:
2021-03-23T09:55:44.2662167Z       -> GITHUB_ACTIONS:    true
2021-03-23T09:55:44.2662943Z       -> GITHUB_HEAD_REF:   cache-pip-workflow
2021-03-23T09:55:44.2664418Z       -> GITHUB_REF:        refs/pull/884/merge
2021-03-23T09:55:44.2665506Z       -> GITHUB_REPOSITORY: VWS-Python/vws-python-mock
2021-03-23T09:55:44.2666213Z       -> GITHUB_RUN_ID:     679141510
2021-03-23T09:55:44.2667021Z       -> GITHUB_SHA:        f1751a730a1eb192fb5181f27a682af8d04a5623
2021-03-23T09:55:44.2667773Z       -> GITHUB_WORKFLOW:   CI
2021-03-23T09:55:44.2884773Z e[0;31m->  Issue detecting commit SHA. Please run actions/checkout with fetch-depth > 1 or set to 0e[0m
2021-03-23T09:55:44.2885923Z     e[0;90mproject root:e[0m .
2021-03-23T09:55:44.2949943Z     e[0;90mYaml found at:e[0m codecov.yaml
2021-03-23T09:55:44.3375513Z e[0;90m==>e[0m Running gcov in . e[0;90m(disable via -X gcov)e[0m
2021-03-23T09:55:44.3418513Z e[0;90m==>e[0m Searching for coverage reports in:
2021-03-23T09:55:44.3419192Z     e[0;32m+e[0m .
2021-03-23T09:55:44.3520052Z     e[0;90m->e[0m Found 2 reports
2021-03-23T09:55:44.3521959Z e[0;90m==>e[0m Detecting git/mercurial file structure
2021-03-23T09:55:44.3604704Z e[0;90m==>e[0m Reading reports
2021-03-23T09:55:44.3692112Z     e[0;32m+e[0m ./coverage.xml e[0;90mbytes=194767e[0m
2021-03-23T09:55:44.3777056Z     e[0;32m+e[0m ./codecov.yaml e[0;90mbytes=105e[0m
2021-03-23T09:55:44.3803260Z e[0;90m==>e[0m Appending adjustments
2021-03-23T09:55:44.3804540Z     e[0;36mhttps://docs.codecov.io/docs/fixing-reportse[0m
2021-03-23T09:55:44.4008115Z     e[0;90m->e[0m No adjustments found
2021-03-23T09:55:44.4010527Z e[0;90m==>e[0m Gzipping contents
2021-03-23T09:55:44.4187587Z         16K	/tmp/codecov.QOpeYK.gz
2021-03-23T09:55:44.4209211Z e[0;90m==>e[0m Uploading reports
2021-03-23T09:55:44.4210396Z     e[0;90murl:e[0m https://codecov.io
2021-03-23T09:55:44.4213079Z     e[0;90mquery:e[0m branch=cache-pip-workflow&commit=f1751a730a1eb192fb5181f27a682af8d04a5623&build=679141510&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F679141510&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=884&job=CI&cmd_args=n,F,Z
2021-03-23T09:55:44.4248655Z e[0;90m->e[0m  Pinging Codecov
2021-03-23T09:55:44.4251144Z https://codecov.io/upload/v4?package=bash-20210309-2b87ace&token=secret&branch=cache-pip-workflow&commit=f1751a730a1eb192fb5181f27a682af8d04a5623&build=679141510&build_url=http%3A%2F%2Fgithub.com%2FVWS-Python%2Fvws-python-mock%2Factions%2Fruns%2F679141510&name=&tag=&slug=VWS-Python%2Fvws-python-mock&service=github-actions&flags=&pr=884&job=CI&cmd_args=n,F,Z
2021-03-23T09:55:44.8889254Z e[0;90m->e[0m  Uploading to
2021-03-23T09:55:44.8894181Z https://storage.googleapis.com/codecov/v4/raw/2021-03-23/875BAD842DFC1254D7E8EE01DC26615F/f1751a730a1eb192fb5181f27a682af8d04a5623/1897308e-fe69-4939-8e29-9f2a13dbf0da.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20210323%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20210323T095544Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=3c69127a0ef9dbe162ce39efd10fe8786e41dfc172a38849c1896868204bbe55
2021-03-23T09:55:44.8980030Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2021-03-23T09:55:44.8981114Z                                  Dload  Upload   Total   Spent    Left  Speed
2021-03-23T09:55:44.8981777Z 
2021-03-23T09:55:45.0755951Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-03-23T09:55:45.0757351Z 100 14471    0     0  100 14471      0  81757 --:--:-- --:--:-- --:--:-- 81757
2021-03-23T09:55:45.0840763Z     e[0;32m->e[0m Reports have been successfully queued for processing at e[0;36mhttps://codecov.io/github/VWS-Python/vws-python-mock/commit/f1751a730a1eb192fb5181f27a682af8d04a5623e[0m
2021-03-23T09:55:45.2138074Z Post job cleanup.
2021-03-23T09:55:45.4176191Z Unable to reserve cache with key Linux-pip-3d792fa9d0121d4cdb0224ac42dc0e457254ec53ae5ba17a270b1836efeb2d12, another job may be creating this cache.
2021-03-23T09:55:45.4347975Z Post job cleanup.
2021-03-23T09:55:45.5420283Z [command]/usr/bin/git version
2021-03-23T09:55:45.5477048Z git version 2.30.2
2021-03-23T09:55:45.5522578Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2021-03-23T09:55:45.5560484Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2021-03-23T09:55:45.5807672Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2021-03-23T09:55:45.5837744Z http.https://github.com/.extraheader
2021-03-23T09:55:45.5847674Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2021-03-23T09:55:45.5883172Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :slight_smile: 
````Preformatted text`

adamtheturtle · May 3, 2021, 6:43pm

@tom do you think:

Issue detecting commit SHA. Please run actions/checkout with fetch-depth > 1 or set to 0

Could be causing the issue?
If so, maybe the action should fail on that?

tom · May 3, 2021, 7:03pm

@adamtheturtle oh that is definitely likely. We have since patched that issue in the latest versions of the Action. Do you see it on any recent runs?

adamtheturtle · May 4, 2021, 2:24pm

Hi @tom , this happened yesterday with codecov/codecov-action@v1.0.13. I will bump to the latest version and keep you up to date if I see this again.

Topic		Replies	Views
Coverage seems to be off Support javascript , github	3	467	October 6, 2020
Codecov reporting coverage that is nonsensical Bug Fixes	8	325	November 11, 2021
Code showing as not covered despite coverage in workflows recorded as 100% Bug Fixes python , github	1	305	August 18, 2021
GitHub Check not properly handling subsets of files Support	6	420	August 31, 2020
Coverage is comparing with incorrect base branch Support golang , github	6	163	February 28, 2023