Report doesn't show the full coverage report

It seems like the report is not completely loaded which cause false low coverage display.
It started in this commit, that just added more tests. https://codecov.io/gh/RediSearch/JRediSearch/branch/master/commits?from=2020-04-26%2000%3A00%3A00&to=2020-07-14%2000%3A00%3A00

Running coverage tools locally shows almost 90% coverage while the report above shows about 65%.

@gkorland can you provide the Codecov output and/or build URL for one of these?

@gkorland, actually that won’t be necessary. I did a compare of the two commits before and after the coverage drop here. You’ll notice that the only changes are on the src/main/java/io/redisearch/client/Client.java file. If you look at the coverage report sent to Codecov here, you’ll notice that in this file, we are being explicitly told that there are lines that are no longer covered. For example,

						<method name="addDocument" signature="(Lio/redisearch/Document;Lio/redisearch/client/AddOptions;)Z" line-rate="0.0" branch-rate="0.75" complexity="0">
							<lines>
								<line number="483" hits="0" branch="false"/>
								<line number="484" hits="0" branch="false"/>
								<line number="485" hits="0" branch="true" condition-coverage="75% (3/4)">
									<conditions>
										<condition number="0" type="jump" coverage="100%"/>
										<condition number="1" type="jump" coverage="50%"/>
									</conditions>
								</line>
							</lines>
						</method>

This aligns with the diff here:

Notice the drop is not only these two line but in 20%, as I wrote above running it locally does show 20% more coverage

here is a concrete example:
In this report https://codecov.io/gh/RediSearch/JRediSearch/src/master/src/main/java/io/redisearch/client/Client.java

From line 861 (dropIndex) most of the methods are marked in red, while you can see that each one of the methods are covered in this test file https://github.com/RediSearch/JRediSearch/blob/5f6b59e9aa9a5a0b89b21c8faf0fd17fbc6bb577/src/test/java/io/redisearch/client/ClientTest.java#L770.

This is the report I’m getting when running locally, as you can see it reports much better coverage

@gkorland, would you be able to show the coverage you see locally for the Client.java file? One thing you’ll note is that your package is showing 3218 + 484 statements, which is far greater than

If we add up all the lines (including blank lines) it still only adds up to 2049 lines.

Also, I recommend pulling down the Cobertura file created in your CI if you want to compare against your local coverage.

Let me give you a very simple example:

This method is called 7 different tests but the report is showing the method is not covered at all.

Here is one of the places this method is called from:

More over if you’ll look at the trend report

you’ll see that on Jun 1st the coverage has drop from 80% to 60% while this commit can’t cause such effect

It’s even worst now, the all history has been deleted and it dropped to 59%…

I found the issue it was indeed an issue in cobertura switched to jacoco and it’s all OK now

1 Like