Codecov invokes gcov incorrectly

When running codecov.sh (from https://codecov.io/bash) I noticed that it invokes gcov incorrectly for project with subdirectories and then gets completely confused where the source files live.

I tried to run it with several different ways to get some sensible output, but I get only nonsense like 8% coverage, and looking at hits, I see it’s missing several that should have been hit. It’s like it can’t figure out the output of gcov.

Any hints on what to do?

@cmouse, we’ll probably need more information here like the Codecov output or a build URL. Have you tried running the bash script in the subdirectories? Is this a monorepo?

Hi!

Codecov output is available here https://codecov.io/gh/dovecot/core/commit/fea1bacab954d2f112da8a95756f198530e274af/build and the repository is at GitHub - dovecot/core: Dovecot mail server.

I tried various ways, including manually running gcov using a find + helper script to generate gcov information in correct directories and then invoking codecov tool with -X gcov. You can also see the output at https://codecov.io/gh/dovecot/core/commit/fea1bacab954d2f112da8a95756f198530e274af/ - I compared this to what lcov gives us and it’s wildly different result.

Hi @cmouse, apologies if I wasn’t clear. I meant the Codecov output that includes the banner which is typically found in the CI run. You can also post a build url.

Hi tom, please find it here codecov output · GitHub

Thank you for looking into this.

edit:

This has been prepared by running gcov -mbupa *.c *.h *.cc *.hh in each source directory.

Hi @cmouse, sorry to keep coming back for more information. I guess what I’m trying to attain is for a recent commit (1) what are you expecting coverage to look like via a report or screenshot, (2) what is the Codecov output, and (3) what is the commit SHA so we can take a look at the build and see where things might have gone wrong.

The output that you copied has gcov disabled, so it’s not getting invoked by Codecov.

You can use the latest master, https://github.com/dovecot/dovecot. You can see from above screenshot that the coverage should be around 50%, and in particular the lib directory should have ~78% coverage.

As I explained in my comment, we need to prepare this outside codecov, because it calls gcov incorrectly, and stuffs all gcov output files to source root directory.

@tom hi, do you still need further information?

Hi @cmouse, sorry for the delay here. This is a bit of a beast to try to debug. Would you happen to have coverage for one file in particular? It would be a little easier to pinpoint where this might be failing.

Hi!

To get coverage for, say, lib directory, you can just run ./test-lib under there. The entire output has been gathered with make check. But, I have included screenshot of some of the files under lib, maybe this helps?

Including also full line coverage for aqueue.c

Hi @cmouse, the raw report here is showing that all of those lines should be uncovered. Would you be able to provide a build URL (from your CI)?

I think I now realize what the problem is. Your script is not dealing correctly with libtool, which puts files into .libs directory.

buildbot@43cb795326f5:/buildslave/core-ci/build/core/src/lib$ gcov .libs/stats-dist  
File 'stats-dist.c'
Lines executed:93.83% of 81
Creating 'stats-dist.c.gcov'

File '/usr/include/x86_64-linux-gnu/bits/string_fortified.h'
Lines executed:100.00% of 1
Creating 'string_fortified.h.gcov'

File 'sort.h'
Lines executed:100.00% of 1
Creating 'sort.h.gcov'

buildbot@43cb795326f5:/buildslave/core-ci/build/core/src/lib$ gcov stats-dist.o
stats-dist.gcda:cannot open data file, assuming not executed
File 'stats-dist.c'
Lines executed:0.00% of 81
Creating 'stats-dist.c.gcov'

File '/usr/include/x86_64-linux-gnu/bits/string_fortified.h'
Lines executed:0.00% of 1
Creating 'string_fortified.h.gcov'

File 'sort.h'
Lines executed:0.00% of 1
Creating 'sort.h.gcov'

I ran this helper script first:

#!/usr/bin/env bash

filelist=`mktemp`

find . -name "*.o" > $filelist
basedir=$PWD

while read f; do
        if [[ $f == */.libs/* ]]; then
                echo $f
                # determine base directory
                d=${f%%.libs*}
                cd $basedir/$d
                f=${f#*/.libs/}
                gcov .libs/$f
        fi
done <$filelist

rm $filelist

followed with invoking codecov.sh as

codecov -X gcov

and now I’m seeing similar results in latest report which lcov gives.

1 Like