Informational: true check is marked as failng

Description

When we set up CodeCov, we noticed that often project coverage would be failing, often saying that “the coverage will decrease by < 0.01%”. We would still like to get project reports, but since it would fail our builds too often, we decided to keep it in informational mode:

informational

Use Codecov in informational mode. Default is false . If true is specified the resulting status will pass > no matter what the coverage is or what other settings are specified. Informational mode is great to use > if you want to expose codecov information to other developers in your pull request without necessarily gating PRs on that information.

This would suggest that the status should pass even if coverage dropped too low. However, we’re still getting failed “informational” status:

Is this desired behaviour?

Commit SHAs

d092e839d92a55908de8b84bf330882cbcc29f6d

Repository

https://github.com/eatclub/web/

CI/CD or Build URL

The CI is private.

Uploader

Codecov bash uploader, version “20200728-9fb7d93”.

Codecov Output

  • bash build/scripts/code_cov_uploader.sh


/ | | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / ` |/ _ / / _ \ \ / /
| |
| (
) | (
| | __/ (
| (
) \ V /
______/ _,|_|____/ _/
Bash-20200728-9fb7d93

==> Jenkins CI detected.
project root: .
Fixing merge commit SHA
Yaml found at: codecov.yml
–> token set from yaml
==> Running gcov in . (disable via -X gcov)
==> Searching for coverage reports in:
+ .
-> Found 2 reports
==> Detecting git/mercurial file structure
==> Reading reports
+ ./nosetests.xml bytes=574109
+ ./coverage.xml bytes=2260886
==> Appending adjustments
https://docs.codecov.io/docs/fixing-reports
+ Found adjustments
==> Gzipping contents
==> Uploading reports
url: https://codecov.io
query: branch=COM-686-bye-bye-base-template&commit=d092e839d92a55908de8b84bf330882cbcc29f6d&build=15751&build_url=https%3A%2F%2Fjenkins.eatclub.net%2Fjob%2FWeb_unittest%2F15751%2F&name=&tag=&slug=eatclub%2Fweb&service=jenkins&flags=&pr=6489&job=&cmd_args=
-> Pinging Codecov
https://codecov.io/upload/v4?package=bash-20200728-9fb7d93&token=secret&branch=COM-686-bye-bye-base-template&commit=d092e839d92a55908de8b84bf330882cbcc29f6d&build=15751&build_url=https%3A%2F%2Fjenkins.eatclub.net%2Fjob%2FWeb_unittest%2F15751%2F&name=&tag=&slug=eatclub%2Fweb&service=jenkins&flags=&pr=6489&job=&cmd_args=
-> Uploading to
https://storage.googleapis.com/codecov/v4/raw/2020-12-17/C0CBE9440179AE1D090801ABC07BD16B/d092e839d92a55908de8b84bf330882cbcc29f6d/e6a7fb0e-a8e1-4e12-8749-67c9cecb800a.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20201217%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20201217T172235Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=0e3efd001e4758a4caa8f7c7f5b01ac9706c013031a7beb72586b210384a6e92
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
100 213k 0 0 100 213k 0 521k --:–:-- --:–:-- --:–:-- 522k
-> View reports at https://codecov.io/github/eatclub/web/commit/d092e839d92a55908de8b84bf330882cbcc29f6d

Expected Results

I’d have expected to have informational status that informs us about the drop in coverage, but still has a passing status check on Github.

Actual Results

The status check on Github is showing up as failed

Additional Information

Our codecov.yml:

coverage:
  status:
    project:
      # Treat Project coverage info as informational only
      # CodeCov too often marks the build as failed when it says that coverage dropped by < 0.01%
      informational: true
    patch:
      default:
        target: 80
        if_not_found: error

I guess my question is:
Is this the expected behaviour of “informational” check and we’re misunderstanding how it’s meant to work? Or is the behaviour we’re seeing a result of some misconfigurion/error/bug?

I think you want to do

coverage:
  status:
    project:
      # Treat Project coverage info as informational only
      # CodeCov too often marks the build as failed when it says that coverage dropped by < 0.01%
      default:
        informational: true
    patch:
      default:
        target: 80
        if_not_found: error

@tom Oops :woman_facepalming: Yes, that was it! Thanks for your help :slight_smile:

1 Like