lcov reports should not use line number zero in BRDA: records

[zackw]

Describe the bug

coverage lcov can generate BRDA: records with line number zero. Current versions of lcov's genhtml reject such lines:

genhtml: ERROR: line 0 of .../sourcefile.py has branchcov but no linecov data

To Reproduce

The bad BRDA records are coming from here: https://github.com/nedbat/coveragepy/blob/5467e1f0c9e16ef5456ecccb56993b15c34b7a3e/coverage/lcovreport.py#L108-L112

I do not understand how to hit this case. The commentary, and the description of what negative numbers mean in arc records (https://coverage.readthedocs.io/en/7.6.1/api_coveragedata.html#coverage.CoverageData.arc) make it sound like unexecuted conditional return statements could cause the problem. Running coverage run --branch test.py on this file

def f(x):
  if x < 0:
    return
  print("got here")

f(0)
f(1)

produces a .coverage database whose arc table does contain negative numbers, but running coverage lcov on that database produces

BRDA:3,0,0,-
BRDA:4,0,1,1
BRF:2
BRH:1

which does not use line zero and looks correct to me (well, modulo the fact that a human would say there isn't a branch on line 4? but that's a separate issue).

Looking at the database for the real program that produced the actual bad lcov-format report that prompted this issue, I see lots more cases of negative numbers in the arc table, but the associated positive numbers are too large to be actual line numbers in the associated file, which means I don't actually understand what the records in the arc table mean, so I'm stuck.

Expected behavior

coverage lcov should generate a .lcov file that genhtml does not reject.

[nedbat]

Looking at the database for the real program that prompted me to file this issue, I see lots more cases of negative numbers in the arc table, but the associated positive numbers are too large to be actual line numbers in the associated file, which means I don't actually understand what the records in the arc table mean, so I'm stuck.

Can you share that project?

[zackw]

Sure: https://github.com/MillionConcepts/pdr

Using that, a repro recipe should be

• create a blank venv
• pip install pytest-cov
• pip install -e '.[browsify,fits,pvl,tiff]'
• pytest --cov --cov-branch --import-mode=importlib
• coverage lcov

[zackw]

If I understand correctly what BRDA: records mean (which I am about 90% sure of), the line number is supposed to be the line where the branch actually is, and the block and branch numbers are supposed to be, together, a unique but meaningless identifier for the branch's destination. Based on this, arc records with negative fromno shouldn't generate BRDA: records at all, and you can just make up block and branch numbers for all the different tonos for any given positive fromno (it seems to be fine to use zero in those fields).

Also, - in the taken field is supposed to happen only when that branch was never reached, i.e. the line containing the branch wasn't executed at all. It should be zero for branches where the conditional statement did execute but the branch was not taken.

[nedbat]

Thanks. Can you point me to specific entries in the .coverage file that have nonsensical entries in the arcs table? BTW, I'm using Python 3.11. Your setup.py says 3.9 should work, but it fails when trying to import inspect.get_annotations.

[zackw]

I do not know which entries in the arcs table are causing the problem. However, if you run "coverage lcov" you should see several BRDA:0 lines in the output. If there aren't any, try "git switch develop" and then re-running the tests, which may also clear up the inspect.get_annotations / python 3.9 issue.

…

On Thu, Sep 5, 2024, at 4:45 PM, Ned Batchelder wrote: Thanks. Can you point me to specific entries in the .coverage file that have nonsensical entries in the arcs table? BTW, I'm using Python 3.11. Your setup.py says 3.9 should work, but it fails when trying to import inspect.get_annotations. — Reply to this email directly, view it on GitHub <#1846 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACPSC7QNSCRP2O7AVK2YILZVC7FTAVCNFSM6AAAAABNVAXYIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZSGU4TSMRYHE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[nedbat]

Sorry, you said:

Looking at the database for the real program that produced the actual bad lcov-format report that prompted this issue, I see lots more cases of negative numbers in the arc table, but the associated positive numbers are too large to be actual line numbers in the associated file...

I'm curious what entries in the arc table seem wrong to you?

[zackw]

Ah, I misunderstood what you meant by "nonsensical entries in the arcs table".

I'm attaching a SQL dump of the coverage database that I'm looking at. I have added a max_lineno column to the file table, giving the maximum line number (as measured by wc -l) for each file. Using that, here are the arc-table entries that seem to violate the specification:

sqlite> select file.path, file.max_lineno, arc.fromno, arc.tono
          from arc, file
         where arc.file_id = file.id
           and (arc.fromno > file.max_lineno
             or arc.tono > file.max_lineno);
path                     max_lineno  fromno  tono
-----------------------  ----------  ------  ----
pdr/pdr.py               968         964     971
pdr/pdr.py               968         967     969
pdr/pdr.py               968         969     -964
pdr/pdr.py               968         971     -959
pdr/loaders/dispatch.py  137         130     138
pdr/loaders/dispatch.py  137         138     -1

I don't know if this has anything to do with the BRDA:0 issue.

It is also not clear to me what negative numbers other than -1 mean.

This database was generated from git commit 092675e442438ddc39d4bb9e9d0d5f4c1e317f29. The .coveragerc was slightly modified from what's checked in:

[run]
source_pkgs = pdr
omit =
	*/six_new.py
	*/tkinter_stub.py

coverage+linemax.sql.gz

[zackw]

Another piece of information that might be useful to you: here is a comparison between the output of coverage lcov (A.lcov in the diff) and the result of running the xml2lcov utility (from the lcov project) on the output of coverage xml (B.lcov in the diff), after normalizing both files for comparison.

"Normalizing both files for comparison" means:

• all TN: lines were stripped
• checksums were removed from all DA: lines
• records were sorted by SF: pathname
• within each record, the sequence of sub-record types was canonicalized to DA, LF, LH, BRDA, BRF, BRH
• DA and BRDA lines were sorted numerically by line number
• vacuous BRF:0 BRH:0 and LF:0 LH:0 pairs were removed

Abstractly, every difference shown in this attachment represents a bug in one of the three tools used, but I do not know which one. For purpose of this bug report, however, please note that the BRDA records are consistently totally different. There are occasional differences in DA records as well.

lcov-gen-comparison.diff.gz

[zackw]

A couple of times earlier I said "GCDA" when I meant "BRDA". Sorry. No idea where GC came from.

[nedbat]

I'm coming to the unfortunate conclusion that the coverage.py lcov output is simply wrong with respect to branches. It produces BRDA lines that name the destination line, but should name the source line. The lcov format is poorly documented. This seems to be the best we have: https://manpages.debian.org/stretch/lcov/geninfo.1.en.html . It says:

Branch coverage information is stored which one line per branch:

BRDA:<line number>,<block number>,<branch number>,<taken>

Block number and branch number are gcc internal IDs for the branch. Taken is either '-' if the basic block containing the branch was never executed or a number indicating how often that branch was taken.

"which" is supposed to be "with" I guess? And line number isn't clarified. The xml2lcov tool reports the source line number, which makes sense. Changing to that interpretation will solve the 0 line number problem.

@bradb423 What do you think?