Differential Coverage for Debugging

Summary

Differential Coverage for Debugging Posted on Friday, April 25, 2025. I have been debugging some code I did not write and was reminded of this technique. I’m sure it’s a very old debugging technique (like bisection), but it should be more widely known. Suppose you have one test case that’s failing. You can get a sense of what code might be involved by comparing the code coverage of successful tests with the code coverage of the failing test. For example, I’ve inserted a bug into my development copy of math/big: $ go test --- FAIL: TestAddSub (0.00s) int_test.go:2020: addSub(-0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff, 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff) = -0x0, -0x1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe, want 0x0, -0x1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe FAIL exit status 1 FAIL math/big 7.528s $ Let’s collect a passing and failing profile: $ go test -coverprofile=c1.prof -skip='TestAddSub$' PASS coverage: 85.0% of statements ok math/big 8.373s % go test -coverprofile=c2.prof -run='TestAddSub$' --- FAIL: TestAddSub (0.00s) int_test.go:2020: addSub(-0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff, 0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff) = -0x0, -0x1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe, want 0x0, -0x1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe FAIL coverage: 4.7% of statements exit status 1 FAIL math/big 0.789s $ Now we can diff them to make a profile showing what’s unique about the failing test: $ (head -1 c1.prof; diff c[12].prof | sed -n 's/^> //p') >c3.prof $ go tool cover -html=c3.prof The head -1 is preserving the one-line coverage profile header. The diff | sed saves only the lines unique to the failing test’s profile, and the go tool cover -html opens the profile in a web browser. In the resulting profile, “covered” (green) means it ran in the failing test bu...