Simple Trick To Speed Up Your Go Tests
If you write Go, you likely come across Google’s excellent go-cmp package. The most common method the package provides is cmp.Diff. What does it do? It takes two values of any type and shows their difference in a human-readable format.
It uses the familiar format for printing “diffs”, - for missing parts and + for parts we get but don’t want.
The package provides a lot more functionality for writing clean and readable tests. Some functionality, like the “options” we explore in Go Testing: How to Communicate Clearly.
Today, we focus on the most fundamental parts: checking for equality and calculating diff. We prove that tiny refactoring increases your test efficiency by more than 7 times. Yes, over 700%.
Setting up the stage
Below, we see a snippet of a table test. We loop through a slice of tests (some structs). Each test variable represents a struct containing input data for two functions we test, and data we want to get after a successful function run.
for _, test := range tests {
resApPolicy := CalculateApPolicy(test.policy)
if diff := cmp.Diff(test.expectedApPolRefs, resApPolicy); diff != "" {
t.Errorf("LoadBalancerController.addWAFPolicyRefs() '%v' mismatch (-want +got):\n%s", test.msg, diff)
}
resLogConf := GenerateConf(test.conf)
if diff := cmp.Diff(test.expectedLogConfRefs, resLogConf); diff != "" {
t.Errorf("LoadBalancerController.addWAFPolicyRefs() '%v' mismatch (-want +got):\n%s", test.msg, diff)
}
}
The first question concerns the decision to include two functions in one table test. We don’t know the answer, and we don’t know what these functions do. We don’t need to know this to start reviewing the Go code.
What’s going on inside the for loop?
We call the first function with the argument test.policy and capture the returned value in the resApPolicy variable. The variable represents what we get - our got.
Next, we use cmp.Diff method to calculate the difference between two arguments: test.expectedApPOlRefs and resApPolicy. The result (of type string) we assign to the variable diff, and finally we use the expression diff !="" to determine if the value diff (of type string) is not the same as the empty string literal"". If the expression is evaluated as true, it means there is a difference between test.expectedApPolRefs and resApPolicy. This means the test fails, and we call t.Errorf on the next line.
The same logic applies to testing the second function GenerateConf(test.conf). We calculate the diff value and compare it to an empty string literal.
That sounds like a lot of work. Isn’t it?
At this stage, it’s useful to put an engineer “efficiencer” hat on and ask if we can get the same functionality and results with less work. In short, can we do it more efficiently?
Doing hard work all the time
The first thing that caught our eyes is that we must calculate the Diff and compare it to an empty string for each test input in the tests slice.
Each input struct can be hundreds of lines long and can include embedded structs that include embedded structs. And so on. That’s a lot of data.
for _, test := range tests {
resApPolicy := CalculateApPolicy(test.policy)
if diff := cmp.Diff(test.expectedApPolRefs, resApPolicy); diff != "" {
t.Errorf("LoadBalancerController.addWAFPolicyRefs() '%v' mismatch (-want +got):\n%s", test.msg, diff)
}
resLogConf := GenerateConf(test.conf)
if diff := cmp.Diff(test.expectedLogConfRefs, resLogConf); diff != "" {
t.Errorf("LoadBalancerController.addWAFPolicyRefs() '%v' mismatch (-want +got):\n%s", test.msg, diff)
}
}
When we have tens or hundreds of such input data in the tests slice, we force Go to do a tremendous amount of labour.
Let’s think if we can do something about it.
Work hard only if necessary
What if we can test our functions, but calculate and print out the Diff only when the test fails?
This way we ask to do a hard labour of calculating differences between what we got and want to a handful of failing tests (the fewer the better, of course).
That sounds like a reasonable ask. Let’s find out how to do it.
Since we already use the cmp package, we can use the cmp.Equal function. cmp.Equal takes two arguments of any type and returns a bool value. True means the two values are equal; False means they differ.
The test code after refactoring looks like this:
for _, test := range tests {
resApPolicy := CalculateApPolicy(test.policy)
if !cmp.Equal(test.expectedApPolRefs, resApPolicy) {
t.Error(cmp.Diff(test.expectedApPolRefs, resApPolicy))
}
resLogConf := GenerateConf(test.conf)
if !cmp.Equal(test.expectedLogConfRefs, resLogConf) {
t.Error(cmp.Diff(test.expectedLogConfRefs, resLogConf))
}
}
We call cmp.Equal with two parameters, exactly like before. The result is the boolean value indicating whether the two values are equal or not. If they are equal, it means our test passes, and we move on.
We save the hard labour of calculating Diff, comparing it to the empty string and then determining if the test passes or fails.
We ask the cmp package to calculate Diff only when the two values are not equal. The cmp.Diff takes care of formatting the string (with + and -). Then t.Error takes care of showing the diff string for the failing test.
Simple, clean, readable. One more example that Go code optimised for readability is usually more efficient.
But how do we know this?
Talk is cheap
The refactored code is easier to understand. We ask if the two inputs are equal. If they are, we move on. If they aren’t, we calculate Diff, fail the test, and print out a nicely formatted Diff for the user.
Talk is cheap. Show me the code. - Linus Torvalds
In order to compare and actually see how much effort it takes to call Diff and Equal, we write two benchmarks. One for the cmp.Diff and one for cmp.Equal.
This time, we use test data from the package cars, we refactor in Go Testing: How to Communicate Clearly.
Benchmark for cmp.Diff:
cars_test.go
func BenchmarkCalculateDiff(b *testing.B) {
for b.Loop() {
for _, tc := range testResellPrices {
_ = cmp.Diff(tc.originalPrice, tc.age) != ""
}
}
}
Benchmark for cmp.Equal:
cars_test.go
func BenchmarkCheckEquality(b *testing.B) {
for b.Loop() {
for _, tc := range testResellPrices {
cmp.Equal(tc.originalPrice, tc.age)
}
}
}
Seeing Is Believing
What’s left is to count 3,2,1 and fire up the benchmark.
go test -run none -bench . -benchmem
And the winner is?
goos: darwin
goarch: arm64
pkg: github.com/qba73/cars
cpu: Apple M3
BenchmarkCalculateDiff-8 159274 7628 ns/op 11836 B/op 324 allocs/op
BenchmarkCheckEquality-8 1000000 1005 ns/op 2576 B/op 56 allocs/op
PASS
ok github.com/qba73/cars 4.640s
We have the final results and see that cmp.Equal is 7x more efficient than cmp.Diff. It’s faster, moves less data and uses fewer allocations.
Lessons learned:
- Check for Equality first
- Calculate Diff only when you have to
- Write simple, glanceable Go