Handling Go Errors In Tests
Errors in Go are like the STOP sign. They send a strong signal to avoid disasters. Just like the Go error yelling don’t trust returned value, don’t use it.
So, STOP means STOP, regardless of any text plates mounted under it, irrespective of the language and country.
Would you ignore the STOP and rush through the road junction or a railway crossing?
Let’s see how the road sign analogy applies in Go.
err := ValidateDosProtectedResource(someStruct)
if err != nil {
// handle error
}
The function above takes a struct and checks if the input value is valid. If the struct meets requirements, the function returns an error value nil. If the struct is not valid, the err value is not nil.
The ValidateDosProtectedResource function calls internally the fmt.Errorf method with additional parameters. It doesn’t return any specific error type.
We test the behaviour by calling the function with valid and invalid structs and verifying whether the function performs its intended task. Nothing more, nothing less.
What’s going on inside table tests
Here is the original test implementation.
func TestValidateDosProtectedResource(t *testing.T) {
t.Parallel()
tests := []struct {
protected *v1beta1.DosProtectedResource
expectErr string
msg string
}{
{
protected: &v1beta1.DosProtectedResource{},
expectErr: "error validating DosProtectedResource: missing value for field: name",
msg: "empty resource",
},
{
protected: &v1beta1.DosProtectedResource{
Spec: v1beta1.DosProtectedResourceSpec{
Name: "name",
ApDosMonitor: &v1beta1.ApDosMonitor{
URI: "example.com",
},
DosAccessLogDest: "example.service.com:123",
},
},
msg: "name, dosAccessLogDest and apDosMonitor specified",
},
// --snip--
}
for _, test := range tests {
err := ValidateDosProtectedResource(test.protected)
if err != nil {
if test.expectErr == "" {
t.Errorf("ValidateDosProtectedResource() returned unexpected error: '%v' for the case of: '%s'", err, test.msg)
continue
}
if test.expectErr != err.Error() {
t.Errorf("ValidateDosProtectedResource() returned error for the case of '%s', expected err: '%s' got err: '%s'", test.msg, test.expectErr, err.Error())
}
} else {
if test.expectErr != "" {
t.Errorf("ValidateDosProtectedResource() failed to return expected error: '%v' for the case of: '%s'", test.expectErr, test.msg)
}
}
}
}
Demistyfing business logic
Each loop iteration runs the following logic and determines if the test fails or passes.
err := ValidateDosProtectedResource(test.protected)
if err != nil {
if test.expectErr == "" {
t.Errorf("ValidateDosProtectedResource() returned unexpected error: '%v' for the case of: '%s'", err, test.msg)
continue
}
if test.expectErr != err.Error() {
t.Errorf("ValidateDosProtectedResource() returned error for the case of '%s', expected err: '%s' got err: '%s'", test.msg, test.expectErr, err.Error())
}
} else {
if test.expectErr != "" {
t.Errorf("ValidateDosProtectedResource() failed to return expected error: '%v' for the case of: '%s'", test.expectErr, test.msg)
}
}
Immediately after calling ValidateDosProtectedResource, we check if err is nil. Then, we have two if branches that compare two values of type string. First, we compare the expectedErr string with an empty string and then with the string value returned from the err.Error() call.
After the two if branches, we have the else branch. Test tool will run this block only if err is not nil. It means only when the validation succeeds.
But right after the else we have another if branch executed when expectedErr value has a different value than an empty string.
It gets complicated quickly.
The value expectErr in the test struct drives decisions to fail or pass the test. If the expectErr value is "", it means that the input struct is valid and the function should return err value nil, indicating the validation works correctly.
If the expression if test.expectErr != "" in the else block fails, it means the code in the branch is not executed, and we exit the main if err != nil branch.
This means the test passes. The loop repeats.
Honestly, glanceability leaves a lot to be desired. You are not alone if you think that we have multiple issues here.
Comparing error strings is a sloppy business
As we know, the error value can be nil or not. Also, the function under test doesn’t return any specific error type.
The test adds additional logic and compares strings to determine success or failure.
if test.expectErr == "" {
t.Errorf(err)
continue
}
if test.expectErr != err.Error() {
t.Errorf("ValidateDosProtectedResource() returned error for the case of '%s', expected err: '%s' got err: '%s'", test.msg, test.expectErr, err.Error())
}
} else {
if test.expectErr != "" {
t.Errorf("ValidateDosProtectedResource() failed to return expected error: '%v' for the case of: '%s'", test.expectErr, test.msg)
}
}
Listing 4: Comparing error strings
Comparing error strings makes the tests brittle, unnecessarily complicated, and plainly wrong.
Let’s consider two examples:
- You edit the error string and add a comma
- You port your app to a client in Berlin and change strings to German
You change the code and suddenly thousands of tests fail. You didn’t touch validation rules, nor did you touch test inputs. Yet you end up with a giant mess.
Coming back to the STOP sign analogy, have you ever seen a STOP sign with a text informing that the sign is valid only on Mondays, between 9 PM and 10 PM, but only in odd years when the cloud coverage level is above 65% and the month is January?
Comparing error strings in tests is precisely like that. If we change punctuation in the error message, would the error change? No. The error will remain the same as before, regardless of the input data passed to the function.
Go has its own way of comparing errors. There are multiple methods in the errors package we can use, depending on the context.
Before even thinking about using error types and comparing them, we need to decide if we really need to do error comparison at all. We must consider whether the business logic in our program relies on any specific error type.
The function ValidateDosProtectedResource doesn’t use any specific error type that would drive application logic. The error bubbles up, and it’s the logger’s job to handle it. That’s it. It doesn’t matter what string message it carries because the behaviour of the application doesn’t depend on it.
Removing the noise
After a short and precise surgery, the test function is slimmer. We just removed all the branching noise.
err := ValidateDosProtectedResource(test.protected)
if err != nil {
t.Error(err)
}
But this refactoring works only if test.protected values are valid. So, the table test must group only valid inputs.
All invalid inputs land in a separate test responsible for verifying how the function handles bad data.
err := ValidateDosProtectedResource(test.protected)
if err == nil {
t.Errorf("want error for: '%+v', got nil", test.protected)
}
This looks much cleaner. We end up with two tests: one for checking function behaviour with invalid inputs, and the second one with valid inputs.
It’s simple, error-prone and documents the function’s behaviour. We handle invalid and valid inputs in two tests.
func TestValidateDosResourceWithInvalidInput(t *testing.T) {
t.Parallel()
tests := []struct {
input *v1beta1.DosProtectedResource
msg string
}{
{
input: &v1beta1.DosProtectedResource{},
msg: "empty resource",
},
// --snip--
}
for _, test := range tests {
err := ValidateDosProtectedResource(test.input)
if err == nil {
t.Errorf("want error for invalid input: '%+v', got nil", test.input)
}
}
}
Listing 5: Handling invalid inputs after refactoring
func TestValidateDosResourceWithValidInput(t *testing.T) {
t.Parallel()
tests := []struct {
input *v1beta1.DosProtectedResource
}{
{
input: &v1beta1.DosProtectedResource{
Spec: v1beta1.DosProtectedResourceSpec{
Name: "name",
},
},
},
// --snip--
}
for _, test := range tests {
err := ValidateDosProtectedResource(test.input)
if err != nil {
t.Error(err)
}
}
}
Listing 6: Handling valid inputs after refactoring
In the end, we isolate valid and invalid inputs and test the function’s behaviour in two separate tests. The tests don’t use branches, and inputs (structs) include only necessary fields.
We managed to improve the glanceability index.
Key takeaways
- separate tests for valid and invalid inputs
- keep test logic as simple as possible