How to Handle Go Security Alerts
“Our reports say your software is not secure and has critical and high vulnerabilities. Our delivery pipeline is not working. We can’t upgrade applications in production environments!”. You see Slack messages and Salesforce escalation tickets. It doesn’t look good.
How do you react? What’s your next step?
Seeing the big picture
You know that no application, Docker image, virtual machine or operating system is free of issues. Flaws are sooner or later exploited, classified as security vulnerabilities, and added to the CVE database. This is the reality.
In December 2024, the Go team announced two security bugs and assigned two CVEs to the golang.org/x/net and golang.org/x/crypto packages.
The corresponding CVEs in the National Institute of Standards and Technology (NIST) database are:
If you work on large Go projects, you must apply security patches, as both packages are likely imported as direct or indirect dependencies.
“An ounce of prevention is worth a pound of cure.” — Benjamin Franklin
How can you monitor, react, and upgrade vulnerable modules, packages and libraries?
The process starts in your mind, attitude, and awareness of cybersecurity. You build your knowledge about secure programming practices and study daily Common Software Weaknesses examples and ways to bullet-proof your software products.
The next part of the puzzle is your local development environment. As Go developers, we are lucky to have excellent tools for supporting dependency management, testing, and security. These tools help you maintain fundamental security hygiene.
Configuring the local development environment
What do we need in our local development environment? We need Go and Git. Then we need govulncheck, and for the purpose of today’s exercise, Docker and Docker Scout.
We will also need a local (cloned) git repository with a Go module.
Assuming we already have Go, Git, and Docker installed, the only two remaining components are docker scout and govulncheck. Let’s install both.
curl -sSfL https://raw.githubusercontent.com/docker/scout-cli/main/install.sh | sh -s --
Verify version:
docker scout version
...
version: v1.16.1 (go1.23.4 - darwin/arm64)
git commit: 987ee4d80c01bae92c78ddd44e1e028762a1f29b
Perfect, we have the latest available version.
go install golang.org/x/vuln/cmd/govulncheck@latest
Next, verify the installed version and check when the vulnerability database was updated.
govulncheck -version
Go: go1.23.4
Scanner: [email protected]
DB: https://vuln.go.dev
DB updated: 2024-12-20 21:48:20 +0000 UTC
The first line is the Go version installed on our machine, and the second is the latest govulncheck version.
The third and fourth lines are the Go vulnerability database maintained by the Go team. The team maintains an automated pipeline that analyses and adds vulnerability reports to the database. The public database allows access to libraries and tools to read and analyse the CVE reports.
Demystifying CVE security alerts
As the Go announcement mailing list subscribers, we receive daily emails about security issues discovered, including issues in the Go standard library and other Go packages.
As soon as we learn about security flaws, we can start planning patching activities and application deployments. We may also need to inform users and clients about upcoming updates.
That is exactly what happened when we were notified about flaws discovered in two Go packages: golang.org/x/net and golang.org/x/crypto.
How can we handle security updates locally? Well, it depends on what type of artifact you build and deliver to your users. It can be source code, a binary, or OS packages like rpm or deb. It can also be a Docker image published in the Docker Hub or a private registry.
If you package your Go application in a Docker image, the first line of security defence is vulnerability scanners. They will check the content of the image, including your Go binaries.
Let’s see how you can navigate the scanning, verification and patching processes.
Scanning Docker images
We will use the open source Nginx Ingress Controller in all examples.
You can clone the ingress controller git repository or use a different Go module whose dependencies are affected by security issuess. The patching process works the same.
git clone [email protected]:nginxinc/kubernetes-ingress.git
Let’s scan the image. Since we are only interested in critical and high vulnerabilities, we filter results using the --only-severity critical,high option. The last argument is the docker image stored in Docker Hub (name and tag) we want to scan.
docker scout cves --only-severity critical,high nginx/nginx-ingress:3.7.2
After a few seconds, our report is ready:
✓ SBOM of image already cached, 333 packages indexed
✗ Detected 2 vulnerable packages with a total of 2 vulnerabilities
## Overview
│ Analyzed Image
────────────────────┼─────────────────────────────────────────────────
Target │ nginx/nginx-ingress:3.7.2
digest │ e96bb172393b
platform │ linux/arm64
provenance │ https://github.com/nginxinc/kubernetes-ingress
│ 01c87ada043161545d2d3e0b9f8c53fe0985c479
vulnerabilities │ 1C 1H 0M 0L
size │ 106 MB
packages │ 333
## Packages and Vulnerabilities
1C 0H 0M 0L golang.org/x/crypto 0.28.0
pkg:golang/golang.org/x/[email protected]
✗ CRITICAL CVE-2024-45337 [Improper Authorization]
https://scout.docker.com/v/CVE-2024-45337
Affected range : <0.31.0
Fixed version : 0.31.0
CVSS Score : 9.1
CVSS Vector : CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N
0C 1H 0M 0L golang.org/x/net 0.30.0
pkg:golang/golang.org/x/[email protected]
✗ HIGH CVE-2024-45338 [Allocation of Resources Without Limits or Throttling]
https://scout.docker.com/v/CVE-2024-45338
Affected range : <0.33.0
Fixed version : 0.33.0
CVSS Score : 8.7
CVSS Vector : CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N
2 vulnerabilities found in 2 packages
CRITICAL 1
HIGH 1
MEDIUM 0
LOW 0
That’s a lot to unpack. Let’s focus on the most important parts.
We have two vulnerabilities. Both of them affect Go packages, which means that our Go application imports two dependencies that may potentially be dangerous.
pkg:golang/golang.org/x/[email protected]
pkg:golang/golang.org/x/[email protected]
Next, we need to verify if the issues have already been fixed. Let’s check the report again.
pkg:golang/golang.org/x/[email protected]
Affected range : <0.31.0
Fixed version : 0.31.0
pkg:golang/golang.org/x/[email protected]
Affected range : <0.33.0
Fixed version : 0.33.0
Perfect! Scout points us to package versions that include fixes for the security issues. This is crucial information for us. We can update the packages and remediate potential security problems.
What can we do right now? Our customers demand answers. They may panic or scratch their heads and think about what to do. Their security dashboards can scream in red: “Critical Vulnerabilities Found” or something similar.
What can be worse? Some application delivery mechanisms block dockerised application deployment! For our end users, this means they are stuck and cannot use the new shiny features we promised! It’s not an ideal situation.
We know that the issues are fixed and can get ready to update our application. But before that, we need to uncover the meaning of the word potential, which we used twice. Why potential security issues when vulnerability scanners clearly warn about critical and high-impact CVEs? Is the application and the production environment at risk?
Imagine you just moved into a new house. You plan how to arrange interiors, exteriors, and small garden, what furniture to buy and how to make your new place cosy and warm. You buy new kitchen appliances, a kettle, a coffee machine, a cooker, an oven, a grill, and a dozen other things.
You noticed that one of the small electric devices you bought is faulty. Some strange sound comes out from the inside when you move it. You suspect some cables inside are not correctly welded. They move and can cause electric shock.
But this device still sits unpacked. Brand new. Wrapped with a bubble foil.
Does this faulty device pose any risk to you, your family, your kitchen, or your house now?
Your next step is clear. You must check if the security issues translate to the security risk.
You have to check if the Go application uses the affected code.
If it does, you are at risk. If it doesn’t, you can take a deep breath and plan updates without unnecessary urgency and stress.
Scanning application source code
Earlier, we scanned the kubernetes-ingress docker image version, which was v3.7.2. Now, we navigate to the project’s root directory and create a new branch, security-test-3.7.2, from the git tag v3.7.2.
git checkout -b security-test-3.7.2 v3.7.2
Currently, your local repository matches the scanned nginx-ingress Go binary installed in the docker image version v3.7.2. We have everything in place and can finally use govulncheck to “X-Ray” the source code.
govulncheck
=== Symbol Results ===
No vulnerabilities found.
Your code is affected by 0 vulnerabilities.
This scan also found 0 vulnerabilities in packages you import and 2
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.
Bingo! In a matter of seconds, you know what is going on. The project is not directly impacted by vulnerabilities. That’s good news! However, you see two issues found in third-party dependencies.
What to do now? We will verify what packages are affected and whether new fixed versions are already available.
What information can we get from the enhanced report? Let’s run the govulncheck command with the -show verbose option.
govulncheck -show verbose ./...
Scanning your code and 1110 packages across 104 dependent modules for known vulnerabilities...
Fetching vulnerabilities from the database...
Checking the code against the vulnerabilities...
=== Symbol Results ===
No vulnerabilities found.
=== Package Results ===
No other vulnerabilities found.
=== Module Results ===
Vulnerability #1: GO-2024-3333
Non-linear parsing of case-insensitive content in golang.org/x/net/html
More info: https://pkg.go.dev/vuln/GO-2024-3333
Module: golang.org/x/net
Found in: golang.org/x/[email protected]
Fixed in: golang.org/x/[email protected]
Vulnerability #2: GO-2024-3321
Misuse of ServerConfig.PublicKeyCallback may cause authorization bypass in
golang.org/x/crypto
More info: https://pkg.go.dev/vuln/GO-2024-3321
Module: golang.org/x/crypto
Found in: golang.org/x/[email protected]
Fixed in: golang.org/x/[email protected]
Your code is affected by 0 vulnerabilities.
This scan also found 0 vulnerabilities in packages you import and 2
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
The verbose report gives you insights into vulnerability details. It looks similar to the one we got from scanning the docker image.
What is the most important takeaway from the report?
This scan also found 0 vulnerabilities in packages you import and 2 vulnerabilities in modules you require, but your code doesn’t appear to call these vulnerabilities.
Our Go application does not use vulnerable code!
Upgrading packages
You start bringing new package versions to your local machine without wasting time.
go get golang.org/x/[email protected]
Excellent! The go tool displays what is happening. New versions are on the way to your git branch.
go: downloading golang.org/x/net v0.33.0
go: upgraded golang.org/x/crypto v0.28.0 => v0.31.0
go: upgraded golang.org/x/net v0.30.0 => v0.33.0
go: upgraded golang.org/x/sync v0.9.0 => v0.10.0
go: upgraded golang.org/x/sys v0.27.0 => v0.28.0
go: upgraded golang.org/x/term v0.25.0 => v0.27.0
go: upgraded golang.org/x/text v0.20.0 => v0.21.0
You confirm updates and run git diff. Here you are! You get precise information about what was added (+) and what was removed (-) from the project’s dependencies.
diff --git a/go.mod b/go.mod
index 8a1325b3a..e879243e3 100644
--- a/go.mod
+++ b/go.mod
@@ -120,14 +120,14 @@ require (
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
- golang.org/x/crypto v0.28.0 // indirect
+ golang.org/x/crypto v0.31.0 // indirect
golang.org/x/mod v0.21.0 // indirect
- golang.org/x/net v0.30.0 // indirect
+ golang.org/x/net v0.33.0 // indirect
golang.org/x/oauth2 v0.23.0 // indirect
...
You brought the new golang.org/x/net version to the project, and it turns out that the package automatically brought along one of our culprits, the golang.org/x/crypto!
You run go mod tidy to update all direct and indirect dependencies. The command ensures that the go.mod file matches the source code in our project (module). It adds any missing module requirements necessary to build the current module’s packages and dependencies and removes requirements on modules that don’t provide relevant packages. It also adds missing entries to go.sum and removes unnecessary entries.
You are close to the finish line. You take the last remaining step and run govulncheck again.
govulncheck -show verbose ./...
Scanning your code and 1110 packages across 104 dependent modules for known vulnerabilities...
Fetching vulnerabilities from the database...
Checking the code against the vulnerabilities...
No vulnerabilities found.
You add and commit updated go.mod and go.sum files.
git status
On branch security-test-3.7.2
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: go.mod
modified: go.sum
...
Hooray! You declare victory. CVEs are no more!
Scanning Go binaries
What to do when we don’t have access to the source code? What should we do if all that we have is a Go binary? You will not be able to update packages. What you still can do is to answer the most critical question:
Does the Go application use the affected code?
You answer the question and run govulncheck with a specific option allowing to introspect binaries!
govulncheck -mode binary -show verbose nginx-ingress
You need only add the option -mode binary and point to the Go binary, as in the example above.
Think of some Go programs packaged in a container. You got a warning about potentially dangerous CVEs. To check if the code is safe, copy the binary from the inside container to your local machine, run govulncheck with the mode binary, and verify if the Go code uses vulnerable packages. Super useful!
Taking next steps
In the article, we went through the manual security patching cycle. You learned how to discover vulnerabilities in Go code and Docker images, how to scan Go source code and Go binaries, and how to update dependencies. You know how to verify if the security issues affect your application and if they introduce security risk.
“Security is not a one-time event. It’s an ongoing process.” – John Malloy
The security testing and patching processes can and should be automated and integrated into your continuous integration pipeline.
Suppose you or your company want to participate in the Open Source Security Foundation’s Best Practices Program and add the security badge to your projects in GitHub or GitLab. In that case, you must automate vulnerability scanning processes and follow specific security guidelines. I will cover security test automation in upcoming articles.
Open Source Software developers, maintainers and organizations should pay close attention to the new security regulations adopted in the European Union. This Linux Foundation’s presentation: The EU Cyber Resilience Act: A New Era for Business Engagement in Open Source Software highlights key points and demonstrates legal requirements for projects and project owners.
If you want to learn more about testing, security and benchmarking in Go, sign up for my mailing list. You will get notified about upcoming books and courses covering the following topics:
- Performance Tuning and Benchmarking in Go
- Go Testing Best Practices
- Go for Cybersecurity
Until next time!