starred/headscale
1
0
Fork 0
mirror of https://github.com/juanfont/headscale.git synced 2026-01-13 12:50:32 +01:00
Code Issues 98 Packages Projects Releases 10 Wiki Activity

[Bug] Calling routes API endpoint crashes with SIGSEGV #909

New Issue
Closed
opened 2025-12-29 02:25:47 +01:00 by adam · 1 comment
adam commented 2025-12-29 02:25:47 +01:00
Owner
Copy Link

Originally created by @strobeltobias on GitHub (Jan 18, 2025).

Is this a support request?

  • This is not a support request

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Headscale crashes with SIGSEGV when the API endpoint https://example.com/api/v1/routes is called.

Log output of headscale running in trace mode:

headscale  | 2025-01-18T15:59:50Z TRC home/runner/work/headscale/headscale/hscontrol/app.go:382 > HTTP authentication invoked client_address=172.20.2.12:45804
headscale  | panic: runtime error: invalid memory address or nil pointer dereference
headscale  | [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xce9bc2]
headscale  | 
headscale  | goroutine 445 [running]:
headscale  | github.com/juanfont/headscale/hscontrol/types.(*Node).Proto(0x0)
headscale  | 	/home/runner/work/headscale/headscale/hscontrol/types/node.go:239 +0x22
headscale  | github.com/juanfont/headscale/hscontrol/types.Routes.Proto({0xc0003acc08?, 0x0?, 0x2294470?})
headscale  | 	/home/runner/work/headscale/headscale/hscontrol/types/routes.go:82 +0xbd
headscale  | github.com/juanfont/headscale/hscontrol.headscaleV1APIServer.GetRoutes({{}, 0x410c00?}, {0x28?, 0x1f95860?}, 0xc0002bc170?)
headscale  | 	/home/runner/work/headscale/headscale/hscontrol/grpcv1.go:558 +0x36
headscale  | github.com/juanfont/headscale/gen/go/headscale/v1._HeadscaleService_GetRoutes_Handler({0x2142b20, 0xc00050e4a0}, {0x25482d8, 0xc00048b140}, 0xc000a2c080, 0x0)
headscale  | 	/home/runner/work/headscale/headscale/gen/go/headscale/v1/headscale_grpc.pb.go:806 +0x1a9
headscale  | google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003fee00, {0x25482d8, 0xc00048b0e0}, 0xc000698240, 0xc0004390b0, 0x36a3118, 0x0)
headscale  | 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1392 +0xfc3
headscale  | google.golang.org/grpc.(*Server).handleStream(0xc0003fee00, {0x2548a98, 0xc000234000}, 0xc000698240)
headscale  | 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1802 +0xbaa
headscale  | google.golang.org/grpc.(*Server).serveStreams.func2.1()
headscale  | 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1030 +0x7f
headscale  | created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 90
headscale  | 	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1041 +0x125
headscale exited with code 0
headscale  | 2025-01-18T15:59:51Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite

Expected Behavior

The API endpoint /api/v1/routes returns the expected response instead of responding with "Bad Gateway".

Steps To Reproduce

Call the API endpoint with a valid api key:

user@host:~$ curl -X GET 'https://headscale.example.com/api/v1/routes' -H 'Authorization: Bearer <redacted>'`
Bad Gateway

Environment

- OS: Docker on ArchLinux
- Headscale version: v0.24.0
- Headplane version: v0.4.1

Runtime environment

  • Headscale is behind a (reverse) proxy
  • Headscale runs in a container

Anything else?

I noticed this when the tab "Machines" of the web UI Headplane showed "Unexpected Server Error". After further investigation it receives the "Bad Gateway" response from Headscale when calling the routes endpoint I was able to reproduce this behavior when calling the endpoint manually using curl.

Originally created by @strobeltobias on GitHub (Jan 18, 2025). ### Is this a support request? - [x] This is not a support request ### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior Headscale crashes with SIGSEGV when the API endpoint https://example.com/api/v1/routes is called. Log output of headscale running in trace mode: ``` headscale | 2025-01-18T15:59:50Z TRC home/runner/work/headscale/headscale/hscontrol/app.go:382 > HTTP authentication invoked client_address=172.20.2.12:45804 headscale | panic: runtime error: invalid memory address or nil pointer dereference headscale | [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xce9bc2] headscale | headscale | goroutine 445 [running]: headscale | github.com/juanfont/headscale/hscontrol/types.(*Node).Proto(0x0) headscale | /home/runner/work/headscale/headscale/hscontrol/types/node.go:239 +0x22 headscale | github.com/juanfont/headscale/hscontrol/types.Routes.Proto({0xc0003acc08?, 0x0?, 0x2294470?}) headscale | /home/runner/work/headscale/headscale/hscontrol/types/routes.go:82 +0xbd headscale | github.com/juanfont/headscale/hscontrol.headscaleV1APIServer.GetRoutes({{}, 0x410c00?}, {0x28?, 0x1f95860?}, 0xc0002bc170?) headscale | /home/runner/work/headscale/headscale/hscontrol/grpcv1.go:558 +0x36 headscale | github.com/juanfont/headscale/gen/go/headscale/v1._HeadscaleService_GetRoutes_Handler({0x2142b20, 0xc00050e4a0}, {0x25482d8, 0xc00048b140}, 0xc000a2c080, 0x0) headscale | /home/runner/work/headscale/headscale/gen/go/headscale/v1/headscale_grpc.pb.go:806 +0x1a9 headscale | google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003fee00, {0x25482d8, 0xc00048b0e0}, 0xc000698240, 0xc0004390b0, 0x36a3118, 0x0) headscale | /home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1392 +0xfc3 headscale | google.golang.org/grpc.(*Server).handleStream(0xc0003fee00, {0x2548a98, 0xc000234000}, 0xc000698240) headscale | /home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1802 +0xbaa headscale | google.golang.org/grpc.(*Server).serveStreams.func2.1() headscale | /home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1030 +0x7f headscale | created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 90 headscale | /home/runner/go/pkg/mod/google.golang.org/grpc@v1.69.0/server.go:1041 +0x125 headscale exited with code 0 headscale | 2025-01-18T15:59:51Z INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite ``` ### Expected Behavior The API endpoint `/api/v1/routes` returns the expected response instead of responding with "Bad Gateway". ### Steps To Reproduce Call the API endpoint with a valid api key: ```shell user@host:~$ curl -X GET 'https://headscale.example.com/api/v1/routes' -H 'Authorization: Bearer <redacted>'` Bad Gateway ``` ### Environment ```markdown - OS: Docker on ArchLinux - Headscale version: v0.24.0 - Headplane version: v0.4.1 ``` ### Runtime environment - [x] Headscale is behind a (reverse) proxy - [x] Headscale runs in a container ### Anything else? I noticed this when the tab "Machines" of the web UI Headplane showed "Unexpected Server Error". After further investigation it receives the "Bad Gateway" response from Headscale when calling the routes endpoint I was able to reproduce this behavior when calling the endpoint manually using curl.
adam added the bug label 2025-12-29 02:25:47 +01:00
adam closed this issue 2025-12-29 02:25:47 +01:00
adam commented 2025-12-29 02:25:47 +01:00
Author
Owner
Copy Link

@strobeltobias commented on GitHub (Jan 18, 2025):

I was curious what's in the routes table and looked into it using sqlite3. By clearing the table manually and removing and re-adding the nodes advertising routes I was able to fix this.

So it probably was a bad route without a valid node, i.e. a route that was not cleaned up after removing the node it belongs to?
As I was removing old nodes earlier via the headscale cli:

docker exec headscale headscale nodes delete -i 2

If this needs no further investigation from this point on, feel free to close the issue.

@strobeltobias commented on GitHub (Jan 18, 2025): I was curious what's in the routes table and looked into it using sqlite3. By clearing the table manually and removing and re-adding the nodes advertising routes I was able to fix this. So it probably was a bad route without a valid node, i.e. a route that was not cleaned up after removing the node it belongs to? As I was removing old nodes earlier via the headscale cli: ```shell docker exec headscale headscale nodes delete -i 2 ``` If this needs no further investigation from this point on, feel free to close the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
CLI
DERP
DNS
Nix
OIDC
SSH
bug
database
documentation
duplicate
enhancement
faq
good first issue
grants
help wanted
might-come
needs design doc
needs investigation
no-stale-bot
out of scope
performance
policy 📝
pull-request
Mirrored from GitHub Pull Request
 question
regression
routes
stale
tags
tailscale-feature-gap
well described ❤️
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#909
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?