[PR #2933] [MERGED] hscontrol: log acme/autocert errors #2942

Closed
opened 2025-12-29 04:19:45 +01:00 by adam · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/juanfont/headscale/pull/2933
Author: @dustymabe
Created: 12/4/2025
Status: Merged
Merged: 12/8/2025
Merged by: @kradalby

Base: mainHead: dusty-log-acme-errors


📝 Commits (1)

  • 53b7cf5 hscontrol: log acme/autocert errors

📊 Changes

1 file changed (+31 additions, -0 deletions)

View changed files

📝 hscontrol/app.go (+31 -0)

📄 Description

My ACME renewals weren't working at all and I had no idea why from looking at the logs of headscale today. This ListenAndServe() seems to not be outputting any logs from the go func().

If I added some debug logging via Environment=GODEBUG=http2debug=1 I could see more information:

2025/11/21 13:11:01 http2: Transport received DATA flags=END_STREAM stream=103 len=520 data="{\n  \"type\": \"urn:ietf:params:acme:error:rateLimited\",\n  \"detail\": \"Your account is temporarily prevented from requesting certificates for <redacted>.com and possibly others. Please visit: https://portal.letsencrypt.org/sfe/v1/unpause?jwt=123456789" (264 bytes omitted)

but as you can see (264 bytes omitted) from [1] made the message unactionable for me because it chopped off the unique ID I was supposed to use to unpause my ratelimiting.

Let's add add some middleware via http.RoundTripper to get the approprate log messages to help me understand what was going on. This is the resulting message I see:

2025-11-21T15:59:19-05:00 ERR ACME request returned error body="{\n  \"type\": \"urn:ietf:params:acme:error:rateLimited\",\n  \"detail\": \"Your account is temporarily prevented from requesting certificates for <redacted>.com and possibly others. Please visit: https://portal.letsencrypt.org/sfe/v1/unpause?jwt=0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012\",\n  \"status\": 429\n}" status_code=429 url=https://acme-v02.api.letsencrypt.org/acme/new-order

Assisted-By: <google/gemini-3-pro-preview>

Fixes: https://github.com/juanfont/headscale/issues/2911

[1] https://cs.opensource.google/go/go/+/master:src/net/http/h2_bundle.go;l=3274;drc=4b0e3cc1d63a00ee184ea1f6b17e79808e3d9fdc


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/juanfont/headscale/pull/2933 **Author:** [@dustymabe](https://github.com/dustymabe) **Created:** 12/4/2025 **Status:** ✅ Merged **Merged:** 12/8/2025 **Merged by:** [@kradalby](https://github.com/kradalby) **Base:** `main` ← **Head:** `dusty-log-acme-errors` --- ### 📝 Commits (1) - [`53b7cf5`](https://github.com/juanfont/headscale/commit/53b7cf5c2e77240f429786d8ad78c570cbbcb355) hscontrol: log acme/autocert errors ### 📊 Changes **1 file changed** (+31 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `hscontrol/app.go` (+31 -0) </details> ### 📄 Description My ACME renewals weren't working at all and I had no idea why from looking at the logs of headscale today. This ListenAndServe() seems to not be outputting any logs from the `go func()`. If I added some debug logging via `Environment=GODEBUG=http2debug=1` I could see more information: ``` 2025/11/21 13:11:01 http2: Transport received DATA flags=END_STREAM stream=103 len=520 data="{\n \"type\": \"urn:ietf:params:acme:error:rateLimited\",\n \"detail\": \"Your account is temporarily prevented from requesting certificates for <redacted>.com and possibly others. Please visit: https://portal.letsencrypt.org/sfe/v1/unpause?jwt=123456789" (264 bytes omitted) ``` but as you can see `(264 bytes omitted)` from [1] made the message unactionable for me because it chopped off the unique ID I was supposed to use to unpause my ratelimiting. Let's add add some middleware via http.RoundTripper to get the approprate log messages to help me understand what was going on. This is the resulting message I see: ``` 2025-11-21T15:59:19-05:00 ERR ACME request returned error body="{\n \"type\": \"urn:ietf:params:acme:error:rateLimited\",\n \"detail\": \"Your account is temporarily prevented from requesting certificates for <redacted>.com and possibly others. Please visit: https://portal.letsencrypt.org/sfe/v1/unpause?jwt=0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012\",\n \"status\": 429\n}" status_code=429 url=https://acme-v02.api.letsencrypt.org/acme/new-order ``` Assisted-By: <google/gemini-3-pro-preview> Fixes: https://github.com/juanfont/headscale/issues/2911 [1] https://cs.opensource.google/go/go/+/master:src/net/http/h2_bundle.go;l=3274;drc=4b0e3cc1d63a00ee184ea1f6b17e79808e3d9fdc --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
adam added the pull-request label 2025-12-29 04:19:45 +01:00
adam closed this issue 2025-12-29 04:19:45 +01:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/headscale#2942