Letsencrypt Certificates #10

Closed
opened 2025-12-29 09:21:37 +01:00 by adam · 11 comments
Owner

Originally created by @zQueal on GitHub (Sep 16, 2024).

I've been testing the proxy and have been continually starting and stopping the binary which has landed me in the dog house;

ERRO[09-15 20:27:05] failed to renew certificate: failed to obtain certificate:
  - acme: error: 429 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: Error creating new order :: too many certificates (5) already issued for this exact set of domains in the last 168 hours: {domains}, retry after 2024-09-17T08:39:24Z: see https://letsencrypt.org/docs/duplicate-certificate-limit/

So it looks like I've been banned from letsencrypt for 168 hours.

==========================

Would it perhaps be beneficial to add a --staging flag to use the letsencrypt staging server (acme-staging-v02) instead of live (acme-v02.api)? This would increase the new order certificates to 1,500 new orders per 3 hour period versus the 5 which you get from live.

Or maybe just logic which searches for cert.crt and priv.key and foregoes grabbing another cert if they're found?

Appreciate your work. Loving the proxy so far.

Originally created by @zQueal on GitHub (Sep 16, 2024). I've been testing the proxy and have been continually starting and stopping the binary which has landed me in the dog house; ```cli ERRO[09-15 20:27:05] failed to renew certificate: failed to obtain certificate: - acme: error: 429 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: Error creating new order :: too many certificates (5) already issued for this exact set of domains in the last 168 hours: {domains}, retry after 2024-09-17T08:39:24Z: see https://letsencrypt.org/docs/duplicate-certificate-limit/ ``` So it looks like I've been banned from letsencrypt for 168 hours. ========================== Would it perhaps be beneficial to add a `--staging` flag to use the letsencrypt [staging server](https://letsencrypt.org/docs/staging-environment/) (`acme-staging-v02`) instead of live (`acme-v02.api`)? This would increase the new order certificates to 1,500 new orders per 3 hour period versus the 5 which you get from live. Or maybe just logic which searches for `cert.crt` and `priv.key` and foregoes grabbing another cert if they're found? Appreciate your work. Loving the proxy so far.
adam closed this issue 2025-12-29 09:21:37 +01:00
Author
Owner

@yusing commented on GitHub (Sep 16, 2024):

please see compose.example.yml, you should mount the ./certs folder to container, in order to store obtained certs: "- ./certs:/app/certs". And happy to inform you that, v0.5-rc1 is out, I may consider this version is way better than v0.4. You may want to give it a try

@yusing commented on GitHub (Sep 16, 2024): please see [compose.example.yml](https://github.com/yusing/go-proxy/blob/v0.4/compose.example.yml), you should mount the ./certs folder to container, in order to store obtained certs: "- ./certs:/app/certs". And happy to inform you that, v0.5-rc1 is out, I may consider this version is way better than v0.4. You may want to give it a try
Author
Owner

@zQueal commented on GitHub (Sep 16, 2024):

I'm not using docker and built from git, so I'm using the v0.5 branch which is the default--not sure if this is what you meant or not.

So what I've been experiencing is running the go-proxy binary will pull new certificates on every runtime because autocert is set in the config regardless of whether or not I already have certs located in certs/ or not. This is a bit counter productive, IMO. The proxy should check certs/ and if there is a key/crt pair, it should not run autocert when you launch the binary.

As it stands, it processes a neworder to letsencrypt even though you already have a valid cert.


❯ tree -I "docs|schema|templates"
.
├── certs
│   ├── cert.crt
│   └── priv.key
├── config
│   ├── config.yml
│   └── domain.yml
└── go-proxy
@zQueal commented on GitHub (Sep 16, 2024): I'm not using docker and built from git, so I'm using the v0.5 branch which is the default--not sure if this is what you meant or not. So what I've been experiencing is running the go-proxy binary will pull new certificates on every runtime because `autocert` is set in the config regardless of whether or not I already have certs located in `certs/` or not. This is a bit counter productive, IMO. The proxy should check `certs/` and if there is a key/crt pair, it should *not* run `autocert` when you launch the binary. As it stands, it processes a neworder to letsencrypt even though you already have a valid cert. ------------------- ```cli ❯ tree -I "docs|schema|templates" . ├── certs │   ├── cert.crt │   └── priv.key ├── config │   ├── config.yml │   └── domain.yml └── go-proxy ```
Author
Owner

@yusing commented on GitHub (Sep 16, 2024):

If there are certs already in certs/, autocert will still run and check for existing cert and its expiry date. It should not request a new cert if the existing one is not expired, I'll check the code again to see if there's a bug.

If it is still trying to obtain a new cert, check the working directory when you run go-proxy.

@yusing commented on GitHub (Sep 16, 2024): If there are certs already in certs/, autocert will still run and check for existing cert and its expiry date. It should not request a new cert if the existing one is not expired, I'll check the code again to see if there's a bug. If it is still trying to obtain a new cert, check the working directory when you run go-proxy.
Author
Owner

@zQueal commented on GitHub (Sep 16, 2024):

It should not request a new cert

This seems to be what I'm experiencing.

I pulled the repo to /opt/proxy and ran the setup. I pulled the initial certificates and they are in /opt/proxy/certs/, but each time I run ./go-proxy it query's letsencrypt for new certificates.

@zQueal commented on GitHub (Sep 16, 2024): > It should not request a new cert This seems to be what I'm experiencing. I pulled the repo to `/opt/proxy` and ran the setup. I pulled the initial certificates and they are in `/opt/proxy/certs/`, but each time I run `./go-proxy` it query's letsencrypt for new certificates.
Author
Owner

@yusing commented on GitHub (Sep 16, 2024):

I pulled the repo to /opt/proxy and ran the setup. I pulled the initial certificates and they are in /opt/proxy/certs/, but each time I run ./go-proxy it query's letsencrypt for new certificates.

Sorry for your bad experience, please give me some time to check and fix. Thank you for trying out go-proxy.

@yusing commented on GitHub (Sep 16, 2024): > I pulled the repo to `/opt/proxy` and ran the setup. I pulled the initial certificates and they are in `/opt/proxy/certs/`, but each time I run `./go-proxy` it query's letsencrypt for new certificates. Sorry for your bad experience, please give me some time to check and fix. Thank you for trying out `go-proxy`.
Author
Owner

@zQueal commented on GitHub (Sep 16, 2024):

It's been a really great experience so far. Really loving the project!

@zQueal commented on GitHub (Sep 16, 2024): It's been a really great experience so far. Really loving the project!
Author
Owner

@yusing commented on GitHub (Sep 16, 2024):

Hi, I have just tested it out and did not face the same issue as yours,

image

Above is the second time starting go-proxy (using cert obtained from the first time).

And my config is:

autocert:
  email: "yusing@domain.tld"
  domains:
    - "domain.tld"
    - "*.domain.tld"
  provider: cloudflare
  options:
    auth_token: <token>
...
@yusing commented on GitHub (Sep 16, 2024): Hi, I have just tested it out and did not face the same issue as yours, ![image](https://github.com/user-attachments/assets/b90a0b9a-0fdd-4e97-8c25-efd163aed2e4) Above is the second time starting `go-proxy` (using cert obtained from the first time). And my config is: ```yaml autocert: email: "yusing@domain.tld" domains: - "domain.tld" - "*.domain.tld" provider: cloudflare options: auth_token: <token> ... ```
Author
Owner

@zQueal commented on GitHub (Sep 16, 2024):

I removed the entire environment and started over from a fresh git pull. I wasn't able to replicate the previous error, or rather I should say I can't get that far anymore;

❯ ./go-proxy
ERRO[09-16 16:26:41] open certs/cert.crt: no such file or directory  module=main
INFO[09-16 16:26:41] Now attempting to obtain a new certificate...  module=main
2024/09/16 16:26:41 [INFO] acme: Registering account for my.email@domain.dev
2024/09/16 16:26:41 [INFO] [hostname.dev, *.hostname.dev] acme: Obtaining bundled SAN certificate
INFO[09-16 16:26:41] starting http proxy server on :80
INFO[09-16 16:26:41] starting http api server on :8888
panic: no certificate available

goroutine 41 [running]:
github.com/yusing/go-proxy/autocert.(*Provider).ShouldRenewOn(...)
        /opt/proxy/src/autocert/provider.go:115
github.com/yusing/go-proxy/autocert.(*Provider).ScheduleRenewal(0xc00044a180, {0xae24a0, 0xc00008c1e0})
        /opt/proxy/src/autocert/provider.go:133 +0x446
created by main.main in goroutine 1
        /opt/proxy/src/main.go:90 +0xd72

config.yml;

autocert:
  email: "my.email@hostname.dev"
  domains:
    - "hostname.dev"
    - "*.hostname.dev"
  provider: cloudflare
  options:
    auth_token: "token"
providers:
  include:
    - providers.yml

providers.yml;

hostname:
  host: {ip}
  port: 9010
docker:
  host: {ip}
  port: 9001

Do you see anything that jumps out at you? I'm at a loss.

@zQueal commented on GitHub (Sep 16, 2024): I removed the entire environment and started over from a fresh git pull. I wasn't able to replicate the previous error, or rather I should say I can't get that far anymore; ``` ❯ ./go-proxy ERRO[09-16 16:26:41] open certs/cert.crt: no such file or directory module=main INFO[09-16 16:26:41] Now attempting to obtain a new certificate... module=main 2024/09/16 16:26:41 [INFO] acme: Registering account for my.email@domain.dev 2024/09/16 16:26:41 [INFO] [hostname.dev, *.hostname.dev] acme: Obtaining bundled SAN certificate INFO[09-16 16:26:41] starting http proxy server on :80 INFO[09-16 16:26:41] starting http api server on :8888 panic: no certificate available goroutine 41 [running]: github.com/yusing/go-proxy/autocert.(*Provider).ShouldRenewOn(...) /opt/proxy/src/autocert/provider.go:115 github.com/yusing/go-proxy/autocert.(*Provider).ScheduleRenewal(0xc00044a180, {0xae24a0, 0xc00008c1e0}) /opt/proxy/src/autocert/provider.go:133 +0x446 created by main.main in goroutine 1 /opt/proxy/src/main.go:90 +0xd72 ``` ----------------- config.yml; ```yml autocert: email: "my.email@hostname.dev" domains: - "hostname.dev" - "*.hostname.dev" provider: cloudflare options: auth_token: "token" providers: include: - providers.yml ``` providers.yml; ```yml hostname: host: {ip} port: 9010 docker: host: {ip} port: 9001 ``` Do you see anything that jumps out at you? I'm at a loss.
Author
Owner

@yusing commented on GitHub (Sep 17, 2024):

I've checked my code for autocert, there are a few bugs. I'm working on it, should inform you in case they're fixed.

@yusing commented on GitHub (Sep 17, 2024): I've checked my code for autocert, there are a few bugs. I'm working on it, should inform you in case they're fixed.
Author
Owner

@yusing commented on GitHub (Sep 17, 2024):

Hi, it should be fixed, please git pull again and test it out.

@yusing commented on GitHub (Sep 17, 2024): Hi, it should be fixed, please `git pull` again and test it out.
Author
Owner

@zQueal commented on GitHub (Sep 17, 2024):

OK. I've pulled the new tag, and built with go.

First run;

2024/09/17 10:25:11 [INFO] acme: Registering account for my.email@domain.dev
2024/09/17 10:25:11 [INFO] [domain.dev, *.domain.dev] acme: Obtaining bundled SAN certificate
2024/09/17 10:25:11 [INFO] [*.domain.dev] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/404829716146
2024/09/17 10:25:11 [INFO] [domain.dev] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/404829716156
2024/09/17 10:25:11 [INFO] [*.domain.dev] acme: use dns-01 solver
2024/09/17 10:25:11 [INFO] [domain.dev] acme: Could not find solver for: tls-alpn-01
2024/09/17 10:25:11 [INFO] [domain.dev] acme: Could not find solver for: http-01
2024/09/17 10:25:11 [INFO] [domain.dev] acme: use dns-01 solver
2024/09/17 10:25:11 [INFO] [*.domain.dev] acme: Preparing to solve DNS-01
2024/09/17 10:25:12 [INFO] cloudflare: new record for domain.dev, ID 83b09b9098c57b78f0faecb72522de1f
2024/09/17 10:25:12 [INFO] [domain.dev] acme: Preparing to solve DNS-01
2024/09/17 10:25:12 [INFO] cloudflare: new record for domain.dev, ID 4af5a69873a0f4df0876745492d4342e
2024/09/17 10:25:12 [INFO] [*.domain.dev] acme: Trying to solve DNS-01
2024/09/17 10:25:12 [INFO] [*.domain.dev] acme: Checking DNS record propagation. [nameservers=127.0.0.53:53]
2024/09/17 10:25:14 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2024/09/17 10:25:15 [INFO] [*.domain.dev] acme: Waiting for DNS record propagation.
2024/09/17 10:25:17 [INFO] [*.domain.dev] acme: Waiting for DNS record propagation.
2024/09/17 10:25:22 [INFO] [*.domain.dev] The server validated our request
2024/09/17 10:25:22 [INFO] [domain.dev] acme: Trying to solve DNS-01
2024/09/17 10:25:22 [INFO] [domain.dev] acme: Checking DNS record propagation. [nameservers=127.0.0.53:53]
2024/09/17 10:25:24 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2024/09/17 10:25:28 [INFO] [domain.dev] The server validated our request
2024/09/17 10:25:28 [INFO] [*.domain.dev] acme: Cleaning DNS-01 challenge
2024/09/17 10:25:29 [INFO] [domain.dev] acme: Cleaning DNS-01 challenge
2024/09/17 10:25:29 [INFO] [domain.dev, *.domain.dev] acme: Validations succeeded; requesting certificates
2024/09/17 10:25:32 [INFO] [domain.dev] Server responded with a certificate.
INFO[09-17 10:25:32] certificate "domain.dev": expire on 2024-12-16 13:26:58 +0000 UTC  module=main
INFO[09-17 10:25:32] certificate "*.domain.dev": expire on 2024-12-16 13:26:58 +0000 UTC  module=main
INFO[09-17 10:25:32] starting http proxy server on :80
INFO[09-17 10:25:32] starting https proxy server on :443
INFO[09-17 10:25:32] starting http api server on :8888

On second run, the previously pulled certificates were used. 👍🏻 Awesome!

@zQueal commented on GitHub (Sep 17, 2024): OK. I've pulled the new tag, and built with go. First run; ```cli 2024/09/17 10:25:11 [INFO] acme: Registering account for my.email@domain.dev 2024/09/17 10:25:11 [INFO] [domain.dev, *.domain.dev] acme: Obtaining bundled SAN certificate 2024/09/17 10:25:11 [INFO] [*.domain.dev] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/404829716146 2024/09/17 10:25:11 [INFO] [domain.dev] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/404829716156 2024/09/17 10:25:11 [INFO] [*.domain.dev] acme: use dns-01 solver 2024/09/17 10:25:11 [INFO] [domain.dev] acme: Could not find solver for: tls-alpn-01 2024/09/17 10:25:11 [INFO] [domain.dev] acme: Could not find solver for: http-01 2024/09/17 10:25:11 [INFO] [domain.dev] acme: use dns-01 solver 2024/09/17 10:25:11 [INFO] [*.domain.dev] acme: Preparing to solve DNS-01 2024/09/17 10:25:12 [INFO] cloudflare: new record for domain.dev, ID 83b09b9098c57b78f0faecb72522de1f 2024/09/17 10:25:12 [INFO] [domain.dev] acme: Preparing to solve DNS-01 2024/09/17 10:25:12 [INFO] cloudflare: new record for domain.dev, ID 4af5a69873a0f4df0876745492d4342e 2024/09/17 10:25:12 [INFO] [*.domain.dev] acme: Trying to solve DNS-01 2024/09/17 10:25:12 [INFO] [*.domain.dev] acme: Checking DNS record propagation. [nameservers=127.0.0.53:53] 2024/09/17 10:25:14 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s] 2024/09/17 10:25:15 [INFO] [*.domain.dev] acme: Waiting for DNS record propagation. 2024/09/17 10:25:17 [INFO] [*.domain.dev] acme: Waiting for DNS record propagation. 2024/09/17 10:25:22 [INFO] [*.domain.dev] The server validated our request 2024/09/17 10:25:22 [INFO] [domain.dev] acme: Trying to solve DNS-01 2024/09/17 10:25:22 [INFO] [domain.dev] acme: Checking DNS record propagation. [nameservers=127.0.0.53:53] 2024/09/17 10:25:24 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s] 2024/09/17 10:25:28 [INFO] [domain.dev] The server validated our request 2024/09/17 10:25:28 [INFO] [*.domain.dev] acme: Cleaning DNS-01 challenge 2024/09/17 10:25:29 [INFO] [domain.dev] acme: Cleaning DNS-01 challenge 2024/09/17 10:25:29 [INFO] [domain.dev, *.domain.dev] acme: Validations succeeded; requesting certificates 2024/09/17 10:25:32 [INFO] [domain.dev] Server responded with a certificate. INFO[09-17 10:25:32] certificate "domain.dev": expire on 2024-12-16 13:26:58 +0000 UTC module=main INFO[09-17 10:25:32] certificate "*.domain.dev": expire on 2024-12-16 13:26:58 +0000 UTC module=main INFO[09-17 10:25:32] starting http proxy server on :80 INFO[09-17 10:25:32] starting https proxy server on :443 INFO[09-17 10:25:32] starting http api server on :8888 ``` On second run, the previously pulled certificates were used. 👍🏻 Awesome!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/godoxy#10