[New Feature] - Enable putting dns challenge token creation inline with validation #317

New Issue

adam · 2025-12-29T01:22:27+01:00

adam commented

2025-12-29 01:22:27 +01:00

Originally created by @JavaScriptDude on GitHub (Mar 26, 2018).

Unfortunately, as a result of the issue of wildcard domain registration with their non-wildcard counterpart (see issue 504), I cannot use dehydrated as the DNS provider's api does not support multiple TXT entry creation. The DNS provider I have only supports this manually in their web GUI.

I am sure that there are several DNS providers that have this same issue with no solution available in the near future.

As a result, I recommend we add a new feature to allow the user to optionally put the challenge token deployment inline with the logic where the validation is done. This way the challenge token is deployed and is then immediately validated before the next token is deployed. I have hacked this up myself and it works like a charm but I'm sure others could benefit from this feature.

Originally created by @JavaScriptDude on GitHub (Mar 26, 2018). Unfortunately, as a result of the issue of wildcard domain registration with their non-wildcard counterpart ([see issue 504](https://github.com/lukas2511/dehydrated/issues/504)), I cannot use dehydrated as the DNS provider's api does not support multiple TXT entry creation. The DNS provider I have only supports this manually in their web GUI. I am sure that there are several DNS providers that have this same issue with no solution available in the near future. As a result, I recommend we add a new feature to allow the user to optionally put the challenge token deployment inline with the logic where the validation is done. This way the challenge token is deployed and is then immediately validated before the next token is deployed. I have hacked this up myself and it works like a charm but I'm sure others could benefit from this feature.

adam closed this issue

2025-12-29 01:22:27 +01:00

adam commented

2025-12-29 01:22:29 +01:00

@JavaScriptDude commented on GitHub (Mar 26, 2018):

For those who are interested, just comment out the "Deploy challenge tokens" block and add the following code to the # Validate pending challenges" immediately after the while ... do statement:

    # Deploy challenge token
    echo " + Deploying challenge token for ${challenge_names[${idx}]}..."
    "${HOOK}" "deploy_challenge" ${deploy_args[${idx}]}

Note: This hack will make HOOK_CHAIN option moot.

@JavaScriptDude commented on GitHub (Mar 26, 2018): For those who are interested, just comment out the "Deploy challenge tokens" block and add the following code to the # Validate pending challenges" immediately after the while ... do statement: ``` # Deploy challenge token echo " + Deploying challenge token for ${challenge_names[${idx}]}..." "${HOOK}" "deploy_challenge" ${deploy_args[${idx}]} ``` Note: This hack will make HOOK_CHAIN option moot.

adam commented

2025-12-29 01:22:29 +01:00

@jobe1986 commented on GitHub (Mar 26, 2018):

Out of curiosity which DNS provider are you using? Because the one mentioned in issue #504 mentions that the hook script used has been fixed to solve the issue.

Because ultimately it sounds to me that the hook script you're using incorrectly "replaces" the TXT record with the same name instead of adding a new one. Which also suggests to me a possibility that the author tried to save time not using the cleanup_challenge hook to remove old TXT records and just overwriting them every time instead.

@jobe1986 commented on GitHub (Mar 26, 2018): Out of curiosity which DNS provider are you using? Because the one mentioned in issue #504 mentions that the hook script used has been fixed to solve the issue. Because ultimately it sounds to me that the hook script you're using incorrectly "replaces" the TXT record with the same name instead of adding a new one. Which also suggests to me a possibility that the author tried to save time not using the cleanup_challenge hook to remove old TXT records and just overwriting them every time instead.

adam commented

2025-12-29 01:22:29 +01:00

@JavaScriptDude commented on GitHub (Mar 26, 2018):

I am stuck using godaddy as its what the customer has 95% of their domains on. From my research, the godaddy API does not allow setting multiple TXT entries with the same name. The only applicable API for creating TXT records that I see is:

[PUT] /v1/domains/{domain}/records/{type}/{name} "Replace all DNS Records for the specified Domain with the specified Type and Name"

If anyone can find a way to programatically set multiple TXT records with the same name in godaddy, please let me know.

@JavaScriptDude commented on GitHub (Mar 26, 2018): I am stuck using godaddy as its what the customer has 95% of their domains on. From my research, the [godaddy API](https://developer.godaddy.com/doc/endpoint/domains) does not allow setting multiple TXT entries with the same name. The only applicable API for creating TXT records that I see is: > [PUT] /v1/domains/{domain}/records/{type}/{name} "Replace all DNS Records for the specified Domain with the specified Type and Name" If anyone can find a way to programatically set multiple TXT records with the same name in godaddy, please let me know.

adam commented

2025-12-29 01:22:30 +01:00

@jobe1986 commented on GitHub (Mar 26, 2018):

GoDaddy's API does support adding instead of replacing too but it appears to me the hook script listed for GoDaddy on https://github.com/lukas2511/dehydrated/wiki/Examples-for-DNS-01-hooks may or may not be using the wrong method of the godaddypy python library to update the DNS thus performing an update/replace instead of an add.

Sadly I don't have the resources to try and patch and test said script (I don't use GoDaddy) otherwise I'd give it a try and test it.

@jobe1986 commented on GitHub (Mar 26, 2018): GoDaddy's API does support adding instead of replacing too but it appears to me the hook script listed for GoDaddy on https://github.com/lukas2511/dehydrated/wiki/Examples-for-DNS-01-hooks may or may not be using the wrong method of the godaddypy python library to update the DNS thus performing an update/replace instead of an add. Sadly I don't have the resources to try and patch and test said script (I don't use GoDaddy) otherwise I'd give it a try and test it.

adam commented

2025-12-29 01:22:30 +01:00

@JavaScriptDude commented on GitHub (Mar 26, 2018):

I was using lexicon which showed godaddy as supported in their docs. I will try the py library listed today to see if they have an "add" api hook.

@JavaScriptDude commented on GitHub (Mar 26, 2018): I was using lexicon which showed godaddy as supported in their docs. I will try the py library listed today to see if they have an "add" api hook.

adam commented

2025-12-29 01:22:33 +01:00

@txr13 commented on GitHub (Mar 26, 2018):

I saw mention of this when someone was developing GoDaddy support for https://github.com/AnalogJ/lexicon. IIRC, GoDaddy’s API erases all TXT records when you upload a new one? I seem to remember something to the effect of having to poll for existing records first and saving them before adding a new one, to work around that...

Is it possible that this poll/save/insert/put flow is malfunctioning here?

@txr13 commented on GitHub (Mar 26, 2018): I saw mention of this when someone was developing GoDaddy support for https://github.com/AnalogJ/lexicon. IIRC, GoDaddy’s API erases all TXT records when you upload a new one? I seem to remember something to the effect of having to poll for existing records first and saving them before adding a new one, to work around that... Is it possible that this poll/save/insert/put flow is malfunctioning here?

adam commented

2025-12-29 01:22:35 +01:00

@JavaScriptDude commented on GitHub (Mar 26, 2018):

I found the API hook in GoDaddy for adding multiple records. Lexicon does not use this Api call but its exposed in pythons core library:

from godaddypy import Client, Account
DOMAIN = "_domain_"
API_KEY = "_api-key_"
API_SECRET = "_api-secret_"
REC_NAME = "_acme-challenge"
client = Client(Account(api_key=API_KEY, api_secret=API_SECRET))
client.add_record(DOMAIN, {'data':'TEST_VAL_1', 'name':REC_NAME, 'ttl':600, 'type':'TXT'})
client.add_record(DOMAIN, {'data':'TEST_VAL_2', 'name':REC_NAME, 'ttl':600, 'type':'TXT'})
print(DOMAIN, ' Records listed (should include TEST_VAL_1 and TEST_VAL_2: ', client.get_records(DOMAIN, record_type='TXT'))

The scary thing I see right now is that I cannot safely delete records via API as godaddy does not expose a safe call to do this. Here is a note from the python core library def delete_records():

This is haphazard functionality. I DO NOT recommend using this in Production code, as your entire DNS record set could be deleted, depending on the fickleness of GoDaddy. Unfortunately, they do not expose a proper "delete record" call, so there isn't much one can do here...

I read the code at it is very unsafe and not recommended. I guess for godaddy we can just spam TXT records in using the godaddypy API and delete them in a more safe way manually at a future date.

I am on the fence regarding this feature request ticket. There may still be merit in the approach. I ran the script on just under 100 dns entries including wildcards using my hack at the top of this thread and it worked like a charm.

@JavaScriptDude commented on GitHub (Mar 26, 2018): I found the [API hook](https://developer.godaddy.com/doc/endpoint/domains#/v1/recordAdd) in GoDaddy for adding multiple records. [Lexicon](https://github.com/AnalogJ/lexicon) does not use this Api call but its exposed in pythons core library: ``` from godaddypy import Client, Account DOMAIN = "_domain_" API_KEY = "_api-key_" API_SECRET = "_api-secret_" REC_NAME = "_acme-challenge" client = Client(Account(api_key=API_KEY, api_secret=API_SECRET)) client.add_record(DOMAIN, {'data':'TEST_VAL_1', 'name':REC_NAME, 'ttl':600, 'type':'TXT'}) client.add_record(DOMAIN, {'data':'TEST_VAL_2', 'name':REC_NAME, 'ttl':600, 'type':'TXT'}) print(DOMAIN, ' Records listed (should include TEST_VAL_1 and TEST_VAL_2: ', client.get_records(DOMAIN, record_type='TXT')) ``` The scary thing I see right now is that I cannot safely delete records via API as godaddy does not expose a safe call to do this. Here is a note from the python core library def delete_records(): >This is haphazard functionality. I DO NOT recommend using this in Production code, as your entire DNS record set could be deleted, depending on the fickleness of GoDaddy. Unfortunately, they do not expose a proper "delete record" call, so there isn't much one can do here... I read the code at it is very unsafe and not recommended. I guess for godaddy we can just spam TXT records in using the godaddypy API and delete them in a more safe way manually at a future date. I am on the fence regarding this feature request ticket. There may still be merit in the approach. I ran the script on just under 100 dns entries including wildcards using my hack at the top of this thread and it worked like a charm.

adam commented

2025-12-29 01:22:35 +01:00

@lukas2511 commented on GitHub (Mar 26, 2018):

@JavaScriptDude There is an issue with both ways of deploying challenges. Deploying both at the same time could potentially be problematic with some APIs, deploying one after another might result in cached DNS entries on ACME Server-Side which would result in the second challenge to simply fail (as it still sees only the first entry, basically becoming an ugly race-condition).
So trying to get around one issue (mostly the hook-script not working correctly) would just end up in the next issue (cached dns entries on server-side).

Technically it shouldn't be a problem to have multiple records set. Up to now every ticket/mail/twitter- and irc-question similar to this could be solved by just patching the hook-script, and this one seems to be exactly the same.
Introducing a different behaviors with a config-option or whatever would just end up with people tinkering around until it somehow works and then creating a ticket here 60 days later when their certificate is going to be renewed but everything fails since it was never set up correctly to begin with.
Unless somebody comes up with an unavoidable scenario which really requires the different behavior I don't want to implement that again (this was actually the old behavior until the caching issues became obvious).

About the last part with the "dangerous" delete option. May I suggest to simply use the original "erroneous" replacement function and instead of deleting the TXT record replace it with a dummy record? I'd suggest setting it to dehydrated was here! 😏 Of course only do this if the deletion really is that problematic, I don't know their API.

@lukas2511 commented on GitHub (Mar 26, 2018): @JavaScriptDude There is an issue with both ways of deploying challenges. Deploying both at the same time could potentially be problematic with some APIs, deploying one after another might result in cached DNS entries on ACME Server-Side which would result in the second challenge to simply fail (as it still sees only the first entry, basically becoming an ugly race-condition). So trying to get around one issue (mostly the hook-script not working correctly) would just end up in the next issue (cached dns entries on server-side). Technically it shouldn't be a problem to have multiple records set. Up to now every ticket/mail/twitter- and irc-question similar to this could be solved by just patching the hook-script, and this one seems to be exactly the same. Introducing a different behaviors with a config-option or whatever would just end up with people tinkering around until it somehow works and then creating a ticket here 60 days later when their certificate is going to be renewed but everything fails since it was never set up correctly to begin with. Unless somebody comes up with an unavoidable scenario which _really_ requires the different behavior I don't want to implement that again (this was actually the old behavior until the caching issues became obvious). About the last part with the "dangerous" delete option. May I suggest to simply use the original "erroneous" replacement function and instead of deleting the TXT record replace it with a dummy record? I'd suggest setting it to `dehydrated was here!` :smirk: Of course only do this if the deletion really is that problematic, I don't know their API.

adam commented

2025-12-29 01:22:35 +01:00

@JavaScriptDude commented on GitHub (Mar 27, 2018):

@lukas2511 - Agreed. I spent the day working with le-godaddy-dns and only had to add a small tweak to make the deploy_challenge hook invoke godaddypy client.add_record() instead of update_record(). I also added a tweak to update records to "dehydrated-was-here" during clean_challenge hook.

The net result is a very fast cycle time for the cert generation and as I have to generate many certs with close to 100 dns entries in each, this is a relief.

I'll just ask the godaddy support guys to kill those TXT entries after I'm done the cert deployment.

FYI to others, I am quite impressed with le-godaddy-dns. Its a very well written python library, however it does need to be hacked to add functionality rather than configured.

Thanks for dehydrated, its an awesome library and will definitely be a reference when I need to do some advanced bashing down the road :)

@JavaScriptDude commented on GitHub (Mar 27, 2018): @lukas2511 - Agreed. I spent the day working with [le-godaddy-dns](https://github.com/josteink/le-godaddy-dns) and only had to add a small tweak to make the deploy_challenge hook invoke godaddypy client.add_record() instead of update_record(). I also added a tweak to update records to "_dehydrated-was-here_" during clean_challenge hook. The net result is a very fast cycle time for the cert generation and as I have to generate many certs with close to 100 dns entries in each, this is a relief. I'll just ask the godaddy support guys to kill those TXT entries after I'm done the cert deployment. FYI to others, I am quite impressed with le-godaddy-dns. Its a very well written python library, however it does need to be hacked to add functionality rather than configured. Thanks for dehydrated, its an awesome library and will definitely be a reference when I need to do some advanced bashing down the road :)

adam commented

2025-12-29 01:22:35 +01:00

@immanuelfodor commented on GitHub (Jul 23, 2018):

@JavaScriptDude Your "hacky" solution in comment https://github.com/lukas2511/dehydrated/issues/522#issuecomment-376044830 saved me after a couple of hours of pulling my hair why the challenge fails for DuckDNS. They only support one TXT record per domain, and sub-subdomains' TXT count towards your account, so you can have only one global TXT record for all your sub-subdomains. E.g. something.duckdns.org returns the same TXT record as sub.something.duckdns.org or anothersub.something.duckdns.org. That is why the wildcard cert creation failed for me many many times before I found your comment. Thanks, random Internet person! 😃

@immanuelfodor commented on GitHub (Jul 23, 2018): @JavaScriptDude Your "hacky" solution in comment https://github.com/lukas2511/dehydrated/issues/522#issuecomment-376044830 saved me after a couple of hours of pulling my hair why the challenge fails for DuckDNS. They only support one TXT record per domain, and sub-subdomains' TXT count towards your account, so you can have only one global TXT record for all your sub-subdomains. E.g. `something.duckdns.org` returns the same TXT record as `sub.something.duckdns.org` or `anothersub.something.duckdns.org`. That is why the wildcard cert creation failed for me many many times before I found your comment. Thanks, random Internet person! :smiley:

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/dehydrated#317