release 0.4.0

Use bash explictly as done for the main script.

Signed-off-by: Simon Deziel <simon.deziel@gmail.com>

.travis.yml

@@ -1,6 +1,10 @@
 sudo: false
 language: shell
 
+os:
+  - linux
+  - osx
+
 cache:
   directories:
     - ngrok

CHANGELOG

@@ -5,6 +5,22 @@ This file contains a log of major changes in dehydrated
 ## Changed
 - ...
 
+## [0.4.0] - 2017-02-05
+## Changed
+- dehydrated now asks you to read and accept the CAs terms of service before creating an account
+- Skip challenges for already validated domains
+- Removed need for some special commands (BusyBox compatibility)
+- Exported a few more variables for use in hook-scripts
+- fullchain.pem now actually contains the full chain instead of just the certificate with an intermediate cert
+
+## Added
+- Added private-key rollover functionality
+- Added `--lock-suffix` option for allowing parallel execution
+- Added `invalid_challenge` hook
+- Added `request_failure` hook
+- Added `exit_hook` hook
+- Added standalone `register` command
+
 ## [0.3.1] - 2016-09-13
 ## Changed
 - Renamed project to `dehydrated`.

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015 Lukas Schauer
+Copyright (c) 2015-2017 Lukas Schauer
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -2,11 +2,11 @@
 
 ![](docs/logo.jpg)
 
-This is a client for signing certificates with an ACME-server (currently only provided by letsencrypt) implemented as a relatively simple bash-script.
+This is a client for signing certificates with an ACME-server (currently only provided by Let's Encrypt) implemented as a relatively simple bash-script.
 
 It uses the `openssl` utility for everything related to actually handling keys and certificates, so you need to have that installed.
 
-Other dependencies are: curl, sed, grep, mktemp (all found on almost any system, curl being the only exception)
+Other dependencies are: cURL, sed, grep, mktemp (all found on almost any system, cURL being the only exception)
 
 Current features:
 - Signing of a list of domains
@@ -14,19 +14,30 @@ Current features:
 - Renewal if a certificate is about to expire or SAN (subdomains) changed
 - Certificate revocation
 
-Please keep in mind that this software and even the acme-protocol are relatively young and may still have some unresolved issues.
-Feel free to report any issues you find with this script or contribute by submitting a pullrequest.
+Please keep in mind that this software and even the acme-protocol are relatively young and may still have some unresolved issues. Feel free to report any issues you find with this script or contribute by submitting a pull request.
 
-### Getting started
+## Getting started
 
 For getting started I recommend taking a look at [docs/domains_txt.md](docs/domains_txt.md), [docs/wellknown.md](docs/wellknown.md) and the [Usage](#usage) section on this page (you'll probably only need the `-c` option).
 
 Generally you want to set up your WELLKNOWN path first, and then fill in domains.txt.
 
-**Please note that you should use the staging URL when experimenting with this script to not hit letsencrypts rate limits.** See [docs/staging.md](docs/staging.md).
+**Please note that you should use the staging URL when experimenting with this script to not hit Let's Encrypt's rate limits.** See [docs/staging.md](docs/staging.md).
 
 If you have any problems take a look at our [Troubleshooting](docs/troubleshooting.md) guide.
 
+## Config
+
+dehydrated is looking for a config file in a few different places, it will use the first one it can find in this order:
+
+- `/etc/dehydrated/config`
+- `/usr/local/etc/dehydrated/config`
+- The current working directory of your shell
+- The directory from which dehydrated was ran
+
+Have a look at [docs/examples/config](docs/examples/config) to get started, copy it to e.g. `/etc/dehydrated/config`
+and edit it to fit your needs.
+
 ## Usage:
 
 ```text
@@ -35,6 +46,7 @@ Usage: ./dehydrated [-h] [command [argument]] [parameter [argument]] [parameter
 Default command: help
 
 Commands:
+ --register                       Register account key
  --cron (-c)                      Sign/renew non-existant/changed/expiring certificates.
  --signcsr (-s) path/to/csr.pem   Sign a given CSR, output CRT on stdout (advanced usage)
  --revoke (-r) path/to/cert.pem   Revoke specified certificate
@@ -43,6 +55,7 @@ Commands:
  --env (-e)                       Output configuration variables for use in other scripts
 
 Parameters:
+ --accept-terms                   Accept CAs terms of service
  --full-chain (-fc)               Print full chain when using --signcsr
  --ipv4 (-4)                      Resolve names to IPv4 addresses only
  --ipv6 (-6)                      Resolve names to IPv6 addresses only
@@ -50,6 +63,7 @@ Parameters:
  --keep-going (-g)                Keep going after encountering an error while creating/renewing multiple certificates in cron mode
  --force (-x)                     Force renew of certificate even if it is longer valid than value in RENEW_DAYS
  --no-lock (-n)                   Don't use lockfile (potentially dangerous!)
+ --lock-suffix example.com        Suffix lockfile name with a string (useful for with -d)
  --ocsp                           Sets option in CSR indicating OCSP stapling to be mandatory
  --privkey (-p) path/to/key.pem   Use specified private key instead of account key (useful for revocation)
  --config (-f) path/to/config     Use specified config file

dehydrated

@@ -34,8 +34,8 @@ check_dependencies() {
   openssl version > /dev/null 2>&1 || _exiterr "This script requires an openssl binary."
   _sed "" < /dev/null > /dev/null 2>&1 || _exiterr "This script requires sed with support for extended (modern) regular expressions."
   command -v grep > /dev/null 2>&1 || _exiterr "This script requires grep."
-  _mktemp -u > /dev/null 2>&1 || _exiterr "This script requires mktemp."
-  diff -u /dev/null /dev/null || _exiterr "This script requires diff."
+  command -v mktemp > /dev/null 2>&1 || _exiterr "This script requires mktemp."
+  command -v diff > /dev/null 2>&1 || _exiterr "This script requires diff."
 
   # curl returns with an error code in some ancient versions so we have to catch that
   set +e
@@ -81,7 +81,7 @@ verify_config() {
   if [[ "${CHALLENGETYPE}" = "dns-01" ]] && [[ -z "${HOOK}" ]]; then
     _exiterr "Challenge type dns-01 needs a hook script for deployment... can not continue."
   fi
-  if [[ "${CHALLENGETYPE}" = "http-01" && ! -d "${WELLKNOWN}" ]]; then
+  if [[ "${CHALLENGETYPE}" = "http-01" && ! -d "${WELLKNOWN}" && ! "${COMMAND:-}" = "register" ]]; then
     _exiterr "WELLKNOWN directory doesn't exist, please create ${WELLKNOWN} and set appropriate permissions."
   fi
   [[ "${KEY_ALGO}" =~ ^(rsa|prime256v1|secp384r1)$ ]] || _exiterr "Unknown public key algorithm ${KEY_ALGO}... can not continue."
@@ -105,7 +105,8 @@ load_config() {
 
   # Default values
   CA="https://acme-v01.api.letsencrypt.org/directory"
-  LICENSE="https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf"
+  CA_TERMS="https://acme-v01.api.letsencrypt.org/terms"
+  LICENSE=
   CERTDIR=
   ACCOUNTDIR=
   CHALLENGETYPE="http-01"
@@ -118,6 +119,7 @@ load_config() {
   KEYSIZE="4096"
   WELLKNOWN=
   PRIVATE_KEY_RENEW="yes"
+  PRIVATE_KEY_ROLLOVER="no"
   KEY_ALGO=rsa
   OPENSSL_CNF="$(openssl version -d | cut -d\" -f2)/openssl.cnf"
   CONTACT_EMAIL=
@@ -183,6 +185,7 @@ load_config() {
   [[ -z "${DOMAINS_TXT}" ]] && DOMAINS_TXT="${BASEDIR}/domains.txt"
   [[ -z "${WELLKNOWN}" ]] && WELLKNOWN="/var/www/dehydrated"
   [[ -z "${LOCKFILE}" ]] && LOCKFILE="${BASEDIR}/lock"
+  [[ -n "${PARAM_LOCKFILE_SUFFIX:-}" ]] && LOCKFILE="${LOCKFILE}-${PARAM_LOCKFILE_SUFFIX}"
   [[ -n "${PARAM_NO_LOCK:-}" ]] && LOCKFILE=""
 
   [[ -n "${PARAM_HOOK:-}" ]] && HOOK="${PARAM_HOOK}"
@@ -219,7 +222,7 @@ init_system() {
   _exiterr "Problem retrieving ACME/CA-URLs, check if your configured CA points to the directory entrypoint."
 
   # Export some environment variables to be used in hook script
-  export WELLKNOWN BASEDIR CERTDIR CONFIG
+  export WELLKNOWN BASEDIR CERTDIR CONFIG COMMAND
 
   # Checking for private key ...
   register_new_key="no"
@@ -231,6 +234,24 @@ init_system() {
   else
     # Check if private account key exists, if it doesn't exist yet generate a new one (rsa key)
     if [[ ! -e "${ACCOUNT_KEY}" ]]; then
+      REAL_LICENSE="$(http_request head "${CA_TERMS}" | (grep Location: || true) | awk -F ': ' '{print $2}' | tr -d '\n\r')"
+      if [[ -z "${REAL_LICENSE}" ]]; then
+        printf '\n'
+        printf 'Error retrieving terms of service from certificate authority.\n'
+        printf 'Please set LICENSE in config manually.\n'
+        exit 1
+      fi
+      if [[ ! "${LICENSE}" = "${REAL_LICENSE}" ]]; then
+        if [[ "${PARAM_ACCEPT_TERMS:-}" = "yes" ]]; then
+          LICENSE="${REAL_LICENSE}"
+        else
+          printf '\n'
+          printf 'To use dehydrated with this certificate authority you have to agree to their terms of service which you can find here: %s\n\n' "${REAL_LICENSE}"
+          printf 'To accept these terms of service run `%s --register --accept-terms`.\n' "${0}"
+          exit 1
+        fi
+      fi
+
       echo "+ Generating account key..."
       _openssl genrsa -out "${ACCOUNT_KEY}" "${KEYSIZE}"
       register_new_key="yes"
@@ -247,14 +268,22 @@ init_system() {
   # If we generated a new private key in the step above we have to register it with the acme-server
   if [[ "${register_new_key}" = "yes" ]]; then
     echo "+ Registering account key with ACME server..."
-    [[ ! -z "${CA_NEW_REG}" ]] || _exiterr "Certificate authority doesn't allow registrations."
-    # If an email for the contact has been provided then adding it to the registration request
     FAILED=false
-    if [[ -n "${CONTACT_EMAIL}" ]]; then
-      (signed_request "${CA_NEW_REG}" '{"resource": "new-reg", "contact":["mailto:'"${CONTACT_EMAIL}"'"], "agreement": "'"$LICENSE"'"}' > "${ACCOUNT_KEY_JSON}") || FAILED=true
-    else
-      (signed_request "${CA_NEW_REG}" '{"resource": "new-reg", "agreement": "'"$LICENSE"'"}' > "${ACCOUNT_KEY_JSON}") || FAILED=true
+
+    if [[ -z "${CA_NEW_REG}" ]]; then
+      echo "Certificate authority doesn't allow registrations."
+      FAILED=true
     fi
+
+    # If an email for the contact has been provided then adding it to the registration request
+    if [[ "${FAILED}" = "false" ]]; then
+      if [[ -n "${CONTACT_EMAIL}" ]]; then
+        (signed_request "${CA_NEW_REG}" '{"resource": "new-reg", "contact":["mailto:'"${CONTACT_EMAIL}"'"], "agreement": "'"$LICENSE"'"}' > "${ACCOUNT_KEY_JSON}") || FAILED=true
+      else
+        (signed_request "${CA_NEW_REG}" '{"resource": "new-reg", "agreement": "'"$LICENSE"'"}' > "${ACCOUNT_KEY_JSON}") || FAILED=true
+      fi
+    fi
+
     if [[ "${FAILED}" = "true" ]]; then
       echo
       echo
@@ -262,8 +291,10 @@ init_system() {
       rm "${ACCOUNT_KEY}" "${ACCOUNT_KEY_JSON}"
       exit 1
     fi
+  elif [[ "${COMMAND:-}" = "register" ]]; then
+    echo "+ Account already registered!"
+    exit 0
   fi
-
 }
 
 # Different sed version for different os types...
@@ -305,6 +336,13 @@ get_json_string_value() {
   sed -n "${filter}"
 }
 
+rm_json_arrays() {
+  local filter
+  filter='s/\[[^][]*\]/null/g'
+  # remove three levels of nested arrays
+  sed -e "${filter}" -e "${filter}" -e "${filter}"
+}
+
 # OpenSSL writes to stderr/stdout even when there are no errors. So just
 # display the output if the exit code was != 0 to simplify debugging.
 _openssl() {
@@ -351,22 +389,31 @@ http_request() {
   fi
 
   if [[ ! "${statuscode:0:1}" = "2" ]]; then
-    echo "  + ERROR: An error occurred while sending ${1}-request to ${2} (Status ${statuscode})" >&2
-    echo >&2
-    echo "Details:" >&2
-    cat "${tempcont}" >&2
-    echo >&2
-    echo >&2
-    rm -f "${tempcont}"
+    if [[ ! "${2}" = "${CA_TERMS}" ]] || [[ ! "${statuscode:0:1}" = "3" ]]; then
+      echo "  + ERROR: An error occurred while sending ${1}-request to ${2} (Status ${statuscode})" >&2
+      echo >&2
+      echo "Details:" >&2
+      cat "${tempcont}" >&2
+      echo >&2
+      echo >&2
 
-    # Wait for hook script to clean the challenge if used
-    if [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && [[ -n "${challenge_token:+set}" ]]; then
-      "${HOOK}" "clean_challenge" '' "${challenge_token}" "${keyauth}"
+      # An exclusive hook for the {1}-request error might be useful (e.g., for sending an e-mail to admins)
+      if [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]]; then
+        errtxt=`cat ${tempcont}`
+        "${HOOK}" "request_failure" "${statuscode}" "${errtxt}" "${1}"
+      fi
+
+      rm -f "${tempcont}"
+
+      # Wait for hook script to clean the challenge if used
+      if [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && [[ -n "${challenge_token:+set}" ]]; then
+        "${HOOK}" "clean_challenge" '' "${challenge_token}" "${keyauth}"
+      fi
+
+      # remove temporary domains.txt file if used
+      [[ -n "${PARAM_DOMAIN:-}" && -n "${DOMAINS_TXT:-}" ]] && rm "${DOMAINS_TXT}"
+      exit 1
     fi
-
-    # remove temporary domains.txt file if used
-    [[ -n "${PARAM_DOMAIN:-}" && -n "${DOMAINS_TXT:-}" ]] && rm "${DOMAINS_TXT}"
-    exit 1
   fi
 
   cat "${tempcont}"
@@ -409,7 +456,7 @@ extract_altnames() {
   reqtext="$( <<<"${csr}" openssl req -noout -text )"
   if <<<"${reqtext}" grep -q '^[[:space:]]*X509v3 Subject Alternative Name:[[:space:]]*$'; then
     # SANs used, extract these
-    altnames="$( <<<"${reqtext}" grep -A1 '^[[:space:]]*X509v3 Subject Alternative Name:[[:space:]]*$' | tail -n1 )"
+    altnames="$( <<<"${reqtext}" awk '/X509v3 Subject Alternative Name:/{print;getline;print;}' | tail -n1 )"
     # split to one per line:
     # shellcheck disable=SC1003
     altnames="$( <<<"${altnames}" _sed -e 's/^[[:space:]]*//; s/, /\'$'\n''/g' )"
@@ -450,9 +497,9 @@ sign_csr() {
 
   local idx=0
   if [[ -n "${ZSH_VERSION:-}" ]]; then
-    local -A challenge_uris challenge_tokens keyauths deploy_args
+    local -A challenge_altnames challenge_uris challenge_tokens keyauths deploy_args
   else
-    local -a challenge_uris challenge_tokens keyauths deploy_args
+    local -a challenge_altnames challenge_uris challenge_tokens keyauths deploy_args
   fi
 
   # Request challenges
@@ -461,6 +508,12 @@ sign_csr() {
     echo " + Requesting challenge for ${altname}..."
     response="$(signed_request "${CA_NEW_AUTHZ}" '{"resource": "new-authz", "identifier": {"type": "dns", "value": "'"${altname}"'"}}' | clean_json)"
 
+    challenge_status="$(printf '%s' "${response}" | rm_json_arrays | get_json_string_value status)"
+    if [ "${challenge_status}" = "valid" ]; then
+       echo " + Already validated!"
+       continue
+    fi
+
     challenges="$(printf '%s\n' "${response}" | sed -n 's/.*\("challenges":[^\[]*\[[^]]*]\).*/\1/p')"
     repl=$'\n''{' # fix syntax highlighting in Vim
     challenge="$(printf "%s" "${challenges//\{/${repl}}" | grep \""${CHALLENGETYPE}"\")"
@@ -487,6 +540,7 @@ sign_csr() {
         ;;
     esac
 
+    challenge_altnames[${idx}]="${altname}"
     challenge_uris[${idx}]="${challenge_uri}"
     keyauths[${idx}]="${keyauth}"
     challenge_tokens[${idx}]="${challenge_token}"
@@ -494,56 +548,64 @@ sign_csr() {
     deploy_args[${idx}]="${altname} ${challenge_token} ${keyauth_hook}"
     idx=$((idx+1))
   done
+  challenge_count="${idx}"
 
   # Wait for hook script to deploy the challenges if used
-  # shellcheck disable=SC2068
-  [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" = "yes" ]] && "${HOOK}" "deploy_challenge" ${deploy_args[@]}
+  if [[ ${challenge_count} -ne 0 ]]; then
+    # shellcheck disable=SC2068
+    [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" = "yes" ]] && "${HOOK}" "deploy_challenge" ${deploy_args[@]}
+  fi
 
   # Respond to challenges
+  reqstatus="valid"
   idx=0
-  for altname in ${altnames}; do
-    challenge_token="${challenge_tokens[${idx}]}"
-    keyauth="${keyauths[${idx}]}"
+  if [ ${challenge_count} -ne 0 ]; then
+    for altname in "${challenge_altnames[@]:0}"; do
+      challenge_token="${challenge_tokens[${idx}]}"
+      keyauth="${keyauths[${idx}]}"
 
-    # Wait for hook script to deploy the challenge if used
-    # shellcheck disable=SC2086
-    [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && "${HOOK}" "deploy_challenge" ${deploy_args[${idx}]}
-
-    # Ask the acme-server to verify our challenge and wait until it is no longer pending
-    echo " + Responding to challenge for ${altname}..."
-    result="$(signed_request "${challenge_uris[${idx}]}" '{"resource": "challenge", "keyAuthorization": "'"${keyauth}"'"}' | clean_json)"
-
-    reqstatus="$(printf '%s\n' "${result}" | get_json_string_value status)"
-
-    while [[ "${reqstatus}" = "pending" ]]; do
-      sleep 1
-      result="$(http_request get "${challenge_uris[${idx}]}")"
-      reqstatus="$(printf '%s\n' "${result}" | get_json_string_value status)"
-    done
-
-    [[ "${CHALLENGETYPE}" = "http-01" ]] && rm -f "${WELLKNOWN}/${challenge_token}"
-
-    # Wait for hook script to clean the challenge if used
-    if [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && [[ -n "${challenge_token}" ]]; then
+      # Wait for hook script to deploy the challenge if used
       # shellcheck disable=SC2086
-      "${HOOK}" "clean_challenge" ${deploy_args[${idx}]}
-    fi
-    idx=$((idx+1))
+      [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && "${HOOK}" "deploy_challenge" ${deploy_args[${idx}]}
 
-    if [[ "${reqstatus}" = "valid" ]]; then
-      echo " + Challenge is valid!"
-    else
-      break
-    fi
-  done
+      # Ask the acme-server to verify our challenge and wait until it is no longer pending
+      echo " + Responding to challenge for ${altname}..."
+      result="$(signed_request "${challenge_uris[${idx}]}" '{"resource": "challenge", "keyAuthorization": "'"${keyauth}"'"}' | clean_json)"
+
+      reqstatus="$(printf '%s\n' "${result}" | get_json_string_value status)"
+
+      while [[ "${reqstatus}" = "pending" ]]; do
+        sleep 1
+        result="$(http_request get "${challenge_uris[${idx}]}")"
+        reqstatus="$(printf '%s\n' "${result}" | get_json_string_value status)"
+      done
+
+      [[ "${CHALLENGETYPE}" = "http-01" ]] && rm -f "${WELLKNOWN}/${challenge_token}"
+
+      # Wait for hook script to clean the challenge if used
+      if [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && [[ -n "${challenge_token}" ]]; then
+        # shellcheck disable=SC2086
+        "${HOOK}" "clean_challenge" ${deploy_args[${idx}]}
+      fi
+      idx=$((idx+1))
+
+      if [[ "${reqstatus}" = "valid" ]]; then
+        echo " + Challenge is valid!"
+      else
+        [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" != "yes" ]] && "${HOOK}" "invalid_challenge" "${altname}" "${result}"
+      fi
+    done
+  fi
 
   # Wait for hook script to clean the challenges if used
   # shellcheck disable=SC2068
-  [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" = "yes" ]] && "${HOOK}" "clean_challenge" ${deploy_args[@]}
+  if [[ ${challenge_count} -ne 0 ]]; then
+    [[ -n "${HOOK}" ]] && [[ "${HOOK_CHAIN}" = "yes" ]] && "${HOOK}" "clean_challenge" ${deploy_args[@]}
+  fi
 
   if [[ "${reqstatus}" != "valid" ]]; then
     # Clean up any remaining challenge_tokens if we stopped early
-    if [[ "${CHALLENGETYPE}" = "http-01" ]]; then
+    if [[ "${CHALLENGETYPE}" = "http-01" ]] && [[ ${challenge_count} -ne 0 ]]; then
       while [ ${idx} -lt ${#challenge_tokens[@]} ]; do
         rm -f "${WELLKNOWN}/${challenge_tokens[${idx}]}"
         idx=$((idx+1))
@@ -569,6 +631,51 @@ sign_csr() {
   echo " + Done!"
 }
 
+# grep issuer cert uri from certificate
+get_issuer_cert_uri() {
+  certificate="${1}"
+  openssl x509 -in "${certificate}" -noout -text | (grep 'CA Issuers - URI:' | cut -d':' -f2-) || true
+}
+
+# walk certificate chain, retrieving all intermediate certificates
+walk_chain() {
+  local certificate
+  certificate="${1}"
+
+  local issuer_cert_uri
+  issuer_cert_uri="${2:-}"
+  if [[ -z "${issuer_cert_uri}" ]]; then issuer_cert_uri="$(get_issuer_cert_uri "${certificate}")"; fi
+  if [[ -n "${issuer_cert_uri}" ]]; then
+    # create temporary files
+    local tmpcert
+    local tmpcert_raw
+    tmpcert_raw="$(_mktemp)"
+    tmpcert="$(_mktemp)"
+
+    # download certificate
+    http_request get "${issuer_cert_uri}" > "${tmpcert_raw}"
+
+    # PEM
+    if grep -q "BEGIN CERTIFICATE" "${tmpcert_raw}"; then mv "${tmpcert_raw}" "${tmpcert}"
+    # DER
+    elif openssl x509 -in "${tmpcert_raw}" -inform DER -out "${tmpcert}" -outform PEM 2> /dev/null > /dev/null; then :
+    # PKCS7
+    elif openssl pkcs7 -in "${tmpcert_raw}" -inform DER -out "${tmpcert}" -outform PEM -print_certs 2> /dev/null > /dev/null; then :
+    # Unknown certificate type
+    else _exiterr "Unknown certificate type in chain"
+    fi
+
+    local next_issuer_cert_uri
+    next_issuer_cert_uri="$(get_issuer_cert_uri "${tmpcert}")"
+    if [[ -n "${next_issuer_cert_uri}" ]]; then
+      printf "\n%s\n" "${issuer_cert_uri}"
+      cat "${tmpcert}"
+      walk_chain "${tmpcert}" "${next_issuer_cert_uri}"
+    fi
+    rm -f "${tmpcert}" "${tmpcert_raw}"
+  fi
+}
+
 # Create certificate for domain(s)
 sign_domain() {
   domain="${1}"
@@ -596,6 +703,26 @@ sign_domain() {
       prime256v1|secp384r1) _openssl ecparam -genkey -name "${KEY_ALGO}" -out "${CERTDIR}/${domain}/privkey-${timestamp}.pem";;
     esac
   fi
+  # move rolloverkey into position (if any)
+  if [[ -r "${CERTDIR}/${domain}/privkey.pem" && -r "${CERTDIR}/${domain}/privkey.roll.pem" && "${PRIVATE_KEY_RENEW}" = "yes" && "${PRIVATE_KEY_ROLLOVER}" = "yes" ]]; then
+    echo " + Moving Rolloverkey into position....  "
+    mv "${CERTDIR}/${domain}/privkey.roll.pem" "${CERTDIR}/${domain}/privkey-tmp.pem"
+    mv "${CERTDIR}/${domain}/privkey-${timestamp}.pem" "${CERTDIR}/${domain}/privkey.roll.pem"
+    mv "${CERTDIR}/${domain}/privkey-tmp.pem" "${CERTDIR}/${domain}/privkey-${timestamp}.pem"
+  fi
+  # generate a new private rollover key if we need or want one
+  if [[ ! -r "${CERTDIR}/${domain}/privkey.roll.pem" && "${PRIVATE_KEY_ROLLOVER}" = "yes" && "${PRIVATE_KEY_RENEW}" = "yes" ]]; then
+    echo " + Generating private rollover key..."
+    case "${KEY_ALGO}" in
+      rsa) _openssl genrsa -out "${CERTDIR}/${domain}/privkey.roll.pem" "${KEYSIZE}";;
+      prime256v1|secp384r1) _openssl ecparam -genkey -name "${KEY_ALGO}" -out "${CERTDIR}/${domain}/privkey.roll.pem";;
+    esac
+  fi
+  # delete rolloverkeys if disabled
+  if [[ -r "${CERTDIR}/${domain}/privkey.roll.pem" && ! "${PRIVATE_KEY_ROLLOVER}" = "yes" ]]; then
+    echo " + Removing Rolloverkey (feature disabled)..."
+    rm -f "${CERTDIR}/${domain}/privkey.roll.pem"
+  fi
 
   # Generate signing request config and the actual signing request
   echo " + Generating signing request..."
@@ -621,10 +748,7 @@ sign_domain() {
   # Create fullchain.pem
   echo " + Creating fullchain.pem..."
   cat "${crt_path}" > "${CERTDIR}/${domain}/fullchain-${timestamp}.pem"
-  http_request get "$(openssl x509 -in "${CERTDIR}/${domain}/cert-${timestamp}.pem" -noout -text | grep 'CA Issuers - URI:' | cut -d':' -f2-)" > "${CERTDIR}/${domain}/chain-${timestamp}.pem"
-  if ! grep -q "BEGIN CERTIFICATE" "${CERTDIR}/${domain}/chain-${timestamp}.pem"; then
-    openssl x509 -in "${CERTDIR}/${domain}/chain-${timestamp}.pem" -inform DER -out "${CERTDIR}/${domain}/chain-${timestamp}.pem" -outform PEM
-  fi
+  walk_chain "${crt_path}" > "${CERTDIR}/${domain}/chain-${timestamp}.pem"
   cat "${CERTDIR}/${domain}/chain-${timestamp}.pem" >> "${CERTDIR}/${domain}/fullchain-${timestamp}.pem"
 
   # Update symlinks
@@ -636,13 +760,20 @@ sign_domain() {
   ln -sf "cert-${timestamp}.pem" "${CERTDIR}/${domain}/cert.pem"
 
   # Wait for hook script to clean the challenge and to deploy cert if used
-  export KEY_ALGO
   [[ -n "${HOOK}" ]] && "${HOOK}" "deploy_cert" "${domain}" "${CERTDIR}/${domain}/privkey.pem" "${CERTDIR}/${domain}/cert.pem" "${CERTDIR}/${domain}/fullchain.pem" "${CERTDIR}/${domain}/chain.pem" "${timestamp}"
 
   unset challenge_token
   echo " + Done!"
 }
 
+# Usage: --register
+# Description: Register account key
+command_register() {
+  init_system
+  echo "+ Done!"
+  exit 0
+}
+
 # Usage: --cron (-c)
 # Description: Sign/renew non-existant/changed/expiring certificates.
 command_sign_domains() {
@@ -662,7 +793,7 @@ command_sign_domains() {
   # Generate certificates for all domains found in domains.txt. Check if existing certificate are about to expire
   ORIGIFS="${IFS}"
   IFS=$'\n'
-  for line in $(<"${DOMAINS_TXT}" tr -d '\r' | tr '[:upper:]' '[:lower:]' | _sed -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*$//g' -e 's/[[:space:]]+/ /g' | (grep -vE '^(#|$)' || true)); do
+  for line in $(<"${DOMAINS_TXT}" tr -d '\r' | awk '{print tolower($0)}' | _sed -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*$//g' -e 's/[[:space:]]+/ /g' | (grep -vE '^(#|$)' || true)); do
     reset_configvars
     IFS="${ORIGIFS}"
     domain="$(printf '%s\n' "${line}" | cut -d' ' -f1)"
@@ -705,7 +836,7 @@ command_sign_domains() {
         config_var="$(echo "${cfgline:1}" | cut -d'=' -f1)"
         config_value="$(echo "${cfgline:1}" | cut -d'=' -f2-)"
         case "${config_var}" in
-          KEY_ALGO|OCSP_MUST_STAPLE|PRIVATE_KEY_RENEW|KEYSIZE|CHALLENGETYPE|HOOK|WELLKNOWN|HOOK_CHAIN|OPENSSL_CNF|RENEW_DAYS)
+          KEY_ALGO|OCSP_MUST_STAPLE|PRIVATE_KEY_RENEW|PRIVATE_KEY_ROLLOVER|KEYSIZE|CHALLENGETYPE|HOOK|WELLKNOWN|HOOK_CHAIN|OPENSSL_CNF|RENEW_DAYS)
             echo "   + ${config_var} = ${config_value}"
             declare -- "${config_var}=${config_value}"
             ;;
@@ -716,6 +847,7 @@ command_sign_domains() {
       IFS="${ORIGIFS}"
     fi
     verify_config
+    export WELLKNOWN CHALLENGETYPE KEY_ALGO PRIVATE_KEY_ROLLOVER
 
     if [[ -e "${cert}" ]]; then
       printf " + Checking domain name(s) of existing cert..."
@@ -767,6 +899,7 @@ command_sign_domains() {
   # remove temporary domains.txt file if used
   [[ -n "${PARAM_DOMAIN:-}" ]] && rm -f "${DOMAINS_TXT}"
 
+  [[ -n "${HOOK}" ]] && "${HOOK}" "exit_hook"
   exit 0
 }
 
@@ -797,10 +930,13 @@ command_sign_csr() {
   if [ -n "${PARAM_FULL_CHAIN:-}" ]; then
     # get and convert ca cert
     chainfile="$(_mktemp)"
-    http_request get "$(openssl x509 -in "${certfile}" -noout -text | grep 'CA Issuers - URI:' | cut -d':' -f2-)" > "${chainfile}"
-
-    if ! grep -q "BEGIN CERTIFICATE" "${chainfile}"; then
-      openssl x509 -inform DER -in "${chainfile}" -outform PEM -out "${chainfile}"
+    tmpchain="$(_mktemp)"
+    http_request get "$(openssl x509 -in "${certfile}" -noout -text | grep 'CA Issuers - URI:' | cut -d':' -f2-)" > "${tmpchain}"
+    if grep -q "BEGIN CERTIFICATE" "${tmpchain}"; then
+      mv "${tmpchain}" "${chainfile}"
+    else
+      openssl x509 -in "${tmpchain}" -inform DER -out "${chainfile}" -outform PEM
+      rm "${tmpchain}"
     fi
 
     echo "# CHAIN #" >&3
@@ -965,6 +1101,16 @@ main() {
         set_command sign_domains
         ;;
 
+      --register)
+        set_command register
+        ;;
+
+      # PARAM_Usage: --accept-terms
+      # PARAM_Description: Accept CAs terms of service
+      --accept-terms)
+        PARAM_ACCEPT_TERMS="yes"
+        ;;
+
       --signcsr|-s)
         shift 1
         set_command sign_csr
@@ -1031,6 +1177,14 @@ main() {
         PARAM_NO_LOCK="yes"
         ;;
 
+      # PARAM_Usage: --lock-suffix example.com
+      # PARAM_Description: Suffix lockfile name with a string (useful for with -d)
+      --lock-suffix)
+      	shift 1
+        check_parameters "${1:-}"
+	PARAM_LOCKFILE_SUFFIX="${1}"
+        ;;
+
       # PARAM_Usage: --ocsp
       # PARAM_Description: Sets option in CSR indicating OCSP stapling to be mandatory
       --ocsp)
@@ -1099,6 +1253,7 @@ main() {
   case "${COMMAND}" in
     env) command_env;;
     sign_domains) command_sign_domains;;
+    register) command_register;;
     sign_csr) command_sign_csr "${PARAM_CSR}";;
     revoke) command_revoke "${PARAM_REVOKECERT}";;
     cleanup) command_cleanup;;

docs/dns-verification.md

@@ -4,7 +4,7 @@ This script also supports the new `dns-01`-type verification. This type of verif
 
 You need a hook script that deploys the challenge to your DNS server!
 
-The hook script (indicated in the config file or the --hook/-k command line argument) gets four arguments: an operation name (clean_challenge, deploy_challenge, or deploy_cert) and some operands for that. For deploy_challenge $2 is the domain name for which the certificate is required, $3 is a "challenge token" (which is not needed for dns-01), and $4 is a token which needs to be inserted in a TXT record for the domain.
+The hook script (indicated in the config file or the --hook/-k command line argument) gets four arguments: an operation name (clean_challenge, deploy_challenge, deploy_cert, invalid_challenge or request_failure) and some operands for that. For deploy_challenge $2 is the domain name for which the certificate is required, $3 is a "challenge token" (which is not needed for dns-01), and $4 is a token which needs to be inserted in a TXT record for the domain.
 
 Typically, you will need to split the subdomain name in two, the subdomain name and the domain name separately. For example, for "my.example.com", you'll need "my" and "example.com" separately. You then have to prefix "_acme-challenge." before the subdomain name, as in "_acme-challenge.my" and set a TXT record for that on the domain (e.g. "example.com") which has the value supplied in $4
 

docs/examples/config

@@ -18,8 +18,11 @@
 # Path to certificate authority (default: https://acme-v01.api.letsencrypt.org/directory)
 #CA="https://acme-v01.api.letsencrypt.org/directory"
 
-# Path to license agreement (default: https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf)
-#LICENSE="https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf"
+# Path to certificate authority license terms redirect (default: https://acme-v01.api.letsencrypt.org/terms)
+#CA_TERMS="https://acme-v01.api.letsencrypt.org/terms"
+
+# Path to license agreement (default: <unset>)
+#LICENSE=""
 
 # Which challenge should be used? Currently http-01 and dns-01 are supported
 #CHALLENGETYPE="http-01"
@@ -72,6 +75,9 @@
 # Regenerate private keys instead of just signing new certificates on renewal (default: yes)
 #PRIVATE_KEY_RENEW="yes"
 
+# Create an extra private key for rollover (default: no)
+#PRIVATE_KEY_ROLLOVER="no"
+
 # Which public key algorithm should be used? Supported: rsa, prime256v1 and secp384r1
 #KEY_ALGO=rsa
 

docs/examples/hook.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 
-function deploy_challenge {
+deploy_challenge() {
     local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
 
     # This hook is called once for every domain that needs to be
@@ -21,7 +21,7 @@ function deploy_challenge {
     #   be found in the $TOKEN_FILENAME file.
 }
 
-function clean_challenge {
+clean_challenge() {
     local DOMAIN="${1}" TOKEN_FILENAME="${2}" TOKEN_VALUE="${3}"
 
     # This hook is called after attempting to validate each domain,
@@ -31,7 +31,7 @@ function clean_challenge {
     # The parameters are the same as for deploy_challenge.
 }
 
-function deploy_cert {
+deploy_cert() {
     local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" CHAINFILE="${5}" TIMESTAMP="${6}"
 
     # This hook is called once for each certificate that has been
@@ -54,7 +54,7 @@ function deploy_cert {
     #   Timestamp when the specified certificate was created.
 }
 
-function unchanged_cert {
+unchanged_cert() {
     local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" CHAINFILE="${5}"
 
     # This hook is called once for each certificate that is still
@@ -74,4 +74,45 @@ function unchanged_cert {
     #   The path of the file containing the intermediate certificate(s).
 }
 
-HANDLER=$1; shift; $HANDLER $@
+invalid_challenge() {
+    local DOMAIN="${1}" RESPONSE="${2}"
+
+    # This hook is called if the challenge response has failed, so domain
+    # owners can be aware and act accordingly.
+    #
+    # Parameters:
+    # - DOMAIN
+    #   The primary domain name, i.e. the certificate common
+    #   name (CN).
+    # - RESPONSE
+    #   The response that the verification server returned
+}
+
+request_failure() {
+    local STATUSCODE="${1}" REASON="${2}" REQTYPE="${3}"
+
+    # This hook is called when a HTTP request fails (e.g., when the ACME
+    # server is busy, returns an error, etc). It will be called upon any
+    # response code that does not start with '2'. Useful to alert admins
+    # about problems with requests.
+    #
+    # Parameters:
+    # - STATUSCODE
+    #   The HTML status code that originated the error.
+    # - REASON
+    #   The specified reason for the error.
+    # - REQTYPE
+    #   The kind of request that was made (GET, POST...)
+}
+
+exit_hook() {
+  # This hook is called at the end of a dehydrated command and can be used
+  # to do some final (cleanup or other) tasks.
+
+  :
+}
+
+HANDLER="$1"; shift
+if [[ "${HANDLER}" =~ ^(deploy_challenge|clean_challenge|deploy_cert|unchanged_cert|invalid_challenge|request_failure|exit_hook)$ ]]; then
+  "$HANDLER" "$@"
+fi

docs/staging.md

@@ -9,4 +9,5 @@ To avoid this, please set the CA property to the Let’s Encrypt staging server
 
 ```bash
 CA="https://acme-staging.api.letsencrypt.org/directory"
+CA_TERMS="https://acme-staging.api.letsencrypt.org/terms"
 ```

docs/wellknown.md

@@ -60,9 +60,8 @@ Alias /.well-known/acme-challenge /var/www/dehydrated
 With Lighttpd just add this to your config and it should work in any VHost:
 
 ```lighttpd
-modules += "alias"
-
+server.modules += ("alias")
 alias.url += (
- "/.well-known/acme-challenge/" => "/var/www/dehydrated/"
+ "/.well-known/acme-challenge/" => "/var/www/dehydrated/",
 )
 ```

test.sh

@@ -69,7 +69,14 @@ if [[ ! -e "ngrok/ngrok" ]]; then
   (
     mkdir -p ngrok
     cd ngrok
-    wget https://dl.ngrok.com/ngrok_2.0.19_linux_amd64.zip -O ngrok.zip
+    if [ "${TRAVIS_OS_NAME}" = "linux" ]; then
+      wget -O ngrok.zip https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
+    elif [ "${TRAVIS_OS_NAME}" = "osx" ]; then
+      wget -O ngrok.zip https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-darwin-amd64.zip
+    else
+      echo "No ngrok for ${TRAVIS_OS_NAME}"
+      exit 1
+    fi
     unzip ngrok.zip ngrok
     chmod +x ngrok
   )
@@ -97,7 +104,7 @@ mkdir -p .acme-challenges/.well-known/acme-challenge
 
 # Generate config and create empty domains.txt
 echo 'CA="https://testca.kurz.pw/directory"' > config
-echo 'LICENSE="https://testca.kurz.pw/terms/v1"' >> config
+echo 'CA_TERMS="https://testca.kurz.pw/terms"' >> config
 echo 'WELLKNOWN=".acme-challenges/.well-known/acme-challenge"' >> config
 echo 'RENEW_DAYS="14"' >> config
 touch domains.txt
@@ -110,6 +117,23 @@ _CHECK_LOG "--help (-h)"
 _CHECK_LOG "--domain (-d) domain.tld"
 _CHECK_ERRORLOG
 
+# Register account key without LICENSE set
+_TEST "Register account key without LICENSE set"
+./dehydrated --register > tmplog 2> errorlog && _FAIL "Script execution failed"
+_CHECK_LOG "To accept these terms"
+_CHECK_ERRORLOG
+
+# Register account key and agreeing to terms
+_TEST "Register account key without LICENSE set"
+./dehydrated --register --accept-terms > tmplog 2> errorlog || _FAIL "Script execution failed"
+_CHECK_LOG "Registering account key"
+_CHECK_FILE accounts/*/account_key.pem
+_CHECK_ERRORLOG
+
+# Delete accounts and add LICENSE to config for normal operation
+rm -rf accounts
+echo 'LICENSE="https://testca.kurz.pw/terms/v1"' >> config
+
 # Run in cron mode with empty domains.txt (should only generate private key and exit)
 _TEST "First run in cron mode, checking if private key is generated and registered"
 ./dehydrated --cron > tmplog 2> errorlog || _FAIL "Script execution failed"
@@ -120,7 +144,7 @@ _CHECK_ERRORLOG
 # Temporarily move config out of the way and try signing certificate by using temporary config location
 _TEST "Try signing using temporary config location and with domain as command line parameter"
 mv config tmp_config
-./dehydrated --cron --domain "${TMP_URL}" --domain "${TMP2_URL}" -f tmp_config > tmplog 2> errorlog || _FAIL "Script execution failed"
+./dehydrated --cron --domain "${TMP_URL}" --domain "${TMP2_URL}" --accept-terms -f tmp_config > tmplog 2> errorlog || _FAIL "Script execution failed"
 _CHECK_NOT_LOG "Checking domain name(s) of existing cert"
 _CHECK_LOG "Generating private key"
 _CHECK_LOG "Requesting challenge for ${TMP_URL}"
@@ -168,7 +192,7 @@ _CHECK_NOT_LOG "Generating private key"
 _CHECK_LOG "Requesting challenge for ${TMP_URL}"
 _CHECK_LOG "Requesting challenge for ${TMP2_URL}"
 _CHECK_LOG "Requesting challenge for ${TMP3_URL}"
-_CHECK_LOG "Challenge is valid!"
+_CHECK_LOG "Already validated!"
 _CHECK_LOG "Creating fullchain.pem"
 _CHECK_LOG "Done!"
 _CHECK_ERRORLOG
@@ -197,7 +221,8 @@ _CHECK_LOG "${TMP2_URL}"
 _SUBTEST "Verifying file with full chain..."
 openssl x509 -in "certs/${TMP_URL}/fullchain.pem" -noout -text > /dev/null 2>> errorlog && _PASS || _FAIL
 _SUBTEST "Verifying certificate against CA certificate..."
-(openssl verify -verbose -CAfile "certs/${TMP_URL}/fullchain.pem" -purpose sslserver "certs/${TMP_URL}/fullchain.pem" 2>&1 || true) | (grep -v ': OK$' || true) >> errorlog 2>> errorlog && _PASS || _FAIL
+curl -s https://testca.kurz.pw/acme/issuer-cert | openssl x509 -inform DER -outform PEM > ca.pem
+(openssl verify -verbose -CAfile "ca.pem" -purpose sslserver "certs/${TMP_URL}/fullchain.pem" 2>&1 || true) | (grep -v ': OK$' || true) >> errorlog 2>> errorlog && _PASS || _FAIL
 _CHECK_ERRORLOG
 
 # Revoke certificate using certificate key
@@ -209,6 +234,26 @@ _CHECK_LOG "Done."
 _CHECK_FILE "certs/${TMP_URL}/${REAL_CERT}-revoked"
 _CHECK_ERRORLOG
 
+# Enable private key renew
+echo 'PRIVATE_KEY_RENEW="yes"' >> config
+echo 'PRIVATE_KEY_ROLLOVER="yes"' >> config
+
+# Check if Rolloverkey creation works
+_TEST "Testing Rolloverkeys..."
+_SUBTEST "First Run: Creating rolloverkey"
+./dehydrated --cron --domain "${TMP2_URL}" > tmplog 2> errorlog || _FAIL "Script execution failed"
+CERT_ROLL_HASH=$(openssl rsa -in certs/${TMP2_URL}/privkey.roll.pem -outform DER -pubout 2>/dev/null | openssl sha -sha256)
+_CHECK_LOG "Generating private key"
+_CHECK_LOG "Generating private rollover key"
+_SUBTEST "Second Run: Force Renew, Use rolloverkey"
+./dehydrated --cron --force --domain "${TMP2_URL}" > tmplog 2> errorlog || _FAIL "Script execution failed"
+CERT_NEW_HASH=$(openssl rsa -in certs/${TMP2_URL}/privkey.pem -outform DER -pubout 2>/dev/null | openssl sha -sha256)
+_CHECK_LOG "Generating private key"
+_CHECK_LOG "Moving Rolloverkey into position"
+_SUBTEST "Verifying Hash Rolloverkey and private key second run"
+[[ "${CERT_ROLL_HASH}" = "${CERT_NEW_HASH}" ]] && _PASS || _FAIL
+_CHECK_ERRORLOG
+
 # Test cleanup command
 _TEST "Cleaning up certificates"
 ./dehydrated --cleanup > tmplog 2> errorlog || _FAIL "Script execution failed"