The IP SAN Constraint: Running a 160-Hour Certificate with Let's Encrypt.

Who needs an IP certificate at all

Start with the client's problem, because the whole architecture follows from it. A device is handed a DNS server the old way - 194.180.189.33 from DHCP, or typed into a settings screen. At that moment the device knows an IP address and nothing else: no hostname, no configuration, no prior trust. Discovery of Designated Resolvers (RFC 9462) lets it ask that resolver "do you offer encrypted transports?" and upgrade to DoH, DoT or DoQ. But an unencrypted answer to that question could come from anyone on the path, so the standard demands proof: the TLS certificate presented during the upgrade must be valid for the IP address the device started with. Not for a hostname the answer happened to mention - for the bare IP. The same requirement appears wherever clients connect by raw address: DoT and DoQ by IP, or any TLS service reached without a name in front of it.

That means the certificate needs the address in its SAN list: IP Address:194.180.189.33, next to the hostnames. If your users always reach you through a hostname, you do not need any of this - an ordinary 90-day certificate is the right tool, and you can stop reading. This runbook is for the operators whose service is an address: public DNS resolvers first among them.

Publicly-trusted IP certificates are rare. Google Trust Services issues them, but registering an ACME account and requesting one gets a hard rejection - "IP addresses are not supported" - unless your organisation owns the address block and arranges entitlement through sales. Let's Encrypt added IP support in 2025, with one condition: IP identifiers are only available in the shortlived certificate profile, which caps validity at about 6.6 days. 160 hours, to be exact. We tested both paths in July 2026; for an operator on rented address space, Let's Encrypt is the only free option, so the lifetime comes with it. The trade, in full:

The gains are properties of the certificate; the costs are all the same cost - the automation has to be real. The rest of this runbook is that automation, exactly as it runs here in production.

Issuance: one command, five identifiers

The lineage was created once, on 3 July 2026, with a single certbot command. Everything after that has been automatic. IP identifiers cannot be validated over DNS-01, so validation is plain http-01 against a shared webroot; the nginx side is one location /.well-known/acme-challenge/ block serving that directory on port 80 for every name and for the bare IP.

$ certbot certonly --webroot -w /var/www/acme \
    --cert-name unified.dnsdoh.art \
    -d dnsdoh.art -d www.dnsdoh.art -d agh.dnsdoh.art -d gr.dnsdoh.art \
    --ip-address 194.180.189.33 \
    --required-profile shortlived \
    --reuse-key --key-type ecdsa --elliptic-curve secp256r1

Three flags carry the design. --ip-address puts the address into the SAN list. --required-profile shortlived demands the 160-hour profile and makes issuance fail rather than silently fall back to a default profile that would drop the IP; for this lineage a loud failure is the correct behaviour, because the watchdog below is listening for exactly that. And --reuse-key freezes the key pair across every future renewal - the entire DANE story in the next sections rests on it. What Let's Encrypt returns is worth a close look:

$ openssl x509 -in /etc/letsencrypt/live/unified.dnsdoh.art/cert.pem \
    -noout -subject -issuer -dates -ext subjectAltName
subject=                                    ← empty, and that is intentional
issuer=C = US, O = Let's Encrypt, CN = YE1
notBefore=Jul 10 04:10:26 2026 GMT
notAfter=Jul 16 20:10:25 2026 GMT          ← exactly 160 hours later
X509v3 Subject Alternative Name: critical
    DNS:agh.dnsdoh.art, DNS:dnsdoh.art, DNS:gr.dnsdoh.art,
    DNS:www.dnsdoh.art, IP Address:194.180.189.33

Two details most people never see. The subject is empty: shortlived-profile certificates carry no Common Name at all, identity lives exclusively in the SAN list. And because the subject is empty, X.509 requires the SAN extension to be marked critical - a client that does not understand SANs must reject the certificate rather than accept it with no name checking. Both are correct, modern behaviour; both will confuse tooling that still expects a CN.

The command's choices persist in the lineage's renewal file, which is what every later automatic run reads. This is the production file verbatim (account id redacted):

[renewalparams]
renew_hook = systemctl reload nginx; systemctl restart AdGuardHome
reuse_key = True
account = (redacted)
key_type = ecdsa
elliptic_curve = secp256r1
required_profile = shortlived
authenticator = webroot
webroot_path = /var/www/acme,
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]
194.180.189.33 = /var/www/acme
agh.dnsdoh.art = /var/www/acme
dnsdoh.art = /var/www/acme
gr.dnsdoh.art = /var/www/acme
www.dnsdoh.art = /var/www/acme

The renew_hook is the deployment step: nginx reloads gracefully, and AdGuardHome (the DNS engine behind DoT and DoQ) restarts to pick up the new file. Both consume the same live/ symlinks, so one lineage serves the website, the DoH endpoint and both port-853 transports. One certificate, one renewal, the whole stack.

The renewal rhythm: ARI decides, two timers execute

Nothing on this server computes "renew at 30 days remaining" or any other local rule. The renewal moment comes from the CA itself, via ACME Renewal Information (ARI, RFC 9773): on every run, certbot asks Let's Encrypt for the suggested renewal window of the exact certificate it holds, and acts only when the window has opened. The CA can move the window - to spread load, or to drain a certificate early during an incident like a mass revocation - and the client follows automatically. For this lineage the observed window opens near the midpoint of the lifetime, about 80 hours before expiry.

That claim is checkable against the lineage's own archive. Certbot keeps every certificate it has ever obtained, numbered in order. This table is the operational history, read straight from disk on 11 July:

Read the rightmost column top to bottom: five certificates, one public key. That is reuse_key at work, and it is the subject of the next section. Read the dates: natural renewals arrive every ~3.3 days, which works out to roughly 110 certificates a year. At that frequency, issuance is not an event. It is background metabolism, and treating it that way - no ceremony, no manual steps, nothing to remember - is the entire discipline.

Two independent timers drive it, on the theory that the process that renews should not be the only process that tries:

• cert-watch.timer - ours, every 4 hours. Runs certbot renew -q (ARI-driven, a no-op outside the window) and then performs the health checks in the next section.
• snap.certbot.renew.timer - certbot's own stock timer, twice a day at randomized times. Kept deliberately: if our unit is ever broken, misconfigured or disabled, the stock path still renews.

Both call the same idempotent command, so overlap costs nothing. certbot renew consults ARI, sees "not yet", and exits - nearly all of the ~2,900 runs a year do exactly that. About two a week hit the open window and do real work.

reuse_key: the DANE pin that never moves

This zone publishes DANE TLSA records on all four encrypted transports: 3 1 1 pins, meaning "the certificate you receive must contain exactly this public key", stated as a SHA-256 digest of the SPKI and protected by DNSSEC. A pin like that and a certificate that changes twice a week would normally be a terrible combination - every renewal would need a TLSA rollover: pre-publish the new pin, wait out the DNS TTL, deploy, prune. Every 3.3 days. Forever.

reuse_key = True dissolves the problem instead of managing it. Certbot generates each renewal's CSR from the same key pair, so the certificate changes - new serial, new dates, new signature - but the SPKI digest is identical, and a 3 1 1 pin never notices. The archive table above is the proof: five certificates, including two obtained seconds apart in a drill, all carrying SPKI 68a92c99f947…. The pin was computed once, published once, and has not moved since.

The flag does not lock the key forever - it freezes it until you decide otherwise. Deliberate rotation is one forced renewal with --no-reuse-key, wrapped in the standard pre-publish rollover (add the new pin alongside the old, wait at least one TTL plus resolver caching margin, deploy, prune the old pin). The difference is that key rotation happens on your schedule, as a planned operation, instead of being forced on you 110 times a year. The 3-command recipe for computing the pin from a certificate is in the hardened-zone guide's DANE layer, together with the live records.

The watchdog: cert-watch, every four hours

Automation that nobody watches is a time bomb with a 160-hour fuse. The watchdog is deliberately boring: one shell script, run by a systemd timer every 4 hours, that renews and then checks three things - did certbot fail, is the certificate's remaining validity above the floor, and is the DANE pin still published where it should be. This is the production script, verbatim except for the alert transport (we use a Telegram bot; the token obviously stays out of published copies, and yours should too):

#!/bin/bash
# cert-watch - keeps the unified.dnsdoh.art shortlived lineage healthy:
#   1. certbot renew -q (ARI-driven eager renewal, no-op outside window)
#   2. alert if certbot failed
#   3. per-lineage validity floor alert (<40 h; ARI renews at ~80 h left)
#   4. DANE: unified-tlsa.py guard (alert-only - never auto-rewrites)
# Alerts repeat every 4 h until the condition clears - deliberate.

set -uo pipefail

LINEAGES="unified.dnsdoh.art"
WARN_HOURS=40
HOST="$(hostname -s)"

notify() {
    # Wire your alerting here: Telegram bot, ntfy, mail, webhook ...
    # Whatever you pick, keep the credentials OUT of the published copy.
    # e.g. curl -s -m 10 -X POST "https://api.telegram.org/bot${TG_TOKEN}/sendMessage" \
    #          -d "chat_id=${TG_CHAT_ID}" --data-urlencode "text=$1" >/dev/null || true
    :
}

renew_out="$(certbot renew -q 2>&1)"
renew_rc=$?
if [ $renew_rc -ne 0 ]; then
    echo "certbot renew FAILED (rc=$renew_rc): $renew_out"
    notify "🚨 [${HOST}] cert-watch: certbot renew FAILED (rc=${renew_rc})
${renew_out:0:500}"
fi

for lineage in $LINEAGES; do
    cert="/etc/letsencrypt/live/${lineage}/cert.pem"
    if [ ! -r "$cert" ]; then
        echo "cert file missing: $cert"
        notify "🚨 [${HOST}] cert-watch: certificate file MISSING: ${cert}"
        continue
    fi
    not_after="$(openssl x509 -in "$cert" -noout -enddate | cut -d= -f2)"
    left_h=$(( ($(date -d "$not_after" +%s) - $(date +%s)) / 3600 ))
    if [ "$left_h" -lt "$WARN_HOURS" ]; then
        echo "${lineage}: ${left_h}h left (<${WARN_HOURS}h) - renewal overdue"
        notify "🚨 [${HOST}] ${lineage} shortlived cert expires in ${left_h}h (threshold ${WARN_HOURS}h).
Renewal overdue - check: certbot certificates / /var/log/letsencrypt/letsencrypt.log
Expiry: ${not_after}"
    else
        echo "${lineage}: OK ${left_h}h remaining (expires ${not_after})"
    fi
done

guard_out="$(/usr/local/bin/unified-tlsa.py guard 2>&1)"
guard_rc=$?
echo "$guard_out"
if [ $guard_rc -ne 0 ]; then
    notify "🚨 [${HOST}] cert-watch: unified DANE guard rc=${guard_rc}:
${guard_out:0:300}"
fi

exit 0

The design decisions, since they matter more than the code:

• The first alarm is the certbot exit code, not the expiry date. A failed renewal alerts within 4 hours of the first failed attempt, while the certificate still has ~80 hours to live. The validity floor is the second net, not the first.
• The 40-hour floor catches silent failures. Exit code 0 does not guarantee a fresh certificate - certbot can be misinstalled, ARI can misbehave, a hook can leave a stale file deployed. Checking the actual notAfter on disk catches every such case: if renewal really happens at ~80 hours left, validity below 40 means at least ten runs have quietly done nothing.
• Alerts repeat every 4 hours until the condition clears. No deduplication, no cooldown. For a certificate this short, an alert you can silence is an alert you will eventually sleep through; the repetition is the feature.
• The script never fixes anything. It renews (which is idempotent and safe) and it reports. Automatic remediation during an unknown failure is how one broken thing becomes two.

The systemd pair that runs it - Persistent=true replays a missed run after downtime, and the randomized delay keeps the schedule from being metronome-predictable:

# cert-watch.service
[Unit]
Description=Shortlived certificate watchdog (unified.dnsdoh.art lineage)

[Service]
Type=oneshot
ExecStart=/usr/local/bin/cert-watch.sh

# cert-watch.timer
[Unit]
Description=Run cert-watch every 4 hours

[Timer]
OnCalendar=0/4:00:00
RandomizedDelaySec=600
Persistent=true

[Install]
WantedBy=timers.target

And what a healthy run looks like in the journal - this is today's, not a mock-up:

$ journalctl -u cert-watch.service -n 6 --no-pager -o cat
Starting cert-watch.service - Shortlived certificate watchdog (unified.dnsdoh.art lineage)...
unified.dnsdoh.art: OK 119h remaining (expires Jul 16 20:10:25 2026 GMT)
TLSA guard OK (68a92c99f947133f... on all 4 names)
cert-watch.service: Deactivated successfully.

Two green lines, six times a day. The value of the watchdog is not the days it says OK; it is that after a week of green runs, a red line is guaranteed to mean something.

The DANE guard: alert-only, by design

The last check in the watchdog deserves its own script, because it closes the loop between the filesystem and the DNS zone. unified-tlsa.py computes the SPKI digest of the certificate actually on disk and compares it against the TLSA records actually published in the zone - all four of them, _443 and _853, TCP and UDP. If the pin ever disappears or diverges, the next watchdog run says so. The same script also performs the two write operations a rollover needs, but the watchdog only ever calls guard. The DANE logic is the same for everyone; only the last hop into your DNS provider's API differs - so here is the core with that boundary marked, and the complete Google Cloud DNS version we actually run follows in the fold-out:

#!/usr/bin/env python3
"""DANE TLSA guard - provider-agnostic core.

Modes:
  prepublish - UNION-add "3 1 1 <current SPKI>" to each name (rollover-safe)
  guard      - exit 0 if the current SPKI is pinned on every name,
               3 if missing anywhere, 1 on error. No writes.
  prune      - rewrite each name to ONLY the current SPKI (destructive;
               run only after a rollover has fully propagated)
"""
import subprocess, sys

CERT  = "/etc/letsencrypt/live/<your-lineage>/cert.pem"
NAMES = ["_443._tcp.example.com.", "_443._udp.example.com.",
         "_853._tcp.example.com.", "_853._udp.example.com."]
TTL   = 300


def spki_sha256(cert_path):
    """SHA-256 of the certificate's SPKI - the '3 1 1' rdata payload."""
    pub = subprocess.run(["openssl", "x509", "-in", cert_path, "-noout", "-pubkey"],
                         capture_output=True, check=True).stdout
    der = subprocess.run(["openssl", "pkey", "-pubin", "-outform", "DER"],
                         input=pub, capture_output=True, check=True).stdout
    dg = subprocess.run(["openssl", "dgst", "-sha256", "-hex"],
                        input=der, capture_output=True, check=True).stdout
    return dg.split()[-1].decode().lower()


# [PROVIDER BOUNDARY] --------------------------------------------------
# Replace these two functions with your DNS provider's API calls
# (Cloudflare dns_records, Route 53 ChangeResourceRecordSets UPSERT,
#  deSEC rrsets, or RFC 2136 nsupdate for self-hosted BIND/Knot).
# The contract is deliberately small:
#   read_tlsa(name)          -> list of rdata strings, e.g. ["3 1 1 68a9..."]
#   write_tlsa(name, rdatas) -> publish EXACTLY this rdata set at TTL
# ----------------------------------------------------------------------

def read_tlsa(name):
    raise NotImplementedError("wire this to your DNS provider's read API")


def write_tlsa(name, rdatas):
    raise NotImplementedError("wire this to your DNS provider's write API")


def main():
    mode = sys.argv[1] if len(sys.argv) > 1 else "guard"
    want = f"3 1 1 {spki_sha256(CERT)}"

    missing = []
    for name in NAMES:
        rdatas = read_tlsa(name)
        if mode == "guard":
            if want not in rdatas:
                missing.append(name)
            continue
        target = sorted(set(rdatas) | {want}) if mode == "prepublish" else [want]
        if rdatas != target:
            write_tlsa(name, target)
            print(f"TLSA {mode}: {name} updated")

    if mode == "guard":
        if missing:
            print(f"TLSA GUARD: pin MISSING on {missing}")
            return 3
        print(f"TLSA guard OK ({want.split()[-1][:16]}... on all {len(NAMES)} names)")
        return 0
    return 0


if __name__ == "__main__":
    try:
        sys.exit(main())
    except Exception as exc:
        print(f"TLSA ERROR: {exc}")
        sys.exit(1)

#!/usr/bin/env python3
"""DANE TLSA management for the unified.dnsdoh.art lineage (main 4 names).

Modes:
  prepublish - UNION-add "3 1 1 <current SPKI>" to each name, keeping existing
               rdatas (rollover-safe). Writes/updates the marker file.
  guard      - exit 0 if current SPKI is present in every name's rrset,
               3 if missing anywhere, 1 on error. No writes.
  prune      - rewrite each name to ONLY the current SPKI (run at ddr
               decommission, >=3 weeks post-cutover, never before).
"""
import json, time, base64, subprocess, sys, tempfile, os
import urllib.request, urllib.parse

CERT = "/etc/letsencrypt/live/unified.dnsdoh.art/cert.pem"
SA_FILE = "/etc/letsencrypt/google_dns.json"
ZONE = "dnsdoh-art"
NAMES = ["_443._tcp.dnsdoh.art.", "_443._udp.dnsdoh.art.",
         "_853._tcp.dnsdoh.art.", "_853._udp.dnsdoh.art."]
TTL = 300
MARKER = "/root/cutover-snapshots/tlsa-prepublished-at"


def spki_sha256(cert_path):
    pub = subprocess.run(["openssl", "x509", "-in", cert_path, "-noout", "-pubkey"],
                         capture_output=True, check=True).stdout
    der = subprocess.run(["openssl", "pkey", "-pubin", "-outform", "DER"],
                         input=pub, capture_output=True, check=True).stdout
    dg = subprocess.run(["openssl", "dgst", "-sha256", "-hex"],
                        input=der, capture_output=True, check=True).stdout
    return dg.split()[-1].decode().lower()


def gcd_token(scope):
    sa = json.load(open(SA_FILE))
    b64 = lambda d: base64.urlsafe_b64encode(d).rstrip(b"=")
    now = int(time.time())
    claims = {"iss": sa["client_email"], "scope": scope,
              "aud": "https://oauth2.googleapis.com/token", "iat": now, "exp": now + 600}
    signing = b64(json.dumps({"alg": "RS256", "typ": "JWT"}).encode()) + b"." + b64(json.dumps(claims).encode())
    kf = tempfile.NamedTemporaryFile(delete=False)
    kf.write(sa["private_key"].encode()); kf.close()
    try:
        sig = subprocess.run(["openssl", "dgst", "-sha256", "-sign", kf.name],
                             input=signing, capture_output=True, check=True).stdout
    finally:
        os.unlink(kf.name)
    jwt = signing + b"." + b64(sig)
    data = urllib.parse.urlencode({"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
                                   "assertion": jwt.decode()}).encode()
    tok = json.load(urllib.request.urlopen(
        urllib.request.Request("https://oauth2.googleapis.com/token", data=data), timeout=15))
    return sa["project_id"], tok["access_token"]


def rrsets(base, hdr, name):
    q = urllib.parse.urlencode({"name": name, "type": "TLSA"})
    r = json.load(urllib.request.urlopen(
        urllib.request.Request(f"{base}/rrsets?{q}", headers=hdr), timeout=15))
    return r.get("rrsets", [])


def main():
    mode = sys.argv[1] if len(sys.argv) > 1 else "guard"
    want = f"3 1 1 {spki_sha256(CERT)}"
    project, tok = gcd_token("https://www.googleapis.com/auth/ndev.clouddns.readwrite")
    base = f"https://dns.googleapis.com/dns/v1/projects/{project}/managedZones/{ZONE}"
    hdr = {"Authorization": "Bearer " + tok, "Content-Type": "application/json"}

    deletions, additions, missing = [], [], []
    for name in NAMES:
        existing = rrsets(base, hdr, name)
        rdatas = existing[0]["rrdatas"] if existing else []
        if mode == "guard":
            if want not in rdatas:
                missing.append(name)
            continue
        target = sorted(set(rdatas) | {want}) if mode == "prepublish" else [want]
        if rdatas == target:
            continue
        if existing:
            deletions.append(existing[0])
        additions.append({"name": name, "type": "TLSA", "ttl": TTL, "rrdatas": target})

    if mode == "guard":
        if missing:
            print(f"TLSA GUARD: pin MISSING on {missing}")
            return 3
        print(f"TLSA guard OK ({want.split()[-1][:16]}... on all {len(NAMES)} names)")
        return 0

    if additions:
        change = {"additions": additions}
        if deletions:
            change["deletions"] = deletions
        req = urllib.request.Request(f"{base}/changes", data=json.dumps(change).encode(), headers=hdr)
        resp = json.load(urllib.request.urlopen(req, timeout=15))
        print(f"TLSA {mode}: {len(additions)} rrset(s) updated, status {resp['status']}")
    else:
        print(f"TLSA {mode}: already in target state")
    if mode == "prepublish":
        os.makedirs(os.path.dirname(MARKER), exist_ok=True)
        if not os.path.exists(MARKER):
            open(MARKER, "w").write(str(int(time.time())))
    return 0


if __name__ == "__main__":
    try:
        sys.exit(main())
    except Exception as exc:  # noqa: BLE001
        print(f"TLSA {sys.argv[1] if len(sys.argv)>1 else '?'} ERROR: {exc}")
        sys.exit(1)

The provider boundary is deliberately two functions wide: read the TLSA rrset at a name, and write an exact rdata set back. Every DNS API can express those verbs - Cloudflare's dns_records endpoints, Route 53's ChangeResourceRecordSets with UPSERT, deSEC's rrsets API, or plain RFC 2136 nsupdate against self-hosted BIND or Knot. Everything DANE-specific - the SPKI computation, the union-add rollover logic, the exit codes the watchdog reads - lives above the boundary and does not change. Our own zone is on Google Cloud DNS, where the write half is a zero-dependency OAuth token exchange (a service-account JWT signed with plain openssl - no SDK, nothing to install); the complete production script is in the fold-out above, verbatim.

Why guard mode never writes: if the pin is missing from the zone, something changed the zone - a bad migration, a fat-fingered console edit, a compromised credential. Every one of those calls for a human looking before anything writes to DNS again. An auto-repairing guard would also happily "repair" the intentional state mid-rollover, when two pins coexist on purpose. Detection is automated, repair is a decision.

One habit worth copying: after prepublish, the script drops a timestamp marker file. The prune step - the destructive one - can then refuse to run until the marker is old enough, which turns "wait at least an hour for DANE caches" from a thing you remember into a thing the tooling enforces.

Verify it from the outside

Nothing above requires trusting this article. The certificate, its lifetime, the IP SAN and the DANE pin are all observable from any machine on the internet:

$ echo | openssl s_client -connect 194.180.189.33:443 -servername dnsdoh.art 2>/dev/null \
    | openssl x509 -noout -dates -ext subjectAltName
notBefore=Jul 10 04:10:26 2026 GMT
notAfter=Jul 16 20:10:25 2026 GMT
X509v3 Subject Alternative Name: critical
    DNS:agh.dnsdoh.art, DNS:dnsdoh.art, DNS:gr.dnsdoh.art,
    DNS:www.dnsdoh.art, IP Address:194.180.189.33

$ dig +short TLSA _443._tcp.dnsdoh.art
3 1 1 68A92C99F947133F4BF4BC3F0A8CAD82DB0D7F51A923B09674420C5B F1BE5C2F

$ echo | openssl s_client -connect 194.180.189.33:443 -servername dnsdoh.art 2>/dev/null \
    | openssl x509 -pubkey -noout | openssl pkey -pubin -outform DER | openssl dgst -sha256 -hex
SHA2-256(stdin)= 68a92c99f947133f4bf4bc3f0a8cad82db0d7f51a923b09674420c5bf1be5c2f

The third check is time itself: run command 1 today and again in four days. The dates will have moved forward by one renewal cycle, and the digest in command 2 will not have moved at all. A changing certificate with an unchanging key, observable from outside, is this whole runbook in two lines of shell. The same pin answers on _853._tcp and _853._udp for the DoT and DoQ transports, each record signed by DNSSEC.

The failure drill, and the worst 80 hours

Before this lineage was allowed to serve production, it had to prove the renewal path twice on demand - rows cert2 and cert3 in the archive table, obtained 31 seconds apart on cutover day:

$ certbot renew --cert-name unified.dnsdoh.art --force-renewal

One drill answers four questions at once: issuance is repeatable on demand, the deploy hook actually reloads both daemons, the key (and therefore the DANE pin) survives a renewal, and the profile's rate limits accommodate back-to-back issuance. It also established the recovery property that makes incidents boring: because reuse_key keeps the pin valid, force-renewal is always safe here. Whatever went wrong, "get a fresh certificate right now" is never the step you have to hesitate over.

So the worst case runs like this, hour by hour. At ~80 hours before expiry the ARI window opens and the renewal fails - the CA is unreachable, the webroot broke, the account is rate-limited, whatever. Within 4 hours, the watchdog's certbot check fires the first alert, and repeats it every 4 hours. Behind the alerts, ~26 automated attempts remain before expiry across both timers, so transient failures - a CA outage, a network blip - heal themselves and show up only as a short alert burst that stops. If the failure is real, the 40-hour floor fires a second, independent alarm at hour 120. From the first alert to expiry is just over three days: enough to fix a webroot, re-register an account, or in the ugliest case issue a hostname-only certificate from a normal profile and accept degraded IP verification while sorting the rest out.

Since cutover the failure branch has never run beyond the drill. What has run is the boring branch: two natural renewals, on time, unattended, each observed by six watchdog checks a day - and one number that has not changed since 3 July: 68a92c99f947…

The runbook, condensed

Everything above, as the ordered checklist you would actually follow to build this from nothing. Each step is covered in detail in its section:

1. Serve the challenge directory. One webroot behind /.well-known/acme-challenge/ on port 80, answering for every hostname and for the bare IP - IP identifiers validate over http-01 only.
2. Issue once with certbot certonly --webroot: all hostnames, --ip-address, --required-profile shortlived, --reuse-key.
3. Inspect what came back: lifetime exactly 160 hours, the IP present in a critical SAN extension, an empty subject. Anything else means the profile fell through.
4. Attach the renew_hook that redeploys every consumer of the lineage - here, an nginx reload and an AdGuardHome restart.
5. Pin the key, not the certificate: compute the SPKI SHA-256, publish it as 3 1 1 TLSA records on every transport name, wait out the TTL, confirm with dig.
6. Install the watchdog (script and units above) and leave certbot's stock timer running as the independent second driver.
7. Adapt the DANE guard to your DNS provider's API - two functions at the marked boundary: read the TLSA rrset at a name, write an exact rdata set back.
8. Drill before you depend on it: force-renew twice, confirm both daemons reloaded and the SPKI digest did not move.
9. Verify from outside: from another network, check the served dates and SAN with openssl s_client and match the served key against the published pin.

Steps 1-5 are an afternoon. Steps 6-9 are what let you never think about steps 1-5 again.

Short does not mean fragile

This lineage cut over to production on 3 July 2026. Since then: five certificates issued (two of them drills), two natural renewals that arrived on the CA's schedule without a human involved, zero alerts outside the drill, and one SPKI digest that has not changed - which any reader can confirm from outside right now, and which makes this article unusually easy to fact-check as time passes: the dates in the verification section will keep moving, the pin will not.

The generalisable lesson is not about DNS. It is that certificate lifetime was never the real risk - unwatched automation was. A 90-day certificate renewed by a cron job nobody has looked at since 2023 is a quiet outage on a timer. A 160-hour certificate forced us to build renewal that is exercised twice a week, watched six times a day, alarmed on two independent signals, and drilled before it carried traffic. The short lifetime did not create fragility; it made the existing fragility impossible to ignore. With the industry's maximum certificate lifetime already legislated to shrink toward 47 days over the coming years, that is a trade every operator will eventually make - the IP SAN just made us make it early.

There is also an architectural reading. This resolver is built so that nothing is taken on faith: DNSSEC signs the answers, DANE pins the key, DDR designations verify against the IP, and each layer carries its own proof. A zero-trust design like that is only as honest as its most perishable link - and the certificate is the most perishable link on purpose. The renewal automation in this runbook is not an accessory to that architecture; it is the foundation that keeps the whole chain verifiable every hour of every day, instead of on the day someone last checked.

If you run a service that is an address rather than a name, the whole path is above: one issuance command, two scripts, two timers, one drill. The certificate this page was served over is the proof it holds up.