Security Changelog

Transparent updates on our infrastructure, security patches, and feature rollouts.

54 updates
SECURITY

Measured: why the hardest challenge runs at a few bits and the cheap one needs 26

A new guide measures the two work functions behind browser proof-of-work challenges in the same WebAssembly runtime a browser uses. One SHA-256 probe costs 0.3997 microseconds; one Argon2id probe at 32 MiB costs 154.6 milliseconds - a ratio of about 387,000 to one. That inverts the difficulty settings: a memory-hard tier runs at a handful of bits where the compute tier needs 20 or more, and pushing the compute tier to 26 bits costs a real user a measured median of 19.1 seconds, with a worst run of 64.4 while a rented GPU farm barely notices. A 12-minute flood on 7 August, 67,751 challenges to 923 addresses across 525 networks in 97 countries, produced zero incorrect proofs - and the article is explicit that this shows nobody attempted the work, not that nobody could. Updated 9 August: a fleet of native solvers now answers what the flood could not - a botnet can earn about 0.6 tokens per second per core at four bits, costing it roughly 16 times our verification cost. Two corrections are published with it: one separating the derived compute figures from the measured memory-hard ones, and one retracting the claim that memory-hardness taxes an attacker's parallelism, which measurement reversed. A second flood on 9 August, 1.18 million requests in twenty-four minutes, reproduced the result on three times the scale: 2,036 challenges issued to the swarm and not one attempted, with the machine at a load average of 1.31 on four cores and DNS still answering in 25 to 43 milliseconds. Updated 11 August: the benchmark scripts are published at github.com/Ozy-666/dns-pow-bench with the raw output of four measurement passes. Repeating the run put real error bars on it - the four-bit solve median moved by 69 per cent across three passes of the same measurement, while the per-probe costs that carry the argument reproduced within 2.7 per cent.

POST-MORTEM

Post-mortem: the QUIC Handshake That Failed One Time in Twenty

DoH3 on this resolver failed about one connection in twenty and logged nothing. The cause is in nginx: when a post-quantum ServerHello spans two QUIC Initial packets and one is lost, the loss is detected and the CRYPTO frame is requeued, but the congestion window has collapsed below the bytes in flight and the output loop sends acknowledgements only. The client cannot derive handshake keys without that frame, so it cannot acknowledge anything, so the window never reopens. Reported as nginx/nginx#1616 with a scripted reproduction; nginx proposed a one-line fix, #1617, which clears the reported case on a different TLS library than the one used to verify it. Extending the test found the limit of that fix: a certificate flight large enough to fill the 12000-byte initial window deadlocks again, and post-quantum certificates are that large.

MEASUREMENT

We Measured Our Own Four Encrypted DNS Transports

DoH, DoH3, DoT and DoQ all run on this resolver, so the same query can go through each of them and be counted. We measured round trips to a first answer, bytes on the wire per cold lookup, latency percentiles at 2% packet loss, and what the post-quantum key exchange costs. DoQ answers a full round trip before DoT and DoH; DoH3 is also QUIC but hands that round trip back to QUIC address validation, which a packet capture pinned down after we first blamed HTTP/3. Under loss the medians barely move while the tails separate completely: across 300 lookups per transport, both TCP transports land almost exactly one second above their own median at the 99th percentile, while the QUIC transports stop at 310 and 329ms. DoH3 is the transport that fails worst, and tracing that led to a bug in nginx itself, now reported upstream as nginx/nginx#1616 with a scripted reproduction.

POST-MORTEM

Post-mortem: Our Leak Test Was Inventing Leaks

Our DNS leak test reported a phone leaking queries to Google Public DNS. It was not. The probe zone is delegated to a host whose port 53 serves strictly as a recursive resolver, so probe queries ended in SERVFAIL. Quad9 retried 18 times, gave up, and the phone fell back to a secondary resolver 57ms later - which we then recorded as an extra 'leaked' resolver on top of the correct ones. The test itself kept working and kept finding real leaks throughout; the defect added entries rather than removing them, which is why it survived so long.

POST-MORTEM

Post-mortem: One Directive Refused Every HTTP/2 POST

A hardening change on this resolver set nginx's http2_body_preread_size to 16k. Any value below 65535 makes nginx refuse every HTTP/2 request carrying a body - every RFC 8484 DoH POST - for at least five weeks. A new post-mortem documents the code path, the measurement, and a reproduction anyone can run on localhost.

DOCUMENTATION

New Guide: Zero Open Admin Ports

A new how-to documents the pattern this server uses to hide its management plane: the AdGuard Home admin panel and SSH answer two devices over Tailscale and nobody else. Three gates - a localhost bind, an nginx geo allowlist, and a DNS split view from the resolver itself - plus one nftables rule for port 22, each verified from both sides.

SECURITY

Amplification Defence, Part Two: EDNS Clamp, DNS Cookies, and a 4 KB Bug

The two remaining defence layers are live: the public edge truncates UDP answers at 1,232 bytes whatever buffer size a client advertises, and a valid DNS Cookie now exempts a client from rate limiting. Testing the truncation promise end to end also surfaced a silent 4 KB limit in dnscrypt-proxy's TCP path that had been failing large answers for years - fixed.

DOCUMENTATION

Runbook Published: Running a 160-Hour Certificate

The unified certificate behind the site and all encrypted DNS transports lives 160 hours - the condition attached to the only free way to put an IP address in a SAN. The complete production operation is now documented: issuance, ARI renewal, the watchdog and DANE guard scripts published in full, and the failure timeline.

DOCUMENTATION

Runbook Published: The Missing NSEC

A DNS provider served our signed zone's delegation without the NSEC record proving DS absence, and validating resolvers refused the whole subtree with EDE 22. We diagnosed it across six providers, reported it with a one-line reproduction, and the platform-wide fix is now verified. The full account is published as a runbook.

UPDATE

Discovery Answers Now Carry Address Hints (One Less Round Trip)

Every DDR designation the resolver returns for _dns.resolver.arpa now includes ipv4hint=194.180.189.33, on DoH, DoT and DoQ alike. A bootstrapping device can open the encrypted handshake immediately instead of first resolving the target name, saving one round trip at the moment latency is most visible.

DOCUMENTATION

New Reference Guide: Anatomy of a Hardened DNS Zone

A new flagship guide walks through the real dnsdoh.art zone record by record: DNSSEC, DANE on all four transports, the HTTPS record with ECH, DDR discovery, mail lockdown and CAA. Published together with an infrastructure change: the ECH key now rotates monthly, automatically.

DOCUMENTATION

Runbook Published: When a Public Blocklist Bans Googlebot

A crowd-sourced IP blocklist this service imports had listed real Googlebot addresses. The firewall dropped them before nginx ever saw a request, and the site began to fall out of search. The full incident writeup is now published as a guide, with every tool included.

MAJOR UPDATE

Automatic Encrypted DNS Discovery Is Live (Verified DDR)

dnsdoh.art now answers Discovery of Designated Resolvers (RFC 9462) in strict verified mode. A device that knows only the resolver address 194.180.189.33 discovers the encrypted endpoints on its own, and Windows 11 fills in the DoH template automatically.

DOCUMENTATION

Two New Guides: DNS Caching and What Your IP Reveals

Two new evergreen guides: one explains DNS caching and TTL, including why a record can keep resolving to an old address after you change it; the other walks through what an IP address lookup actually shows about you, and what it does not.