Moving a Public Resolver from Our AdGuard Home Fork to dnsdist and Unbound

What each piece does, in plain words

A resolver is the server your device asks "what is the address of example.com?". This one is built from a few programs, each with one job. If you already know them, skip to the next section.

Before the change, the front desk was AdGuardHome-edge, our own fork of AdGuard Home, and our fork of a fourth program, dnscrypt-proxy, sat behind Unbound to encrypt the upstream queries. Unbound stayed. It was the only part that was never a fork.

Why we changed it

AdGuard Home is built for a home or office network: a web panel, a query log, DHCP, per-device settings, all in one program, and it does that well. This resolver never ran it as released. From the start it ran AdGuardHome-edge, our own fork, because we were using it as a public edge on the open internet, which is a different job, and we changed it to fit: a smaller answer size limit, a cleared authority flag, ranked encrypted-resolver discovery, stricter DNS over QUIC handling, and more. Those changes lived in that fork and in our forks of the two libraries inside it, dnsproxy and urlfilter, plus a fourth fork, of dnscrypt-proxy.

That is a reasonable thing to do once. The cost arrives with every release afterwards. Each new upstream version had to be read, our patches carried across, and the result built and tested again before it could serve anyone. Sometimes a release fixed something we cared about. Often the honest conclusion of an afternoon's review was "nothing here for us", which is still an afternoon. None of this is a criticism of the authors. Releases are the point of maintained software. The cost was ours, because we had chosen to keep copies.

The model we wanted was the one Unbound already followed here: software that is configured, not patched, and updated by installing the new release. dnsdist from PowerDNS fit that model for the front desk. It is a single-purpose DNS front end, it speaks every encrypted transport we offer, and anything we needed that it did not do by default turned out to be configuration, plus a few small tools of our own that sit beside it rather than in the path of a query.

Where each job of the forks went

Before anything moved, we listed every behaviour the forks added and gave each one an answer: rebuilt in configuration, already built in, or knowingly dropped. The last column was checked on 10 October from a second server, not from the resolver itself.

"Encrypted-resolver discovery" is how Windows 11 and some other systems find out, from a plain DNS query, that this resolver also speaks DoH3, DoH, DoQ and DoT, and switch to one of them on their own. It is described in RFC 9462, and our earlier work on it is in verified DDR, end to end. It answered identically before and after.

Blocklists, still in AdGuard syntax

The blocklists did not change: the AdGuard DNS filter, OISD Small, the PhishTank and OpenPhish phishing list, and URLHaus. They are written in AdGuard's rule syntax, which dnsdist does not read. Rather than translate them by hand, a small program of ours fetches them once a day and compiles them into a database file that dnsdist looks names up in. It sits beside the DNS path, not on it: if it fails, dnsdist goes on answering from the last good database.

Two rules shaped it. Nothing is skipped silently: every rule the compiler cannot express is counted and reported per list. And a list that suddenly shrinks is not trusted: if a list arrives more than 25% smaller than last time, the previous copy is kept until someone looks. The whole database is replaced in a single transaction, so dnsdist sees either the old lists or the new ones, never a mix of both.

Today's compile: 223,250 blocked names from the four lists (181,791, 55,994, 34,530 and 2,775 rules before merging), and unsupported: 0 for each. How the blocking itself works for a reader's own setup is in how DNS ad blocking works.

Proving the answers matched before switching

For two days the new front desk ran beside the old one on other ports, reading the same blocklist database. A test suite on a second server asked both the same 19 questions and compared the answers field by field: every transport, discovery, the flags, the size limit, ANY, AAAA, blocked and allowed names, and malformed packets. Each run first checked itself against a known-good target and a closed port, so a broken test could not report a pass.

The rule was that the switch would happen only at zero unexplained differences. Three differences remained, all in the shape of an empty answer, and each was accepted deliberately:

• A query for the discovery name's A record: the old edge answered "no data" without the record that tells a client how long to remember that. dnsdist includes it, so the answer can be cached.
• A blocked name asked for its HTTPS record: the old edge left out that same record here, though it sent it for AAAA. dnsdist sends it for both.
• The refusal of ANY: the old edge advertised a 1,452-byte answer size in it, while everything else said 1,232. dnsdist says 1,232.

The switch: 2.4 seconds, and gaps we cannot fully explain

The two programs cannot share the ports, so the switch is: stop one, start the other. On 27 September it ran three times on purpose, forward, back as a rollback drill, and forward again, so the way back had been tested on the real system before it was needed:

A script's run time is not what a client sees. So a probe on the second server sent a query every 200 milliseconds per transport for 15 minutes across all three steps, and recorded the longest silence on each. These are the numbers from that evening's log. They cannot be measured again, because the old edge no longer exists:

Two of those are longer than the switch itself. UDP was silent for 4.2 seconds and DNS over QUIC for 5.2, against a 2.4-second script. We do not know where the extra time went. A client waiting out its own timeout before retrying, or a QUIC client reconnecting, would both explain it, and we tested neither. The probe also has limits worth stating: it reports only the longest gap per transport across the whole window, so the first forward step's gaps are hidden under the larger ones, and DoH3 was not probed at all.

Something that started working: DNS cookies

DNS cookies (RFC 7873, with the server side in RFC 9018) are a small token the server hands a client and the client sends back. A forged query from a spoofed address cannot carry the right token, because the token is computed from the client's real address.

That last part is where the old layout fell short, and nobody had noticed. Unbound mints the cookies, but behind the old edge every query reached Unbound from 127.0.0.1, so every cookie was computed for the same address. On 27 September we confirmed it: a cookie minted for one client was accepted when replayed from a different one.

dnsdist can pass the real client address to Unbound using the PROXY protocol, a short header in front of each query. Since 28 September, plain DNS on port 53 reaches Unbound that way, and cookies belong to the address they were issued to. Before turning it on, we added a test that fails if Unbound's configuration ever enables anything that would write those addresses to a log.

Retiring dnscrypt-proxy: no better, no worse, one trade

With the front desk changed, the next question was whether Unbound still needed dnscrypt-proxy behind it, or whether Unbound could encrypt its own upstream queries with DNS over TLS (RFC 7858). We tested both on 4 October with a copy of the live configuration, 500 real domain names per round, cold caches each round, and the two setups alternating.

The first run said DoT was worse, and the first run was wrong. Median 20.4 ms against 16.2, and a much longer tail. But the dnscrypt-proxy in that test was the live one, already warm from production traffic, while the DoT side started cold every round. Once both sides got the same 50-name warm-up, the picture changed:

DoT had the lower median in 4 of 7 rounds and the lower 95th percentile in 6 of 7. Seven rounds is fewer than we accept as a finding, so we read this as "no regression", not as "faster".

Unbound has forwarded over DoT since 18:36 (UTC+3) on 4 October. Against the last hour on dnscrypt-proxy, eight hourly windows the next day showed no slowdown and fewer SERVFAIL answers in all eight, 0.42% against 0.70%. That baseline is a single window, so we claim no cause for the lower rate. On 10 October Unbound's own counters, kept since its restart on 4 October, put the median time to fetch a name it did not have cached at about 13 ms, and SERVFAIL at 0.58% of answers.

A setting of ours that the review caught

Going through Unbound's settings during the move found one that predated it and was our own mistake. On 4 October, 13% of all answers were SERVFAIL, and the names failing most often ended in .top. The cause was harden-algo-downgrade: yes.

At the time, the root zone listed two signing algorithms for .top, 8 and 13, while the zone itself was signed with 13 only. With that setting on, Unbound demands a valid signature for every algorithm listed and rejects the zone when one is missing. Unbound's own manual says the option violates RFC 6840, which tells validators to accept any single valid path, and that it is meant for troubleshooting. We set it back to the default, no. Cloudflare and Quad9 had been answering those names correctly all along. As of 10 October the root lists only algorithm 13 for .top, so that transition is over.

What we gave up

• A reply to malformed queries. The fork answered them with FORMERR. dnsdist drops them without a reply. We judged that not worth a rule of its own.
• A fast failure for broken domains, as described above, when dnscrypt-proxy left.
• Our fork's panel and query log. In their place is a small dashboard of our own, reachable only over our private network, showing what is happening now. It receives client addresses only in encrypted, pseudonymised form. How the private network keeps admin panels off the internet is in zero open admin ports.
• The way back. AdGuardHome-edge was uninstalled on 4 October, so the tested rollback no longer exists. The code is archived, not lost: see where the old code lives, below.

Older guides on this site that describe the AdGuardHome-edge setup, such as the UDP listener that died without a log and the amplification investigation, are records of what happened at the time and stay as they were written.

Where the old code lives

Nothing was deleted. The public repositories of the old stack were archived on GitHub on 5 October: they are read-only, and they stay online so the commits, benchmarks and notes behind earlier articles can still be linked. Each public one carries a notice at the top saying it is no longer maintained and that nothing changes for users of the service. The AdGuard Home fork itself was always private.

Two repositories of today's stack are active. Ozy-666/unbound-edge is the build and configuration of the Unbound that serves today. It is not a fork: it downloads the official release from NLnet Labs at build time and applies no patches. Ozy-666/nginx-edge builds the nginx that terminates DoH3 and DoH, on BoringSSL. It is not a fork either, but it applies two small patches of its own at build time: server-side Encrypted Client Hello for BoringSSL builds, and a fix for the QUIC handshake deadlock we reported as nginx issue #1616, which is still open (the full write-up). dnsdist runs exactly as released, with no patches at all.

Our other public repositories hold measurement tools and reproductions, such as the transport benchmark behind our protocol comparisons. All of them are listed at github.com/Ozy-666.

Check it yourself

Every claim in the "checked from outside" column can be repeated from any Linux or macOS machine with dig. This is the output from our second server on 10 October, trimmed to the lines that matter:

# 1. discovery: three encrypted transports, ranked
dig @194.180.189.33 _dns.resolver.arpa SVCB +short

# 2. cookie, flags and size limit in one answer
dig @194.180.189.33 example.com +cookie +bufsize=4096 +noall +comments

# 3. a blocked name
dig @194.180.189.33 doubleclick.net +short

# 4. a name with a broken DNSSEC signature
dig @194.180.189.33 dnssec-failed.org +noall +comments

# 1 (complete)
1 dnsdoh.art. alpn="h3,h2" port=443 ipv4hint=194.180.189.33 key7="/dns-query{?dns}"
2 dnsdoh.art. alpn="doq" port=853 ipv4hint=194.180.189.33
3 dnsdoh.art. alpn="dot" port=853 ipv4hint=194.180.189.33
# 2 (excerpt)
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: b6b2115284d0c91f010000006aca9a134ab60be8aa5dd982 (good)
# 3 (complete)
0.0.0.0
# 4 (excerpt)
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 10466
; EDE: 6 (DNSSEC Bogus)

In the second answer, flags has no aa, the size offered is 1,232 although 4,096 was asked for, and (good) means the server returned a cookie that matches the one your dig sent. To check the encrypted transports rather than plain DNS, the per-protocol commands are in how to test whether your DNS is really encrypted.

The general lesson

A fork costs nothing on the day it is made. The cost is every release afterwards, paid in reviews, and it grows with the number of copies, not the size of the patches. If the changes you need can be expressed as configuration of software that already does the job, that is usually the cheaper road, even when the first version takes longer to set up.

Two habits from this move transfer to any resolver change. Decide what "the same" means before you switch, as a list of questions with expected answers, and refuse to switch while anything differs without an explanation. And measure the gap from outside: the script said 2.4 seconds, the clients saw up to 5.2, and only the second number is the one your users live with.