When your NOTIFY won't work

Last week I was working on a DNS Migration project. During one of the migration phases, the old platform was acting as a hidden master to the new PowerDNS based platform.

My client had some issues getting their old platform to send proper NOTIFY messages to the new platform, which caused unwanted delays. An excellent occasion to get my hands dirty and use one of PowerDNS's underestimated features: LUA-AXFR-SCRIPT.

The Notify command

When a DNS Master has updated a zone, it might send out a DNS Notify command to its slaves to notify them a zone has been updated and a new version is available. If the Slave is configured to respond to those Notify signals, they would then initiate a zone transfer (AXFR or IXFR) to download the new version from the Master.

This behavior is described in RFC1996, which was published in 1996. It's the cross-vendor mechanism to trigger zone updates when using multiple DNS Server implementations.

In our case, the old platform claims to support the NOTIFY command, but we couldn't get the NOTIFY commands to reach our new DNS Platform.

LUA-AXFR-SCRIPT to the rescue

This PowerDNS feature, which is defined on a per-domain basis, allows you to change the contents of a zone after each AXFR zone transfer. In essence, a Lua function is called for every record in the zone, before committing the zone to the Backend. You can use this function to replace NS records, add a timestamp to a TXT file, filter out records you don't want to end up in your zone, or to other exiting things.

In my case, I wrote a tiny Lua-script that changed the SOA record of the zone we just transferred. As you might recall, the contents of the SOA record is a string of text consisting of the following fields:

• MNAME: the primary nameserver for this zone
• RNAME: the email address of the administrator (in a funny format, as we don't want @ symbols in the RNAME for historical reasons)
• SERIAL: the serial number (version number) of the zone
• REFRESH: number of seconds after which a secondary name servers should query the master to detect zone changes
• RETRY: number of seconds after which secondary name servers should retry to request the serial number from the master if the master does not respond
• EXPIRE: number of seconds after which secondary name servers should stop answering requests for this zone if the master does not respond
• TTL or MINIMUM: TTL for negative caching

The REFRESH parameter is the one I wanted to change: if a Slave gets a NOTIFY, it should immediately check if the Master has a new version of the zone. However, if it doesn't get a NOTIFY, it will actively check every Refresh seconds to see if a new version is available.

Changing the SOA record of all domains on the old platform wasn't an option (because $reasons), so I decided to change the SOA using an axfrfilter Lua script.

Configuring

The configuration happens in two parts.

First of all, we must create our Lua script. Create a .lua file somewhere on the PowerDNS Slave. In my case, I used /etc/pdns/xfer.lua as a file name.

function axfrfilter(remoteip, zone, record)
    
       -- Replace all refresh vars in SOA records
       if record:qtype() == pdns.SOA then
         resp = {}
         newcontent=string.gsub(record:content(), "^(.+)%s+(.+)%s+(%d+)%s+(%d+)%s+(%d+)%s+(%d+)%s+(%d+)","%1 %2 %3 120 30 %6 %7")
         resp[1] = {
	       qname   = record:qname():toString(),
	       qtype   = record:qtype(),
	       ttl     = record:ttl(),
	       content = newcontent
         }
         return 0, resp
       end 
	 

       -- preserve all others
       resp = {}
       return -1, resp

end

Notice the strange "gsub" command on line 6. Lua has funky pattern matching functions. If you know a better solution, please let me know in the comments.

Once we've created our Lua function, we need to configure our Slave domains to use that function.

For every slave domain, we need to set the LUA-AXFR-SCRIPT domain metavariable to the path of our script. In my case, I adjusted the migration script for our customer do to a pdnsutil set-meta domainname.tld LUA-AXFR-SCRIPT "/etc/pdns/xfer.lua".

Caveat Emptor!

At the end of this article, and before you start copy/pasting my code into your production servers, there are two major things I wanted to point out.

Dnssec

If your master is doing "offline" (I prefer the term "in-zone" or "in-backend") signing, your SOA record signatures won't match, as we've just changed the content. So either use a different signing strategy or don't use this when handling Dnssec zones.

Beware of Lua errors!

If there's a problem with your Lua function, the zone won't be committed to the backend. So make sure you test your script and keep an eye on the PowerDNS logs!

Alternatives?

If you don't want to go this route, there are a few alternative strategies you could consider:

• Fix the damn code! Instead of working around the problem, you could also try to fix the original problem. Why doesn't the Notify work? In our case, there were very good reasons not to touch the existing platform, so while we tried to configure it on the old platform, we quickly abandoned that path.
• Use a fake notify-er. There are a few tools out there, that can "fake" NOTIFY requests. You could set up such a tool, and have it send out NOTIFY signals every two minutes. Would work, but requires an additional tool, scheduling of the cronjob to send the false signals, etc.
• Fiddle with your backend. As a variation of the above strategy, you could have a script update your PowerDNS backend (often a database) to change the `last_check` column for instance.

Need help with your DNS Migration? Want the opinion of a PowerDNS Certified consultant? Get in touch!