DNS: next steps [meta]

I had a good chat with mig5 about the frontend implementation today, and I wanted to sum up our discussions here (as I seem to be doing this speech over and over again ;).

To get the real guts moving forward with DNS in the frontend, tasks should probably look something like this, in order of dependency (e.g. #2 depends on #1:

1. import the dev_dns code in a dev-dns branch in the frontend
2. port the code to the new services API (may be a noop), especially to associate zones with *master* DNS servers
3. make sure the editor actually works (that editing and saving nodes actually operates the database)
4. create a 'zone verify' task in the frontend that will call provision-zone in the backend
5. bring the guess_zone() (and create_host) logic upwards to the frontend
6. create zone records in the frontend on site manipulations: install/verify/delete/migrate/clone (right now records are created only in the backend)

For step #4 (and others), a big design decision that needs to happen is where the reference data sits (I'm not using the word "authoritative" here to avoid confusion with authoritative/master nameservers). Right now, the reference data is in the backend. By adding DNS zone structures to the frontend, we add a third source: the database, which adds confusion and creates all sorts of synchronisation issues. So in short we end up with those 3 data sources:

1. the zonefile itself - just a cache, we can ignore it for now
2. .php files that store a copy of zone structure - the current reference data (since #3 doesn't exist yet)
3. the mysql db - potentially the new reference data

This is important to decide how data propagates between the frontend and the backend. I tend right now to go for #3 as a reference source, if only because we have DNS servers that require *no* backend at all (e.g. MyDNS, PowerDNS, etc). This means, however, that *all* the data needs to be sent on zone verification, and that it will trash existing records, or at least make it very difficult to respect manual changes to zonefiles (see #922262: parse existing zone file records AKA non-destructive zone file edition for that other discussion, a potential solution would be to send *changes* instead of the full zonefile to the backend).

#5 also requires a bit more explanations. Right now, there's logic in the backend (the "guess_zone()" function) to figure out what records to create when a given site (say www.example.com) is created (guess_zone() will tell you the zone is example.com, unless the zone www.example.com exists, which would be weird but applies in cases like example.co.uk). If the frontend will hold the reference data, it needs to make those choices itself, while the backend just obeys its choices. In other words, the frontend *knows*, and the backend *does*, which is a recurring pattern in aegir (for example with access control). "create_host" is a command to test that functionality in the backend, and it should probably go away, as #5 would move that logic to the frontend, and the backend would just create the records the frontend tells it to create. (as a side note, I think we should still keep the rr-add/rr-del commands for testing purposes, but make it clear that data would be lost when the frontend comes around, unless we figure out a better way.)

Another question that was raised is whether we want to handle DNS on all domains, regardless of the user's desires. I believe we should create domains for any site created (e.g. www.example.com should create the example.com zone), if only for access control (e.g. i own example.com now). If aegir is the authority for this zone is up to the user (through the registrar): if Aegir's DNS servers are not registered for the domain, changes will have absolutely no effect (unless there's a misconfiguration, like the server's recursive DNS servers being also authoritative, which is usually bad bad bad).

We *could* allow the user to choose whether a zone should have DNS handling, however. This would improve performance of the DNS server (since it wouldn't handle domains needlessly) and remove potentially confusing misconfigurations. It would also allow Aegir to handle non-authoritative domains. So that could be a field for the zone that would be "authority", with the value: "master", "slave", "forbid", forbid simply meaning that no DNS records are created in the backend when the zone is handled.