Agent index (llms.txt) — fetch this first, then the page it names.

DNS and nameservers

An Orama cluster is its own DNS server. Its base domain, and every name under it (each deployment, each namespace gateway, each node, the TURN hosts) is answered by CoreDNS running on the cluster's nameserver nodes, from a table in the replicated registry. For the internet to ask your nodes, the parent zone of the base domain must delegate to them: NS records naming the nameservers, and glue A records giving their addresses.

This page explains the records, how to choose the base domain, how to create the records, and how to verify them.

Why you need your own nameservers

• +Deployments get names like myapp-f3o4if.cluster.example.com the moment they are created. A row is inserted, and the next DNS query sees it. There is no zone file and no external DNS provider.
• +Certificates are issued through the ACME DNS-01 challenge. The challenge record is served by your own CoreDNS, and the certificate authority finds that CoreDNS through the delegation. Until the parent zone delegates to the cluster, the genesis node cannot get a certificate. Before it asks the certificate authority, Caddy on a nameserver node checks that the challenge record is served by checking the node's own CoreDNS (resolvers 127.0.0.1:53), after a 15-second delay that lets the other nameservers reload the record. It never looks the zone up through public DNS: a lookup made while the node's CoreDNS was restarting and the domain not yet delegated would end at the parent zone, and Caddy keeps that answer for hours.
• +Wildcard names (*.cluster.example.com) work for every subdomain.

How a nameserver serves the zone

When a node is installed as a nameserver, orama-node starts orama-namespace-coredns@nameserver after the index RQLite is up. CoreDNS binds :53 and reads the zone from the index RQLite table dns_records, through /etc/coredns/Corefile. The installer writes the rqlite plugin's dsn as this node's WireGuard address (http://<wg-ip>:10100, from discovery.http_adv_address in node.yaml) with a username and password: rqlited listens on nothing else and always requires authentication. The plugin refuses a Corefile without dsn.

You need a domain you control (a whole registered domain, or a subdomain of one), one or more nodes installed as nameservers, each with a static public IPv4 address (three or more for redundancy: the cluster publishes exactly as many nameservers as it has), and access to the parent zone: the registrar's nameserver settings for a whole domain, or the DNS zone of the parent domain for a subdomain.

Nameserver slots

Each node installed with --role nameserver (--nameserver on orama maint node install) claims a slot, ns1, ns2, and so on, the first time its DNS sweep runs (every 30 seconds, once it has registered). Slots are claimed dynamically, lowest free number first, so you cannot know in advance which address holds which name. A node keeps its slot across restarts and missed heartbeats. Two minutes of silence takes it out of the round-robin, not out of its slot, because freeing the slot would drop the zone's glue. orama node remove releases the slot. There are at most 13 slots.

Each slot holder writes its glue record nsN.<base> pointing at its public IP (node.public_ip in node.yaml). The zone's own NS set and SOA derive from the slots: one NS record for every slot that has glue, and nothing else (while no slot is glued at all, because every nameserver missed its heartbeat at once, the last known NS set is kept rather than leaving the zone with none). A one-nameserver cluster publishes only ns1; a five-nameserver cluster publishes ns1 to ns5. The SOA names the lowest glued slot as the primary. Every node re-derives both on its 30-second DNS sweep, so a record set written any other way is brought back to the slots within one sweep. Install and upgrade write no zone records at all: a new nameserver's zone appears on its first sweep after orama-node starts.

Negative answers

A negative answer carries the zone's own SOA from dns_records in its authority section, the one naming the lowest glued slot. A zone with no SOA yet (no slot glued) answers SERVFAIL rather than inventing one.

The negative answer is NXDOMAIN only for a name that does not exist. Every other name that lacks the type asked is answered NOERROR with an empty answer section and the SOA (NODATA); an AAAA query for a name that has only A records is the common case. A name exists when:

A name that owns a record, or has names below it, is never covered by a wildcard (RFC 4592): _acme-challenge.<host>, which owns only a TXT row, is NODATA for A, not answered from *.<base>. The nearest wildcard that owns any record decides; a farther one is not consulted. Because the base zone carries *.<base>, a name under it that nothing else owns exists, and a type the wildcard lacks is NODATA there.

The two answers are not interchangeable: a resolver that gets NXDOMAIN for one type concludes the name does not exist at all and caches that for every type (RFC 8020), so the next A lookup of a name that is being served fails with "no such host". The rcode is cached with the answer, for 30 seconds, and never served stale. Whatever the depth of the name, a negative answer costs three indexed queries on the index rqlite (the typed read, one read of what the name and its in-zone wildcard candidates own, and the zone SOA); an answer from the name's own records costs one, from a wildcard two, and concurrent identical misses share one resolution. Whether a name has names below it (the empty non-terminal row of the table) is not asked of the database: it is a suffix match with no index, and the query comes from the internet, so it is answered from a set of ancestor names in the plugin's memory. The set is rebuilt from the table every refresh period (5 s in the generated Corefile; one read of the distinct active names per period, whatever the query rate) and extended at once by every record a lookup finds. An empty non-terminal is therefore NODATA as of the last refresh: a name whose only descendants are new is NXDOMAIN on a node that has not served one of them until the next refresh, and one whose descendants are gone stays NODATA until then. The set holds one entry per distinct ancestor name, about one per record (tens of MB at 10x the platform's records). If a query fails the answer is SERVFAIL (or a stale entry), never a guessed NXDOMAIN; a failure is logged once per 10 seconds with a count of those it suppressed.

Seeing which address holds which slot

Read it from the cluster:

orama node dns delegation --env mycluster

; cluster.example.com — create these in the parent zone example.com: as records in that zone, wherever its DNS is hosted
cluster.example.com.	IN	NS	ns1.cluster.example.com.
cluster.example.com.	IN	NS	ns2.cluster.example.com.
cluster.example.com.	IN	NS	ns3.cluster.example.com.
ns1.cluster.example.com.	IN	A	203.0.113.10
ns2.cluster.example.com.	IN	A	203.0.113.11
ns3.cluster.example.com.	IN	A	203.0.113.12

The fields are separated by tabs. When the base domain is a whole registered domain, the header reads .com: at your registrar, as custom nameservers with glue (host) records instead. After the records, the command asks DNS whether the parent zone returns them and says which NS or glue record is missing or points at another address, and it stores that answer on the environment. If the resolver itself cannot answer (a timeout, SERVFAIL), the command says the delegation is unverified and stores nothing; run it again once the records have propagated. --json prints the same as data, with delegated and findings for each domain. It reads the registry over SSH from the environment's recorded nodes, so it works before the cluster's name resolves, and it lists only slots whose glue exists. Run it again after adding or removing a nameserver, and update the parent zone to match.

NS records and glue

The NS records tell resolvers which servers are authoritative for the base domain. Because those servers are named inside the domain they serve (ns1.cluster.example.com serves cluster.example.com), resolving them would need the very servers they name. So the parent also carries glue: an A record for each nameserver name, handed out alongside the delegation.

Choosing the base domain

A subdomain (recommended)

Delegate a subdomain such as cluster.example.com from the example.com zone you already have. The NS and glue records are ordinary records inside the parent zone, created wherever that zone's DNS is hosted. The registrar is not involved and the rest of the domain keeps working.

This is the only option for a domain registered with Cloudflare Registrar, which does not let its domains use custom nameservers.

In the Cloudflare dashboard, for the parent zone: DNS, then Records, and add one row for each line the command printed:

The glue A records must be DNS only (grey cloud). A proxied record answers with Cloudflare's addresses and resolvers would send your cluster's queries there. Any DNS host works the same way; the records are what matter.

--cloudflare-token-file writes the records into the parent zone for you when it is hosted at Cloudflare, then checks DNS. It refuses a registry TLD (a parent with no dot, such as com):

orama node dns delegation --env mycluster --cloudflare-token-file /path/to/cloudflare-token

A whole registered domain

Use the registered domain itself (example.com) as the base domain. The NS and glue records then live at the registry and are set through your registrar: glue first (called host records, child nameservers or personal DNS servers), then the domain's nameservers switched to the nsN names.

• +Namecheap: Domain List, Manage, Advanced DNS, Personal DNS Servers: add each nsN.<domain> with its IP. Then Domain, Nameservers, Custom DNS, and enter the nsN.<domain> names.
• +GoDaddy: DNS, Hostnames: add each nsN with its IP. Then Nameservers, Change, Enter my own nameservers.
• +Cloudflare Registrar: not possible. Use a subdomain.

Registry changes can take hours to propagate (up to 48); most are visible within 1 to 4 hours. The cluster then answers for the whole domain, because its nameservers are the domain's nameservers. Prefer a subdomain unless the domain is dedicated to Orama.

Order: delegation before certificates

1. +Install the genesis node with --role nameserver --genesis.
2. +About 30 seconds later it holds ns1. Run orama node dns delegation --env <env> and create the records.
3. +Wait until dig NS <base> @8.8.8.8 shows the delegation. The genesis node's certificate is issued then. Caddy keeps retrying; nothing needs restarting.
4. +Join the other nameservers. The invite pins the genesis certificate, so it must exist.
5. +Run orama node dns delegation --env <env> again and add their records.

Full sequence: Install a cluster from scratch.

Installing the nodes

A first cluster, from three VPS through a deployed site, is Run your own cluster. orama node setup provisions a fresh VPS end to end. It reads the VPS password from your RootWallet vault (rw vault add <ip>), never from the command line, and installs the archive orama maint build printed:

# Genesis nameserver: creates the cluster
orama node setup --ip <ip> --password --env <env> --archive <archive path> \
  --base-domain <base-domain> --role nameserver --genesis

# Each further nameserver: joins it
orama node setup --ip <ip> --password --env <env> --archive <archive path> \
  --base-domain <base-domain> --role nameserver

To install by hand on the VPS instead, the genesis node is installed without a token and every other node with an invite (orama maint invite from your machine, or sudo orama node invite on a node already in the cluster):

# Genesis: creates the cluster
sudo orama maint node install --nameserver \
  --domain <base-domain> --base-domain <base-domain> --vps-ip <genesis public IP>

# Every other nameserver: joins it; the invite carries the node to join and
# the certificate to pin
sudo orama maint node install --token <invite> --nameserver \
  --domain <base-domain> --base-domain <base-domain> --vps-ip <this node's public IP>

A node installed without a token creates a new cluster, so only ever do that for the genesis node. --base-domain sets the base domain used for DNS routing and the zone CoreDNS serves; if omitted, the installer prompts for it.

Verify

orama node dns delegation --env mycluster    # what it should be
dig NS cluster.example.com @8.8.8.8           # what the internet sees; the same nsN names
dig A ns1.cluster.example.com @8.8.8.8        # glue; the IP printed for ns1
dig @ns1.cluster.example.com test.cluster.example.com        # your nameserver directly
dig @ns1.cluster.example.com myapp.node-abc123.cluster.example.com   # wildcard

For global propagation use dnschecker.org or whatsmydns.net. Inside the CLI, orama status dns --env mycluster shows CoreDNS, Caddy, certificate expiry and resolution tests.

How the records stay correct

Per-namespace records are round-robin sets: ns-<namespace>.<base> (the gateway host, origin of the signaling WebSocket and all RPC), its *.ns-<namespace>.<base> deployment wildcard, turn.ns-<namespace>.<base> (plain UDP and TCP TURN), turn-<namespace>.<base> (TURNS) and the stealth TURNS host, each with one A record per node serving that role.

These are created at provision or WebRTC-enable time, so these reconcilers keep them true as the topology changes. They run from every node: they are per-node and idempotent, so no leader election is needed.

A namespace's records are removed with its cluster: DeprovisionCluster, a rolled-back or health-failed provision, and the retry that discards a failed cluster row all delete the cluster's node membership first and then its DNS records (gateway, wildcard, TURN, stealth). Membership goes first because each node's sweep re-advertises itself for every cluster it is a running gateway of. That ensure skips a cluster that is failed or deprovisioning, and a namespace that is not in namespaces, so it cannot write a record back after a delete.

Two safety properties matter:

• +The staleness window (15 minutes) is far longer than the 120-second active-to-inactive threshold. That flag flips on a transient blip (rolling restart, brief rqlite unavailability), and a blip must never delete records.
• +The gateway-host purge never empties a name. A record is removed only when the same FQDN still has another resolvable (is_active = TRUE) record. Without that guard, a cluster-wide heartbeat failure could delete every record for a namespace host, and the result would not be a clean failure: the resolver rewrites a 3-label miss to the base wildcard, and *.<base> exists, so ns-<namespace>.<base> would silently fall through to the nameserver nodes, which may not host that namespace. Clients would connect, pass TLS on the wildcard certificate and reach the wrong backend. A silent misroute is harder to diagnose than an outright failure. The TURN purge is deliberately not guarded this way, on recoverability grounds: each live TURN node re-advertises itself on boot, so an over-purged TURN host repopulates, and for a relay-only client a host that resolves nowhere is no worse than one resolving to a dead node, because ICE falls through to the next server either way.

If a namespace host is down to a single record pointing at a departed node, the guard keeps it, because deletion would take the namespace fully offline. Add a replacement node and it re-advertises itself on the next sweep.

Serving node names

On a network such as stagenet, an operator can claim a short identification name for each node on the chain (MsgClaimNodeName, see Nodes), and <name>.<zone> must then resolve to the node. The zone is a dedicated sub-zone below the network's base domain, for example nodes.stagenet.orama.network, never the base domain itself: a name is claimed by anyone, and under the base domain a claim such as push or ns-myapp would add an address next to the hostnames the cluster publishes there. The cluster whose nameservers answer the zone serves the names from the chain, with no one writing a record by hand. A name is an address record (A or AAAA) for each literal public IP among the node's endpoints, and nothing more: no NS record and no glue, so a name delegates nothing. A private cluster does not use this; it serves its own domain with its own NS and glue (above).

Turn it on in node.yaml of the cluster's nodes that have the chain next to them (a global layer installed on the same machine):

http_gateway:
  base_domain: stagenet.orama.network
dns:
  node_names_zone: nodes.stagenet.orama.network   # empty or absent: this node publishes no names

Install the node with --node-names-zone nodes.stagenet.orama.network (orama maint node install, which writes the key into node.yaml), or write the key into node.yaml yourself and restart the node (orama node restart). The flag must name a zone strictly below --base-domain, and an install with any other value stops before it changes anything. orama setup --create-network <name> --domain <domain> passes --node-names-zone nodes.<domain> to every node it installs, because that cluster is the network's own and answers the network's names; a joining operator's private cluster (orama setup --network <name> --domain <domain>) passes none. An upgrade keeps it: orama node upgrade rewrites node.yaml from the template and carries dns.node_names_zone forward, and refuses to continue if the value in the file is no longer a valid zone. The zone must be strictly below the cluster's http_gateway.base_domain, in lower case with no trailing dot. The base domain itself, a domain elsewhere or a malformed value stops orama-node at start with the key's name in the message. Only the cluster that is the authority for the zone sets it.

Each such node pages the chain's NodeNames query (1,000 names a page) once a minute and keeps its dns_records equal to the answer, under the owner tag node-names:

• +It adds the records that are missing, reactivates one that something deactivated, and removes the records of a name that was released or whose node's address changed. Records are added before any is removed, so a name that moves never stops resolving.
• +It writes nothing unless it read the whole list. A chain that fails half way, a page key that repeats or more than a million names leaves the zone as it was and is logged as a failed pass (Node names sync failed).
• +It removes nothing while the chain node may be behind: its status says catching_up, or its newest block (latest_block_time) is more than five minutes old, which also covers a node that stopped or was cut off while still reporting caught up. A pass also removes at most a quarter of the rows it owns, or 50 rows for a small zone, and logs Node name removals held back for the rest, so a truncated read costs a minute of DNS and not the zone; the next correct read puts the rows back.
• +It publishes only a literal public IP. An address in a private, loopback, link-local, multicast or otherwise reserved range is refused and logged once (Node name not published), and so is a name that is not a valid label, a reserved label (the chain's reserved list is mirrored in core/pkg/nodenames and a test fails when the two differ) and a name that already has a record of another owner under the zone. Rows of any other owner are never touched.
• +Rows are written 100 to a request, each request one transaction, and one pass writes at most 2,000 rows; a large first pass, or a pass that ran into its 45-second limit, finishes over the next passes, a second apart. A row the database rejects is named in the log and does not hold up the others. Every node of the cluster runs the same idempotent sync, so losing one does not stop the zone.

A claim or a release is in DNS about a minute after it is in a block, plus the 300-second record TTL. Check a name with dig @<nameserver> alice.nodes.stagenet.orama.network, or read it from the chain with orama chain query orama.nodes.v1.Query/NodeByName '{"name":"alice"}'.

Troubleshooting

DNS not resolving.

1. +Is CoreDNS running? sudo orama node status on the node.
2. +Logs: sudo orama node logs coredns -f.
3. +Is port 53 open (TCP and UDP)? sudo ufw status, and your provider's firewall.
4. +Test locally: dig @localhost <base-domain>.

Glue not propagating. For a whole registered domain, glue is stored at the registry and can take up to 48 hours; confirm at your registrar it saved. For a subdomain, glue is an ordinary A record in the parent zone; check it is not proxied. A slot does not move because a heartbeat was missed. It moves when its holder is removed with orama node remove, or when that node's public address changes. If a glue record names an address that no longer answers, run orama node dns delegation --env <env> and update the parent zone.

A node name does not resolve. Is dns.node_names_zone set (a sub-zone of the base domain) on a node of the cluster whose nameservers answer the zone, and does sudo orama node logs node show Started node names sync? Node names sync failed names the cause: the chain not reachable from the node (the node must have the global layer next to it), or the registry not ready. A name whose node has no literal public IP among its endpoints has no record, and Node name not published says when an address, a reserved name or a name that another record already holds was refused. Node name removals held back means the chain node is catching up, or a pass reached its bound.

SERVFAIL. Usually CoreDNS configuration: check the Corefile syntax, RQLite connectivity and firewall rules. A zone with no SOA yet (no slot has glue) answers SERVFAIL rather than inventing one.

CoreDNS not responding. If dig @ns1.example.com example.com times out or returns no answer:

1. +Verify CoreDNS is running on the node with sudo orama node status. orama-namespace-coredns@nameserver should show active.
2. +Check its logs with sudo orama node logs coredns -n 50. Common causes: RQLite connection failures, port 53 already in use (a systemd-resolved conflict), Corefile syntax errors.
3. +Test locally on the node with dig @localhost example.com. If this works but external queries fail, the issue is a firewall.
4. +Verify port 53 is not held by systemd-resolved. On Ubuntu it listens on 127.0.0.53:53 by default. The installer handles it, but if it was re-enabled, sudo ss -tlnp | grep ':53' shows who holds the port.
5. +Run the node doctor: sudo orama node doctor.

Wrong IPs in glue records. If your node IPs changed after the initial setup, update the glue records (at your registrar, or in the parent zone) with the new addresses, wait for propagation (1 to 4 hours is typical for a registrar) and verify with dig A ns1.example.com @8.8.8.8. The node's heartbeat updates the internal DNS records (the A records in RQLite) when a node's IP changes, but glue records outside the cluster must be updated by hand: there is no API for this on most registrars.

Monitoring DNS with the CLI.

orama status dns --env mycluster                        # DNS and TLS health of the nameservers
orama maint inspect --env mycluster --subsystem dns     # deeper SSH-based DNS inspection

Security

Only expose what is needed: ufw allow 53/tcp and ufw allow 53/udp. The generated Corefile does not enable CoreDNS's health or prometheus plugins, so there are no extra ports. The recursive block for non-authoritative queries answers only loopback clients (an acl allowing 127.0.0.0/8 and ::1), so the node cannot be used as an open resolver. It shares one listener with the authoritative zone rather than binding 127.0.0.1 separately: a socket bound to 127.0.0.1:53 takes every local query, which sent the node's own zone out to a public resolver, and Caddy's ACME DNS-01 propagation check, which resolves through the node, never saw its challenge record. Consider rate limiting to blunt DNS amplification; that goes in the Corefile.

When running several nameservers: they share one RQLite cluster, so DNS records synchronise automatically; all must use the same --base-domain; install them in order, one at a time; and re-run orama node dns delegation --env <env> whenever the set changes.

Related

• +The CoreDNS rqlite plugin: core/pkg/coredns in the repository.
• +Deployments: full deployment instructions.
• +Architecture reference: system architecture overview.