Agent index (llms.txt) — fetch this first, then the page it names.

Joining nodes

Adding a node to a running cluster is one command from your computer, using a single-use invite. This page covers the invite, the three kinds of node, servers that have only an SSH key, and how a node is removed.

The invite

An invite is one string. It names one node of the cluster: its public address, the domain to present to it, and the SHA-256 fingerprint of the TLS certificate it serves. The joining node connects to exactly that node and pins exactly that certificate. It does not resolve the cluster's domain (which reaches any nameserver, each with a certificate of its own) and it does not trust whatever certificate it is first shown.

• +Single use. Mint one per join.
• +Short lived. orama maint invite defaults to one hour, and the gateway caps it at one hour. sudo orama node invite --expiry 24h on a node can mint a longer one. The invite that orama node setup mints for its own run lives 15 minutes, since the install that uses it starts seconds later.
• +Needs the named node's certificate. The certificate is issued only after the domain is delegated (DNS and nameservers). That is why you delegate before you join.

orama node setup mints and uses the invite for you (below). To mint one yourself:

orama maint invite --env mycluster                  # names the lowest address the domain resolves to
orama maint invite --env mycluster --node 203.0.113.11   # name a specific node
sudo orama node invite --expiry 24h           # on a node already in the cluster
sudo orama node invite --raw                  # print only the string, for scripts

orama setup joins the machines you give it to the environment's cluster by itself: it mints each invite over SSH on a node that is already in, and orama setup --ip <new ip> with the same --env adds a node later (Join with orama setup). The rest of this page is orama node setup and the invite by hand.

Join with orama node setup

orama node setup \
  --ip 203.0.113.11 \
  --password \
  --env mycluster \
  --archive $ARCHIVE \
  --base-domain cluster.example.com \
  --role nameserver \
  --join-via root@203.0.113.10

--join-via <user>@<ip> mints the invite over SSH on a node already in the cluster, with your RootWallet key. Your computer does not need to resolve the cluster name, and you do not need to be signed in. That node's SSH host key must already be in ~/.orama/known_hosts (setup writes it there when it first pins the node).

Without --join-via, setup asks the environment's gateway for an invite, which needs orama auth login first. --gateway <url> overrides which gateway. --join-via and --gateway are alternatives: passing both is refused.

If a node is already running this exact build, setup does not upload the archive again.

Setup flags

The kinds of node

Every node, of either kind, runs the registry (RQLite voter), Olric, IPFS and IPFS Cluster, the vault guardian, the gateway and the Tor client. Put the first three machines in as nameservers.

Install joiners one at a time, waiting for each to finish. After each, run orama status --env mycluster and carry on only when every node is healthy. (orama status needs an operator sign-in, and a CA file for a staging cluster; orama status --ssh --env mycluster reads the nodes directly and needs neither.)

A server with only a key

Many images (OVH, most clouds) have no password login and open with a key you already own, often for user ubuntu. Do not pass --password. Pass that key once so setup can install the RootWallet key:

orama node setup \
  --ip 203.0.113.11 \
  --user ubuntu \
  --bootstrap-key ~/.ssh/id_ed25519 \
  --env mycluster --archive $ARCHIVE \
  --base-domain cluster.example.com --role nameserver \
  --join-via root@203.0.113.10

The bootstrap key is used once to install the RootWallet key and is never stored. The user needs passwordless sudo; setup checks that before installing. --password and --bootstrap-key are alternatives, and the command rejects both together. If you pass neither, setup assumes the RootWallet key is already on the server.

What a join does

1. +Pins the server's SSH host key and installs the RootWallet key.
2. +Verifies the archive on your computer (for --release, fetches and verifies the release first and has your RootWallet sign it), uploads it, and runs orama maint node install on the server, passing the invite on stdin (the hidden --secrets-stdin flag) so it never appears on a command line.
3. +The node validates the invite and the pinned certificate, receives the cluster's join response over HTTPS (archive trust list, secrets, mesh peers, nameserver information), and sets up WireGuard to every peer.
4. +It starts RQLite as a voter, joins the Raft group, and starts the remaining services in dependency order.
5. +It registers itself with the cluster and claims a slot if it is a nameserver.

A node installed without a token creates a new cluster. Never do that on a machine meant to join.

Removing a node

orama remove --node 203.0.113.12 --dry-run   # print the plan, change nothing
orama remove --node 203.0.113.12
orama remove --node 203.0.113.12 --offline   # the machine is already gone

It runs from a survivor. It first prints what removal costs every Raft group the node votes in (the registry and each namespace it serves) and refuses if any would lose quorum. Then it takes the node out of the Raft configuration, writes an eviction tombstone so nothing re-adds it, releases its mesh address, nameserver slot, namespace memberships, port blocks and TURN and SFU allocations, marks it retired so DNS purges its records, and wipes it (unless --offline). Every step is safe to repeat. Afterwards run orama node dns delegation --env mycluster and update the parent zone. --yes skips the confirmation.

orama remove adds what a newcomer needs on top of the cluster-side removal:

• +A node that signs for the validator set is refused (orama status shows which node it is). Erasing it destroys the validator's consensus key, and the validator is jailed for the blocks it misses. Move the key first (orama maint global validator migrate), or pass --drop-validator to remove it anyway. A node with the chain unit whose report says nothing about the chain, or whose chain is not answering, is refused too: the telemetry has to say it is not a validator. A jailed validator is not in the active set, so check orama chain validator before removing a node that was one. A cluster-only node has no key and is not asked.
• +A node with the global layer needs a decision about the chain. It may be registered with a bond. --chain-node-id <id> (the node's id in the registry; orama chain node <id> shows it) retires it there first, after checking that the chain's record lists this node's IPv4 address among its endpoints (a record with only hostnames or IPv6 endpoints cannot be compared, and the plan says so): a MsgRetireNode signed by your RootWallet, sent through a surviving node's chain over an SSH tunnel, which revokes the node's service keys and starts its bonds unbonding. --no-chain leaves the registration, and its bonds stay locked until you retire the node yourself with orama global retire. A machine that is already gone (--offline) cannot be asked: its registration, if it has one, is left alone and the plan says so; --chain-node-id retires it anyway, through a surviving node.
• +The wallet signs only for the chain the network runs. The retirement is signed for the chain id the environment's registry network names, and the chain of the node it is sent through must be that one; for an environment on no registry network, pass --chain-id <id>. A node that reports another chain stops the removal with nothing changed.
• +The chain's refusals stop the removal with nothing changed: a locked RootWallet, a node that belongs to another operator, or storage deals that still reserve bytes on the node.

orama node remove is the same cluster-side removal under its old name and prints a notice that it is replaced.

To swap a nameserver for a new machine without losing quorum, follow Replace a node.

If a join fails