Agent index (llms.txt) — fetch this first, then the page it names.
Wipe a node
How to take Orama off a server so it can be installed fresh, or so it can leave the cluster for good.
Use the CLI, and pick the right command
| Situation | Command |
|---|---|
| The node is, or was, a member of a running cluster | orama remove (from your computer; it uses a survivor) |
| The node is already retired from the cluster and you only need the machine erased | orama node wipe |
| You are rebuilding every node of a test cluster | orama node wipe --env <env> |
| The server is already deleted at your provider | orama remove --offline |
| The CLI cannot reach the machine (lost SSH key, never enrolled) | The manual steps below |
wipe erases the target and says nothing to the cluster. If the node is still a member, the survivors keep counting it toward quorum and keep re-adding its WireGuard peer. That is why remove exists: it checks that no cluster loses quorum, retires the node from Raft, the mesh, every namespace it served and the node registry (and, with --chain-node-id, from the chain), and only then erases it. It refuses a node that signs for the validator set unless you pass --drop-validator.
orama remove --node 203.0.113.12 --dry-run # the plan, and the quorum arithmetic orama remove --node 203.0.113.12 orama remove --node 203.0.113.12 --offline # machine already gone orama node wipe --env mycluster --node 203.0.113.12 # erase only orama node wipe --env mycluster --node 203.0.113.12 --nuclear # also remove shared binaries, the Tor package and Orama's system accounts
--yes (--force for wipe) skips the confirmation. Both are destructive. A wipe uses the node's RootWallet SSH key; afterwards that key is deleted from your vault, because a retired node's key opens nothing. If the vault cannot be reached the command says so and fails: run it again once RootWallet is unlocked.
What a wipe removes
| Category | What |
|---|---|
| App data | /opt/orama/.orama/ (configs, secrets, logs, IPFS, RQLite, Olric) |
| Binaries | /opt/orama/bin/… |
| Systemd | orama-node, every orama-namespace-* instance, leftover orama-* units, coredns, caddy, orama-deploy-*, the privileged-helper socket and service. Everything named orama-* under /etc/systemd/system goes too: the drop-in directories deployments leave (orama-deploy-build@<namespace>.service.d and its siblings) and the .wants links. A unit that outlives its file (a transient orama-namespace-wireguard@index, listed not-found active exited) is stopped and its failed state reset, then systemd reloads |
| Privileged helper | /usr/local/bin/orama-privhelper and its socket |
| WireGuard | /etc/wireguard/wg0.conf and the wg-quick@wg0 unit |
| Firewall | The ufw rules Orama added: those tagged orama (the cluster) and orama-global (a global node, route rules included), and the mesh accept rule in iptables. Your own rules and the rules for the ports sshd listens on stay, and ufw stays enabled as it is: the wipe runs over SSH. If sshd -T cannot be read no rule is touched, and the check below reports it |
| Accounts | Only with --nuclear: the system accounts Orama created below uid 1000 (the orama user, orama-* such as orama-sfu, orama-chain and orama-ipfs-pub, and ntfy) with their groups, after their processes are killed. A plain wipe keeps them, and a reinstall reuses them |
| Kernel settings | /etc/sysctl.d/99-orama-disable-ipv6.conf, on every wipe: the installer turns IPv6 off at every boot with it. The wipe sets net.ipv6.conf.all.disable_ipv6 and net.ipv6.conf.default.disable_ipv6 back to what the install found (each drop-in ends with # prior: <key>=<value> lines recording it), or to 0 (IPv6 enabled, the kernel default) when the install recorded none. Only with --nuclear: /etc/sysctl.d/99-orama-ram-hygiene.conf (suid dumps, core pattern, ptrace scope), with fs.suid_dumpable, kernel.core_pattern and kernel.yama.ptrace_scope set back to the recorded values; a machine installed before the values were recorded keeps its running values until the next boot, which no longer reads the drop-in |
| Configuration directory | /etc/orama itself, not only the files in it |
| CoreDNS and Caddy | /etc/coredns/Corefile, /etc/caddy/, /var/lib/caddy/ (certificates live in the cluster's shared store, not here) |
| ntfy | /etc/ntfy/ and /var/lib/ntfy/; the ntfy binary and account go with --nuclear |
| Global layer | /var/lib/orama-global (the chain with its keys, public storage, the relay), /usr/lib/orama-global (its binaries), /etc/orama-global and /etc/sysctl.d/60-orama-global-netns.conf, on every wipe. Stopping its units removes the orama-global network namespace, its ogl-host veth and its orama_global nft table, and the check below looks for each |
| Tor | The unit, /etc/orama/tor/torrc, /var/lib/orama-tor/ (the tor package stays unless --nuclear), and on a directory authority of a created network its keys in /var/lib/orama-tor-authority (the identity key and its passphrase), on every wipe |
| Anyone network leftovers | On a node never upgraded past it |
| Trust and update state | The archive trust anchor /etc/orama/archive-signers and its rotation mark, so a machine set up again trusts the cluster it next joins or creates, not this one. The auto-update timer is stopped first, then /var/lib/orama-autoupdate (fetched releases and the install intent), /etc/orama/update-notice.json and the adopted release root with its state (release-root.json, release-seen.json, release-staged.json in /etc/orama) are removed |
| Temporary files | /tmp/orama, build directories |
rm -rf is unlink, not cryptographic erase. Assume a decommissioned provider disk is readable.
The check after the wipe
The wipe ends by listing what of Orama is still on the machine: the paths above, anything named orama-* under /etc/systemd/system, units still listed by systemd, the wg0 interface and its iptables rule, the ufw rules tagged orama or orama-global that are not SSH rules, a kernel setting that could not be put back (LEFTOVER: sysctl <key> ...), and with --nuclear the binaries, the RAM hygiene drop-in and accounts. Each is printed as LEFTOVER: <what>. If there is any, the script says wipe INCOMPLETE and fails, and so does the command: it never reports success over state it could not remove. Fix the cause, then run the wipe again.
Not removed without --nuclear: the RAM hygiene drop-in /etc/sysctl.d/99-orama-ram-hygiene.conf, the system accounts, binaries in /usr/local/bin (rqlited, ipfs, ipfs-cluster-service, olric-server, coredns, ntfy, orama, orama-node, gateway, identity, sfu, turn, orama-sni-router), Caddy in /usr/bin, and the tor package with its apt source and key.
OramaOS nodes
This applies to Ubuntu-based nodes only. OramaOS has no SSH and no shell. To remove an OramaOS node call POST /v1/node/leave on the gateway API for a graceful departure, or reflash the image at your provider for a factory reset. See OramaOS.
Manual cleanup (when the CLI cannot reach the machine)
Only for a node whose SSH key is lost or that was never enrolled. Over your own SSH session, as root, and in this order:
- +Stop and disable
orama-autoupdate.timerand its service (an install in progress would restart the services you are stopping), then the privileged helper's socket, thenorama-node, then everyorama-namespace-*unit, then any leftoverorama-*,coredns,caddy,ntfyandwg-quick@wg0units. Kill leftoverorama-node,ipfs,ipfs-cluster-service,olric-server,rqlited,corednsandvault-guardianprocesses. - +Delete everything named
orama-*under/etc/systemd/system(units, timers,.service.ddrop-in directories),coredns.service,caddy.service, andsystemctl daemon-reload; thensystemctl stopandsystemctl reset-failedanyorama-*unit still listed, and reload again. - +Take WireGuard down (
wg-quick@wg0is a oneshot that stays "active (exited)" until you stop the unit) and delete/etc/wireguard/wg0.conf. - +Delete the
ufwrules taggedoramaororama-global, highest number first. Leave your own and the SSH rule. - +Delete
/opt/orama, the Orama environment files,/etc/orama,/etc/coredns,/etc/caddy,/var/lib/caddy,/etc/ntfy,/var/lib/ntfy,/var/lib/orama-global,/usr/lib/orama-global,/etc/orama-global,/etc/sysctl.d/60-orama-global-netns.conf,/var/lib/orama-tor,/var/lib/orama-tor-authority,/var/lib/orama-autoupdate. - +Read the
# prior:lines at the end of/etc/sysctl.d/99-orama-disable-ipv6.confand/etc/sysctl.d/99-orama-ram-hygiene.conf, set each key back withsysctl -w(IPv6 to0when there is no line), then delete both files. - +Delete the
oramauser and the otherorama-*system accounts and their groups (services must be stopped first oruserdelfails).
Then the machine is ready for orama node setup.
Multi-node: every node of a test environment
orama node wipe --env mycluster # every node in the environment orama node wipe --env mycluster --nuclear
Bringing a removed machine back
remove revokes the node's key: its row in node_credentials keeps its node_id and gains a revoked_at, so the machine's disk stops being a working credential and cannot re-admit itself. Re-admitting it is a join. Mint an invite and install as you would a new node (Joining nodes). The join clears the revoked row, and the machine records a fresh key on its next start. No manual SQL.
The same applies to a node that lost secrets/node-key.pem while keeping its peer identity: its new key will not match the one on record, the cluster will refuse it and the gateway log will say so. Re-join it; do not repair the row by hand.