Agent index (llms.txt) — fetch this first, then the page it names.

Run a global node

Reference. To join the network, start with Get started. This page is reference for running a global node by hand.

A global node runs the public Orama L1 (oramad) and, optionally, the services beside it: the storage provider, the history archiver, or the repair delegate. It needs no domain, no WireGuard and no cluster. This guide covers what the orama CLI does today on such a machine. The chain itself is described in What the chain is and Running a chain node.

To put a global node beside a cluster node on fresh machines, and register it on the chain, use orama setup: it runs the commands on this page for you, over SSH. This page is what they do, and what to run by hand. The same command creates a chain: orama setup --create-network makes each machine's chain home with --init-chain and a placeholder genesis, builds the real genesis from all the machines' keys, and installs it (Create a network).

Every command on the node runs as root. None of them needs raw systemctl.

Hardware

The code does not check hardware. These are the sizes the network plan gives:

Install

The staged directory is a release archive's bin/. An amd64 release (orama maint build, or a published one) carries everything the install reads: oramad (built with the Orchard library), its out-of-process verifier orama-orchard-verifier and the digest file beside it, orama-global, orama (this CLI; the chain unit runs its sign-floor check), Kubo's ipfs (v0.43.1) and the official cosmovisor release tarball cosmovisor-v1.7.3-linux-amd64.tar.gz. Once the release is extracted on the node, --staged-dir /opt/orama/bin and its manifest /opt/orama/manifest.json (the default of --manifest) are all the install needs. A directory you assemble yourself works too, if root owns it, nobody else may write it, and you give a manifest that lists its files. The installer refuses a symlink or a directory another account could change.

What is checked. Every file the install reads is hashed as it is read and must match the release manifest (--manifest), so a file that is not the one the release shipped is refused before anything on the host changes. The verifier must match its digest file, and oramad must pin that digest (a release links it in; oramad runs no other verifier). The cosmovisor tarball is checked against a SHA-256 pinned in the CLI, and ipfs --version runs as an unprivileged account to require Kubo v0.43.1. The manifest itself is trusted because the archive it came from was verified when it was extracted: by the signature of a wallet you trust, or by the release root. The chain unit runs the staged orama as root before every start (the sign-floor check), oramad holds the validator key, and orama-global holds hot keys and repair seeds; if you assemble the directory yourself, you are the root of trust for what you put in it.

sudo orama global install \
  --services chain,ipfs,provider \
  --public-storage-gb 500 \
  --staged-dir /opt/orama/bin \
  --persistent-peers <node-id>@<host>:31000,<node-id>@<host>:31000 \
  --init-chain --chain-id <chain-id> --moniker <name> --genesis /root/genesis.json \
  --enable-firewall --ssh-port 22

What the flags and the install do:

• +--services is chain plus any of ipfs, provider, archiver, indexer, repair. The Tor roles dirauth, relay (with exit for an exit) and onion, and a directory authority's bandwidth reporter (beside chain and dirauth), are described in Running the Tor network. The chain is required: the other services reach it only through its RPC on 127.0.0.1:31001. A dirauth or relay host needs no chain (the reporter does), and onion joins a chain that is installed already (the chain unit is not rewritten when chain is not named). Adding a service to a co-located machine later keeps the ports of the services already installed.
• +provider needs ipfs beside it, since it pins public deals through this host's public Kubo. provider and repair are never on the same host: a repair delegate holds repair seeds, and a provider must not. indexer is optional; add it on every node whose gateway serves the chain read API (loopback 31015, proxied by that node's own gateway, which answers 503 for /v1/chain/index/ where no indexer is installed).
• +Each service gets its own system account (orama-chain, orama-ipfs-pub, orama-provider, orama-archiver, orama-indexer, orama-repair; the ipfs service also creates the orama-ipfs-pub-rpc group, which its unit and the provider's unit name). Binaries go to /usr/lib/orama-global/bin, root-owned, 0755.
• +Units are written to /etc/systemd/system/orama-global-*.service and enabled, not started. None of them is part of orama-node.service. The public Kubo, provider, archiver, indexer and repair units are ordered after the chain unit and want it (After= and Wants=); the provider is also ordered after the public Kubo. The indexer unit is limited to one CPU (CPUQuota=100%) at a lower priority (Nice=10), so a heavy read or a catch-up from height 1 cannot take the cores the chain signs blocks on. The chain unit runs the sign-floor check before every start (see the double-sign guard below), with the orama CLI installed beside oramad.
• +The chain unit runs cosmovisor v1.7.3 (DAEMON_ALLOW_DOWNLOAD_BINARIES=false), which runs oramad from /var/lib/orama-global/chain/cosmovisor/current/bin. Install checks the staged tarball's SHA-256 against the pin built into the CLI (an unofficial or altered tarball is refused), installs only its cosmovisor file, and places the staged oramad as the genesis binary in that layout, with orama-orchard-verifier beside it (root-owned, 0755). The chain unit passes oramad --shielded-verifier /var/lib/orama-global/chain/cosmovisor/current/bin/orama-orchard-verifier, so a stagenet, testnet or mainnet node starts, and after an upgrade it runs the verifier that rides with the new oramad. A node installed before the verifier was staged gets it beside its unchanged oramad on the next install. The chain home must therefore already have a genesis: use --init-chain on the first install. A second install with the same oramad changes nothing; one with different oramad bytes is refused. To change the chain binary use orama maint global stage-oramad --upgrade <plan> --verifier <file> --verifier-target <name> (the new release's verifier goes in the same directory) and let cosmovisor switch at the plan's height; a plan with no staged binary halts the chain until one is staged. A patch that does not change consensus has no installed update path yet (the updater is not built): stage it as an upgrade plan.
• +The public Kubo (ipfs) is a second daemon, never the private cluster's: its own repo in /var/lib/orama-global/ipfs created with ipfs init --profile=server as orama-ipfs-pub, no swarm.key, the swarm on 31010 tcp+udp (open in ufw), private address ranges filtered and not announced, and only pinned content announced. Its RPC is 127.0.0.1:31011 (on a co-located machine, 198.18.0.2:31011) behind a bearer token in api-token (mode 0640, group orama-ipfs-pub-rpc, which only the provider joins; a second install keeps the token). The token allows only add, cat, pin/add, pin/rm, repo/gc and the read-only repo/stat, which the node report and the inspector read. The GC unit runs orama node ipfs-gc, which reads the token from the environment file gc.env (mode 0600), so it is never on a command line. The command also refuses an RPC address that is not on the host.
• +--public-storage-gb is the capacity you will declare with orama global capacity; Kubo's StorageMax is that plus 10%, and orama-global-ipfs-gc.timer (20 minutes after start, then every 6 hours) garbage-collects through the daemon's RPC, so a provider is not a free public cache. The timer starts and stops with the ipfs service.
• +With provider installed, PUBLIC_PIN and ARCHIVE deals are pinned in the public Kubo before the slot is accepted, so the bytes are fetchable by CID. PRIVATE deals never touch it.
• +--init-chain runs oramad init as orama-chain and puts --genesis in place (its chain_id must equal --chain-id). Without the flag the chain home is never created or changed, and the flag is refused when the home already has a genesis.
• +ufw: the chain's 31000 tcp+udp, the public Kubo's 31010 tcp+udp and the provider's 31013 tcp are allowed with the comment orama-global. A cluster reconcile never removes those rules. An inactive ufw is refused unless --enable-firewall is given; then incoming is denied by default, the SSH port is allowed, and ufw is enabled. --ssh-port must be a port sshd listens on according to sshd -T (its listenaddress ports when there are any, its port lines otherwise); otherwise the install is refused before anything changes. Loopback listeners (RPC 31001, gRPC 31002, REST 31003, Prometheus 31004) are not opened. The installer does not change IPv6.
• +oramad start refuses to start when config/app.toml in the chain home has query-gas-limit = "0" (unbounded) on any chain id that is not a localnet. --init-chain runs oramad init, which writes query-gas-limit = "2000000". A chain home created by an earlier build keeps its old app.toml unless the install is given --external-address, which rewrites the settings in the next bullets, query-gas-limit among them: either pass it, or set query-gas-limit = "2000000" there by hand before starting the new binary. A release can also be state-breaking, in which case a chain home from an earlier build cannot be carried over and the chain restarts from a new genesis.
• +--external-address <public ip>:31000 writes the chain's config.toml and app.toml (the settings chain/scripts/stagenet/deploy.sh used to sed in). The chain listens on the namespace address 198.18.0.2, which no peer can reach, so the node announces its public address instead; peer exchange is off, Prometheus listens on 127.0.0.1:31004, pruning is custom (keep 100 states, every 10 blocks), min-retain-blocks is 201600, query-gas-limit and iavl-cache-size are what oramad init writes, and the node cuts a state-sync snapshot every 1000 blocks and keeps two (the interval is a multiple of the pruning interval). A key the chain's template no longer has is an error, not a skipped line. The files must exist: use --init-chain on the first install.
• +--statesync-rpc (twice), --statesync-trust-height and --statesync-trust-hash make the node restore a snapshot on its first start instead of replaying the chain. The servers are two seeds' light-client routes (https://<host>/v1/chain/light, which the gateway serves; it forwards only status, commit, validators and consensus_params, and answers status without the node's listen_addr and rpc_address); the height and hash are a block both returned, read by orama setup through the two seeds (the newest block less 100, refused unless the seeds agree); the trust period is 7 days, shorter than the 21-day unbonding.
• +Running the command again with the same flags changes nothing but the binaries' bytes. A service left out of --services is not removed.

The repair delegate needs <home>/operator (the address deals name as repair_delegate) and <home>/deals/<id>.json files; the provider needs node-id after registration.

Start, stop, status

sudo orama global start            # chain, wait for its RPC, then the rest
sudo orama global start provider   # needs the chain running
sudo orama global stop chain       # stops the provider/archiver/repair first
sudo orama global restart chain    # restarts everything installed, in order
sudo orama global status

The chain always starts first and stops last. Before it starts, the double-sign guard below runs. The other services start only after http://127.0.0.1:31001/status answers (up to five minutes); if it does not, they are left stopped and the command fails.

Register and bond

The operator's wallet never touches the node. On the operator's machine:

1. +orama global bind signs each service key's binding.
2. +orama global register builds MsgRegisterNode: roles, hot key, bindings, endpoints and --asn, the autonomous system number this node declares. The chain refuses reserved, documentation and private-use numbers and cannot verify the number. A protocol deal slot goes only to a node with a declared ASN, and slots go to distinct ASNs, so declare the real one. With --node the RootWallet agent signs it and it is broadcast.
3. +orama global bond --role <role> --amount <norama> bonds each role.
4. +orama global capacity declares storage bytes for a provider.

Every command that signs a transaction (orama global register, bond, capacity, the validator commands, orama cluster register-onchain and retire-onchain, and orama storage create, grant and prove) returns only once the transaction is in a block, and prints its hash and height. Admission to the mempool is not success: a transaction the block refuses (out of gas, insufficient funds, a failed check) is reported as a failure with the chain's log, and one that is not in a block within two minutes is reported with its hash so it can be looked up.

A role is active only while its bond is at least min_bond. Until the genesis parameters are signed off, every role's minimum is 1 ORAMA, bond_per_gib is 1 ORAMA (1 ORAMA of storage bond backs 1 GiB), and unbonding takes 21 days.

Validators

Bootstrap committee

The first validators are a bootstrap committee named in genesis (oramad genesis add-bootstrap-validator, x/power). A member needs no stake: each has an equal share of voting power while the hand-over factor lambda is below 1. There is no on-chain application or scoring for a seat; the committee is whoever genesis names. 50% of a member's own reward share is force-bonded into its self-delegation until that reaches 2 × 1,000 ORAMA. At lambda = 1 a committee-only seat has no power.

Anyone else becomes a validator with a normal MsgCreateValidator, bonded from earnings if needed. The orama CLI does not build that message.

Rewards

x/emission mints once per epoch. The validator share is paid by x/power on capped power (5% per operator, or 3% once many operators are active), split between the validator (commission and its own delegation) and its delegators, and credited to earnings accounts, not bank balances. A delegation below 1 ORAMA is refused. See Validators for the power rules.

Using your rewards

Rewards sit in your earnings account until you move them. See what you have with orama chain earnings <your-address>, then choose:

orama chain withdraw-earnings 25            # earnings -> your own bank balance (25 ORAMA)
orama chain send <to> 5 --public            # pay another account in the open

Both are signed by your RootWallet, which shows the transaction first. withdraw-earnings can only pay you; the destination is not a field. send --public shows the sender, the recipient and the amount on the chain to everyone, permanently: the command prints that warning and asks you to type yes (--yes skips the question). orama chain send without --public is a private send, the default so that a payment cannot become public by omission. A private send needs a wallet that builds the shielded bundle, and the RootWallet cannot yet, so it stops with that explanation and sends nothing. Earnings can also go straight into the shielded pool (MsgShieldEarnings) by a tool that builds the bundle. Fees has the whole picture.

Slashing and tombstoning

A tombstoned validator can never unjail, and x/power gives it no power even while its tokens stay bonded. The slash applies to the bonded tokens and to unbonding started at or after the infraction.

For storage, x/storage slashes 10% of a replica's epoch price once two consecutive proof misses accrue.

Unjail and edit

orama global validator unjail --chain-id <id> --operator orama1... --fee <n> --gas <n> [--node http://127.0.0.1:31003]
orama global validator edit   --chain-id <id> --operator orama1... --moniker <name> --commission-rate 0.05 --fee <n> --gas <n>

Both are signed by the operator account (its bytes are the oramavaloper address). Without --node they print the sign document; with it the RootWallet agent signs and the transaction is broadcast. Unjail is refused while the jail runs, without a self-delegation at least the validator's minimum, and for a tombstoned validator. A bootstrap member with no self-delegation yet cannot unjail until force-bonded rewards give it one. edit sends every field you do not give as [do-not-modify]; one commission change is allowed per 24 hours.

The hot key

The provider and the archiver create their own signing key (hot-key, 0600) on first start and log its address. Registering a node needs a binding signed by this key (orama global bind --service hot-key, with the hot key's secret) so the chain knows the address is yours; a hot key that is another node's, or any operator's, is refused. It pays its transaction fees from its own bank balance or earnings, or from a fee-only balance the operator gives it with oramad tx nodes fund-hot-key [node-id] [amount-norama] (MsgFundHotKey, out of your earnings, or out of your bank balance with --from-bank); that balance can pay base fees and nothing else. A bank send from another user account to it is refused. orama node report and the inspector read the provider's hot-key balance from its monitor.json.

Back up the consensus key

sudo orama global validator export-key --recipient <operator X25519 public key, hex> --to /root/validator-key.orbk

This seals priv_validator_key.json to your public key with the same ORBK seal as a namespace backup. The node never holds the private half. Keep the file off the node.

Move a validator: the double-sign guard

Never run two copies of one key. The guard is the sign floor: a sign state root records, per validator key, in /var/lib/orama-global/validator-sign-floor.json ({"floors":{"<public key>":<priv_validator_state.json>}}). The public key is the one CometBFT signs as, derived from priv_key; a key file whose pub_key does not match its priv_key is refused.

The chain unit runs orama global validator check-sign-floor as root before every start (ExecStartPre), so it applies at boot, on Restart=always and on any manual start, not only to orama global start. The check refuses while a migration export is in progress. With any floor recorded, the chain starts only when a priv_validator_key.json is in the chain home, and, when a floor is recorded for that key, only when priv_validator_state.json is not behind it. Each key keeps its own floor: importing another key adds its entry and never removes one, and an import never lowers a key's floor; an export also refuses a chain home whose state is behind that key's floor. Exports and imports hold an exclusive lock (validator-sign-floor.lock) while they update the file.

A floor file in any other format (anything but one JSON object of that shape) stops the chain. A bundle cannot be imported again (its one-time key is gone), so repair the file by hand: move it aside, then write it again, root-owned, mode 0600, with one entry per migrated key. The key is the pub_key value of its priv_validator_key.json, or of its validator-key-*.json copy in /var/lib/orama-global. The state is its last known priv_validator_state.json: the newest of the chain home's data/priv_validator_state.json, the validator-state-*.json copies in /var/lib/orama-global, and the state file of any other host that ran the key. Check that state before you write it: a floor below what the key signed does not protect it.

The floor, the migration key and the key copies are trusted only while /var/lib/orama-global is root's and not writable by its group or others; otherwise every one of these commands, and the check, refuses. orama global install and orama global start print a warning when this host's key was migrated away.

1. +New host: sudo orama global validator migrate prepare prints a one-time key. The private half stays in /var/lib/orama-global/migrate-recipient.key, root's, 0600, until an import uses it or sudo orama global validator migrate cancel removes it.
2. +Old host: sudo orama global validator migrate export --recipient <key> --to /root/move.orbk (--to must not exist). It stops the chain and the services that need it and disables the chain unit. It writes an export-in-progress marker (/var/lib/orama-global/validator-export-in-progress) that makes the check refuse any start, reads the key and priv_validator_state.json, confirms the chain is still stopped, and seals both in memory. Then it records that state as the old host's floor, copies the state to /var/lib/orama-global/validator-state-migrated-<time>-<random>.json, moves the key to /var/lib/orama-global/validator-key-migrated-<time>-<random>.json, removes the marker, and finally writes the bundle. From then on the old host's chain does not start: the floor is recorded and the key is gone, so oramad cannot generate a fresh key and a zero state in its place.
3. +Copy the bundle to the new host.
4. +New host: sudo orama global validator migrate import --from /root/move.orbk. It refuses while the chain runs. It records the old host's state as the floor first, then writes that state (unless the new host's is already ahead), and installs the key last; a different key already there is moved aside, never overwritten. If a step fails, the migrated key is not installed, and the floor refuses its state should it be put in place by hand, so the chain cannot start signing as the validator below the old host.
5. +New host: sudo orama global start.

What this does not cover: a copy of the key made any other way (a disk image, a manual copy) or a key put back by hand together with an older state file. The floor is only on the hosts that ran these commands.

To abandon a migration, on the old host put both files back, owned by orama-chain, mode 0600: the key copy as config/priv_validator_key.json and the state copy as data/priv_validator_state.json in /var/lib/orama-global/chain. Then sudo orama global start enables and starts the chain; the floor passes because the state is the one it recorded. Do this only to abandon a migration, and only if the new host never started the chain with the key.

An export that was killed part-way can leave the marker behind; the chain then refuses to start until root removes it, after checking whether the key is still in the chain home.

Restore from a backup

Only when the old host is gone for good:

1. +New host: sudo orama global validator migrate prepare.
2. +Your machine: orama global validator reseal --from validator-key.orbk --identity-file <private key file, 0600> --recipient <key> --to restore.orbk.
3. +New host: sudo orama global validator migrate import --from restore.orbk --old-host-destroyed --floor-height <the network's latest committed height>.

A backup carries no sign state, so the import refuses it without both flags. Give the network's latest committed height H, read from a node you trust right before the import. The floor and the state become H+1, round 0, before any step: the restored key signs nothing at or below H, in any round. It can sign at H+1, so a vote the lost host cast at H+1 is excluded only if that host stopped before H+1 began. An import never lowers a floor recorded for the same key on the host; a bundle or height below it is refused. If the old host can still start with its copy of the key, it and the new host will double sign; the flag is your statement that it cannot.

Sharing a machine with a cluster node

A machine can be a cluster node and a global node at once (role: both). The global services run in their own Linux network namespace, orama-global; the cluster node stays in the root namespace. Inside the namespace 127.0.0.1 is the global services' own loopback, so the cluster's loopback (Kubo RPC 10107, the gateway's loopback trust, the Caddy admin socket) is not reachable from a global unit, and the two sides have separate port spaces: both can listen on the same port number. Users were already separate (orama-chain, orama-provider and so on against orama), and so are the state directories (/var/lib/orama-global against /opt/orama/.orama), which each unit already hides from the other.

sudo orama global install --colocated --services chain,ipfs,provider \
  --public-storage-gb 500 --staged-dir /opt/orama/bin --enable-firewall --ssh-port 22

Run it on a machine where orama node setup has already installed the cluster node. It refuses, before changing anything, when:

• +the machine has no cluster node (/opt/orama/.orama/preferences.yaml is missing; orama node setup would overwrite the co-located role if it ran later) or its role is global (a plain orama global install records that role on a machine with no preferences file);
• +it is not Linux, its kernel has no network namespaces or veth, or systemd is older than 242 (NetworkNamespacePath=). A missing ip (iproute2), nft (nftables) or sysctl (procps) is installed with apt-get first, the one change the install makes before this check (a stock Debian 12 image has no nftables); a machine with no apt-get, or an install that fails, is refused with the package to install. The check creates and deletes a throwaway namespace and veth pair to prove the kernel allows them, so a container without CAP_NET_ADMIN is refused here and not halfway through;
• +198.18.0.0/24 is already routed on the machine.

Without --colocated, an install on a machine whose role is both is refused: it would put the units back in the root namespace.

What it writes

On top of a global-only install:

Every orama-global-* unit (the public Kubo, its GC oneshot, the indexer and the cosmovisor chain unit included; the GC timer only triggers the oneshot and stays in the root namespace) gains BindsTo= and After=orama-global-netns.service, NetworkNamespacePath=/run/netns/orama-global, and a read-only bind of the namespace's resolv.conf over /etc/resolv.conf. The install also records role: both and global_netns: orama-global in preferences.yaml, after everything else. The node refuses role: both at boot unless that field and every file above exist; it then runs the same graph as a cluster node, because the global services are their own units, not components of orama-node.

Units are enabled and not started, like a global-only install: orama global start starts them, and its wait for the chain's RPC dials it at the namespace address. The provider and the GC oneshot reach the public Kubo's RPC on 198.18.0.2:31011 in the same namespace (the install writes that address as Kubo's Addresses.API, also over an existing repo config); the cluster's own Kubo (10107) is a different daemon on the other side.

The rulesets

• +Root namespace. The public ports of the chosen services (31000 tcp+udp for the chain, 31010 tcp+udp for the public Kubo swarm, 31013 tcp for the provider) are DNAT'd to 198.18.0.2; the namespace's outbound traffic is masqueraded. Nothing from the namespace may be forwarded to 10.0.0.0/8 (the WireGuard mesh is in it), 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16 (cloud metadata) or 100.64.0.0/10. Nothing arriving on the veth may reach a service on the host itself, so the namespace cannot get to the cluster through the host's public or WireGuard address, and only DNAT'd connections and their replies are forwarded into the namespace. The output chain lets only root and the account the cluster node runs as (orama, whose uid the install resolves; the install is refused when it cannot) open a connection to 198.18.0.2 on the host-only ports (31001, 31003, with the indexer 31015, and with the public Kubo its RPC 31011). Every local process shares the veth's route to the namespace, and a tenant deployment (a systemd dynamic user) must not reach the chain's RPC, REST API or indexer through it.
• +Inside the namespace. Input is default-drop except loopback, replies, the published ports, and the chain's RPC and REST API, the indexer's read API and the public Kubo's RPC (TCP 31001, 31003 and, when installed, the indexer's 31015 and Kubo's 31011) from 198.18.0.1, the host's veth address, alone. Forwarding is off; output to the private ranges above is dropped. The units' own IPAddressDeny= on the same ranges is a third layer for IPv4. Both rulesets are table ip, so they say nothing about IPv6: the namespace has none (IPv6 is disabled on ogl-host and inside the namespace before either veth end comes up), and the IPv6 private ranges are denied only by IPAddressDeny=.
• +ufw. Input rules cannot see DNAT'd traffic, so the install adds ufw route allow rules tagged orama-global (one for the namespace's own outbound traffic, from 198.18.0.2 in on ogl-host, one per published port to 198.18.0.2) instead of the allow rules a global-only install adds. A cluster reconcile does not remove them. The install also deletes the broader route allow in on ogl-host rule (any source) that earlier releases added.

What this does not isolate

It is one kernel and one root. The cluster node can reach the global services only through their published ports at 198.18.0.2, and a global service reaches the outside only through the masqueraded veth. The namespace has no IPv6 (the units may open only IPv4 and Unix sockets).

The chain's RPC (31001) and REST API (31003), the indexer's read API (31015) and the public Kubo's RPC (31011) are not published: on a co-located machine they listen on the namespace address 198.18.0.2 instead of loopback, the namespace firewall admits them from the host's veth address 198.18.0.1 alone, and they are not DNAT'd, so neither the public network nor the WireGuard mesh reaches them. The host reaches them directly: curl http://198.18.0.2:31001/status, and the Kubo RPC (bearer token, root only) at http://198.18.0.2:31011/api/v0/.... orama global register and the other signed transactions take --node http://198.18.0.2:31003. The global services' own clients (the provider, archiver, repair delegate and indexer) are given --rpc tcp://198.18.0.2:31001 by their units. The chain's gRPC (31002) stays on the namespace's loopback.

The cluster gateway's /v1/chain/ proxy reads them at those addresses on a co-located machine (it detects the installed layout; ORAMA_CHAIN_RPC_URL, ORAMA_CHAIN_REST_URL and ORAMA_CHAIN_INDEX_URL still override), and so do the node report and the inspector. The node report resolves the endpoints on every report, so it follows a co-located install without a restart. The cluster gateway resolves them when it starts: a gateway started before a co-located install keeps the loopback defaults for its /v1/chain/ route until the cluster node restarts. The install does not restart it (orama node restart takes the node's quorum duties with it, and a restart of one node at a time is the rule), and prints that the restart is needed: run orama node restart on the machine after the install, on one node at a time.

Re-installing over a running co-located node

Two things need attention:

• +The rulesets (netns-host.nft, netns.nft in /etc/orama-global) are loaded by orama-global-netns.service, a oneshot that stays active. Each file replaces its table in one nft -f transaction (it declares the table, deletes it and defines it again), so whenever that unit is active the install loads them itself (nft -f on the host, ip netns exec orama-global nft -f inside) with no moment without rules, whether or not they changed: a retry after a failed load therefore loads them too. It never restarts the unit: the global units are bound to it and would stop. A failed load fails the install and names the file. When the unit is not active, nothing is loaded; it loads the files when it starts.
• +The public Kubo's RPC moves to 198.18.0.2:31011 (Addresses.API) and the provider and GC units are rewritten to call it there, but the install starts and restarts nothing. Run sudo orama global restart afterwards: it restarts the installed services in start order, so Kubo listens before the provider starts (the install prints this when it installed the public Kubo).

orama global install --colocated --chain-client-user <name> (repeatable) adds local accounts to the set that may reach the host-only ports. Names are resolved to uids at install; an unknown one (or root, which is always allowed) refuses the install, and the set is kept in /var/lib/orama-global/netns-chain-clients so a later install without the flag keeps it. The set only grows: to drop an account, edit or delete that file and run the install again, which re-renders the host rules. The rule holds numeric uids, resolved at install, so an account that is deleted and re-created (a new uid) needs a re-install too. The stagenet deploy.sh passes each node's ssh login user, which the smoke tunnel (ssh -L ...:198.18.0.2:port) and orama chain run as.

Residual. The output rule names accounts, not programs: root, the allowed accounts and every process of the orama account (the cluster node, its gateways and the services it supervises) can reach the host-only ports, and so can anything on the machine that runs as them. A tenant deployment runs as its own dynamic user and cannot. Ad-hoc operator access (orama chain, curl to 198.18.0.2) therefore needs root (sudo) or an allowed account. The inspector asks through sudo -n on a co-located machine and reports an error, not an empty section, when that fails. The cluster gateway's chain proxy works because the gateway runs as orama. CometBFT's RPC has its unsafe routes off (rpc.unsafe = false, which oramad init writes and the chain unit never overrides).

Remove a global node

There is no orama global uninstall. Removal has a chain half, signed from the operator's machine, and a host half, done by hand as root.

On the chain. Each role is bonded separately, so each is unbonded separately:

orama global unbond --chain-id <id> --operator orama1... --id <node-id> --role <role> --amount <norama> --fee <n> --gas <n> --node http://127.0.0.1:31003
orama global retire --chain-id <id> --operator orama1... --id <node-id> --fee <n> --gas <n> --node http://127.0.0.1:31003

• +orama global capacity --bytes 0 first, for a provider: zero is a declaration of no capacity, and the chain refuses an unbond that would leave declared capacity above what the remaining bond backs. A declaration also cannot go below the bytes already reserved by deals, so let the deals end first.
• +orama global unbond moves the amount onto the unbonding queue. The queue entry is paid back to the operator at the end of the unbonding delay (21 days at the genesis default).
• +orama global retire retires the node. The chain records its service public keys so they cannot be bound again; register a new node, with new keys, to come back.
• +A validator also needs its delegations undelegated and its validator exit, which the orama CLI does not build; use oramad tx staking.

On the host. Stop the services through the CLI, then remove what the install put there:

sudo orama global stop

Then, as root: disable and delete the orama-global-*.service units and the orama-global-ipfs-gc.timer; delete /usr/lib/orama-global, the orama-global ufw rules (ufw status numbered), and the accounts the install created (orama-chain, orama-ipfs-pub, orama-provider, orama-archiver, orama-indexer, orama-repair, and the orama-ipfs-pub-rpc group). Before deleting /var/lib/orama-global, keep what you cannot recreate: the validator key (see "Back up the consensus key"; a key that is still bonded must never end up running on two hosts) and the provider's and repair delegate's key files.

On a co-located machine there is more. Removing the layout is not built: stop and disable the orama-global-* units and orama-global-netns.service, delete the files in the table above and the ufw route rules tagged orama-global, set role back to cluster in preferences.yaml, and put net.ipv4.ip_forward back to the value in /var/lib/orama-global/netns-prior-ip-forward before deleting /var/lib/orama-global:

sysctl -w net.ipv4.ip_forward=$(cat /var/lib/orama-global/netns-prior-ip-forward)

The stagenet deploy.sh reset does this last step itself.

Tests

The rendering of the unit and rulesets, the machine checks, the install plan and the boot-time role check are unit tests in make test. The layout built for real is make -C core test-netns (tag netns_integration, Linux and root only, skipped elsewhere): it creates the namespace and checks that the same address binds on both sides, that a cluster listener on the root loopback, on 0.0.0.0 and on 198.18.0.1 is unreachable from inside, that only published ports are reachable into the namespace, and that private ranges are unreachable from it.

Not built yet

• +orama node setup --role global from the operator's machine; install runs on the node.
• +An installed update path for a chain patch that changes no consensus behaviour.
• +An orama global command for MsgUpdateNode: an ASN is set at registration only.
• +An orama command for MsgRegisterOperator. register needs the operator to be registered first, and no CLI command builds that message (the stagenet deploy sends it with its own helper, stagenet-node).
• +Removing a service, or its firewall rule, that a later install leaves out; removing the co-located layout.
• +A remote signer (TMKMS, Horcrux) or sentry topology.