How it works
A permissioned oracle network. Workflows are written in Rust and compiled to WASM; seven nodes run each one, agree on what it observed with a Tendermint-style consensus instance per call, sign a report, and deliver it to EVM chains through a Forwarder contract. Secrets are encrypted to a key only a threshold of nodes can use, and confidential handlers run inside an AWS Nitro Enclave. This page follows the repository's README and crate docs.
1. Architecture
The code is hexagonal. Core crates do no I/O, read no clock and hold no key; they talk to the outside only through traits (ports), and each adapter crate implements one port. crates/node is the composition root that wires them together. The consensus protocol is a pure state machine fed inputs and returning effects, which is what lets it run under a deterministic simulation with a hostile network and hostile peers (cxyz_testkit::consensus, one seed fixes a whole run).
2. The life of one report
A price feed as the example: a cron trigger fires on every node, each node runs the workflow, each fetches the price on its own, the network agrees on the median, every node signs the same report, and the nodes take turns delivering it until it is on chain.
Replay model
To the workflow, ctx.observe, ctx.write_report and the other calls simply return. Underneath, the node stops the workflow at the first call whose result is not known yet (the host ends the WASM instance instead of returning). When the result is known it is appended to the execution's journal and the workflow is run again from the start in a fresh instance; calls already in the journal are answered from it at once. The journal is the whole state of an execution. That is why a handler must be deterministic: no wall clock, no randomness, no floating point in results, and nothing from inside observe used afterwards except its return value.
Who decides what
- Every node derives the same execution id for the same occurrence: sha256 over the workflow, the handler and the trigger occurrence. Consensus instances of the same call find each other by it.
- A node only runs occurrences that come after it got the workflow; an occurrence fewer than a quorum of nodes run cannot finish.
- The report's
configVersionis the version of the on-chain node set the node follows. When the node set changes, every report still on its way is signed again for the new set. - Delivery: each node's turn lasts
delivery_turn_millis(5 000 ms by default); the order of the nodes is taken from the report digest so that the same node does not always pay. A node whose turn comes withoutf + 1signatures only checks whether the report is already on chain. A delivery that failsdelivery_attemptstimes on one node fails the call on that node. - The Forwarder accepts a report signed by exactly
faultTolerance + 1distinct signers of the node set in force, in strictly increasing address order, and never delivers the same(receiver, executionId, reportIndex)twice. If the receiver reverts, the whole delivery reverts and can be retried.
3. Consensus
Each node observes something on its own; one consensus instance makes the honest nodes agree on a single set of signed observations and on the value an aggregator derives from it: the median of integers, or the value most nodes saw identically. The protocol is the Tendermint consensus algorithm (Buchman, Kwon and Milosevic, Algorithm 1) run once per instance instead of once per block height.
What makes it work on a network that loses, duplicates and reorders messages, with peers that tell different nodes different things:
- A re-proposed value carries the prevotes that justify it, and those count even when their author told this node something else.
- Nodes periodically send what they said in the current round again.
- A node sends a peer still voting in an earlier round every vote it knows of that round and of its own.
- A node that has decided answers anyone still voting with a certificate of the decision, which is enough to decide without having taken part.
None of these touches the voting rules, on which safety rests. A node whose data source is down still votes on what the others observed.
What the consensus crate does not do yet
- Survive a restart. Safety assumes an honest node votes once per round. A restarted node keeps no state, so it must not rejoin a running instance.
- End an instance. One that cannot decide, because too few nodes observed anything, keeps trying; giving up is the caller's decision.
- Cope with nodes that disagree about each other's keys. If some nodes accept a node's votes and others cannot verify them, the two groups count different votes; a seven-node network split four to three can stall for good (it happened in a test). Safety is not affected. See the key replacement procedure below.
4. Talking to chains
No single RPC provider is trusted. A node is given several endpoints per chain and a quorum that must be a strict majority; QuorumRpc in crates/adapters/evm-rpc asks every endpoint at once and only returns what that many agree on: block hashes, call results pinned to a block hash, logs, receipts. Nothing else in the node talks to a chain, and this page reads the chain through the same type.
- Endpoints behind one provider are one source, not several: the protection is only as good as the independence of the endpoints. Each node should use different providers from the other nodes where it can.
- Agreement is no protection against the chain itself: an agreed block can still be reorganised away unless it is final. When a node treats a block as final is configured per chain (
finalized,safeor a number of confirmations). Reads and event triggers only use final blocks, and a report counts as delivered once the block of its transaction is final. - The node set is followed at the latest block: the Forwarder checks reports against the node set of the latest block, and a node set never decides what a node signs, only whether its signature counts.
- Methods that combine answers instead of requiring equal ones (next nonce: the highest; fees: the median; gas estimate: the highest) document how far one endpoint can move the result.
- A chain reached through fewer than two agreeing endpoints is refused by the node's configuration.
5. Secrets and confidential compute
The vault key
- Threshold encryption. A secret is encrypted to the network's public key with TDH2 (a published scheme with a security proof, on curve25519-dalek) and can only be decrypted by a threshold of nodes together: five of seven in production. Fewer learn nothing.
- Generated, never whole. The nodes run FROST's distributed key generation (the Zcash Foundation's audited frost-ristretto255, used unchanged).
crates/keygenadds a confidential channel for round 2 (packages sealed with HPKE to their recipient), a confirmation round in place of a broadcast channel (every node confirms what it saw and computed, and the ceremony succeeds only if all confirmed the same), and tolerance of lost, duplicated and reordered messages. Any misbehaviour fails the ceremony; it is run again. - Published only with every node's signature. Each node signs the key it got;
KeyRegistry(no owner) records a key only with one signature from every node of the node set in force, insigners()order, and gives it the next epoch. - Refresh and repair. A refresh gives every node a new share of the same key (the public key stays, verification keys change). A repair gives one node the share of its position back from a threshold of helpers. The node set and the threshold are fixed for the life of a key.
- Bound to a label and to code. A ciphertext is bound to a label it cannot be moved from, and nodes refuse altered ciphertexts. For confidential handlers the label is made from the hash of the workflow's binary, so only that exact code can ever be given the secret.
- Decryption parts open one ciphertext only and prove they were computed correctly, so a faulty node cannot spoil a decryption and nothing about its key share leaks.
A confidential handler
6. Data schemas
Report and its EIP-712 signature
struct Report {
bytes32 workflowId; // the workflow that produced it (hash of binary + configuration)
bytes32 executionId; // identical on every node for the same trigger occurrence
uint32 reportIndex; // the number of the workflow call that produced it
uint32 configVersion; // version of the node set the signers belong to
address receiver; // the only contract it may be delivered to
bytes payload; // for the receiver, e.g. abi.encode(int256 value, uint64 observedAt)
}
REPORT_TYPEHASH = keccak256("Report(bytes32 workflowId,bytes32 executionId,uint32 reportIndex,uint32 configVersion,address receiver,bytes payload)")
structHash = keccak256(abi.encode(REPORT_TYPEHASH, workflowId, executionId, reportIndex,
configVersion, receiver, keccak256(payload)))
domain = EIP712Domain(name "CxyzForwarder", version "1", chainId, verifyingContract = Forwarder)
digest = keccak256("\x19\x01" || domainSeparator || structHash)
signature = 65 bytes r || s || v, secp256k1, by the node's report key
transmissionId = keccak256(abi.encode(receiver, executionId, reportIndex)) // replay protectionWorkflow ↔ host ABI
guest exports (module "cxyz", ABI_VERSION = 4)
memory
cxyz_abi_version() -> u32 must equal ABI_VERSION
cxyz_alloc(len: u32) -> u32 room for the host to write into
cxyz_describe(ptr, len) -> u64 configuration in, encoded WorkflowManifest out
cxyz_handle(handler: u32, ptr, len) -> u64 HandleRequest { event, config } in, HandleResponse out
cxyz_confidential(ptr, len) -> u64 only ever called inside an enclave
guest imports from "cxyz"
log(level: u32, ptr, len)
call(ptr, len) -> u64 CallRequest in, CallResult out; if the result is not known yet
the host ends the instance instead of returning
buffers: (ptr, len) packed into one u64; messages encoded with postcard;
a fresh instance for every run, so the guest keeps no state between executions
CallRequest = HttpGet { url }
| Agree { aggregation: Median | Identical, observation: Option<Value> }
| WriteReport { chain_id, receiver: [u8; 20], payload }
| EvmFinalizedBlock { chain_id }
| EvmCall { chain_id, to, data, block_number }
| Confidential { handler, input, secrets }
| Secret { name } (inside an enclave only)
| Http { method, url, headers, body } (inside an enclave only)Node configuration (node.toml)
Every node gets the same peers in the same order, the same fault tolerance and the same workflows; paths are relative to the file. The node set is not configured: it is read from the registry the Forwarder names. The key file holds the three secrets and is readable by the node only.
node = 0 # this node's position among the peers
listen = "0.0.0.0:7000"
keys = "node.keys" # consensus (Ed25519), report and transmitter (secp256k1) secrets
fault_tolerance = 2 # seven nodes survive two faulty ones
executions = "executions.jsonl" # optional: a record of every execution
delivery_turn_millis = 5000 # optional: each node's turn to deliver
[[peers]] # every node, this one included, in the same order everywhere
address = "node-0.example:7000"
consensus_key = "0x…"
also_accepts = ["0x…"] # optional: while that peer's key is being replaced
[[chains]]
id = 84532
rpc = ["https://…", "https://…", "https://…"]
quorum = 2 # a strict majority of the endpoints
finality = "finalized" # or "safe", or { confirmations = 12 }
forwarder = "0x…" # without it the chain is only read
[[workflows]]
wasm = "price-feed.wasm"
config = "price-feed.conf" # optionalExecution record (executions.jsonl)
{"execution":"<64 hex>","workflow":"<64 hex>","handler":0,
"trigger":{"cron":{"scheduled_at":1700000000000}},
"started_at":1700000000123,
"outcome":"succeeded","value":4200,
"logs":[{"level":"info","message":"agreed on 4200"}]}
failed: "outcome":"failed","reason":"…" instead of "value"
evm log: "trigger":{"evm_log":{"chain_id","block_number","block_hash","transaction_hash",
"log_index","address","topics","data"}}
times are milliseconds since 1970, bytes are 0x hex; one line per finished execution,
written with a single write to a file opened for appending; the node never reads it backStatus (status.json)
{"at": 1795057064040, "node": 0, "started_at": 1795050000000,
"consensus_key": "0x…", "report_signer": "0x…", "transmitter": "0x…",
"workflows": ["<64 hex>", …],
"node_sets": [{"chain": 84532, "version": 1, "fault_tolerance": 2,
"signers": ["0x…", …], "member": true}],
"executions": 1, // started, not finished
"reports": 0, // signed, not yet seen on chain
"agreements": [{"execution": "<64 hex>", "call": 1, "round": 0,
"step": "propose" | "prevote" | "precommit", "observations": 6}]}
replaced about every 5 s; this page treats one older than 20 s as not reporting.
Agreements, executions or reports that stay for minutes are stuck.Contracts
7. Operating a network
Restarting a node
Safe at any moment, including by killing it. It comes back knowing nothing of what it was doing: executions in progress are lost on that node, and it only takes part in what is triggered after it is up again. The rest finish without it. Restart one node at a time and wait for an execution line with "outcome":"succeeded" before the next: each node that is down or freshly restarted uses up one of the faults the network tolerates. Chain events that happen while a node is down are not handled by that node, and a node's clock must not be set back across a restart to before an occurrence it already took part in: it would vote on it a second time.
Changing the node set
The owner of NodeRegistry calls setConfig; nothing is restarted. Each node reads the node set from the registry its Forwarder names and, when it changes, signs every report still on its way again for the new set. A node whose signing key is not in the set keeps running and agreeing; its signature just does not count there (member: false). Each chain is followed on its own. Only who signs can change this way: the number of nodes is fixed for the life of a network.
Replacing a node
Why the order matters. A node must never sign with a consensus key some of its peers do not accept yet: those peers and the rest would count different votes, and agreement can stall until the configurations are in step again. An earlier version of the test, which started the new node before its peers knew its key, lost one price in eleven that way. Accepting both keys for a while (step 2) avoids it; while both are accepted, whoever holds either key speaks as that node, so step 5 must not be skipped. On the vault side, five old shares still open everything, refresh or not; a refresh only makes old shares useless together with new ones. The protection rests on step 8: honest nodes really erasing their old shares.
8. Limits of what exists
- The cryptography written here is new and unreviewed: TDH2 in
crates/vault, and the ceremony, refresh and repair logic incrates/keygen(around FROST's own functions, which are used unchanged). All of it needs an external audit before it protects anything of value. - Nothing is durable across a restart except a node's keys and configuration; a restarted node must not rejoin a running consensus instance.
- Confidential compute: one enclave, fronted by one node, with no pool and no reassignment if that node is down. The trusted images come from each node's configuration;
EnclaveRegistryholds the same list on chain but nothing reads it yet. Decrypted secrets are not yet erased from the WASM runtime's memory after use. cxyz-nodegenerates the vault key with the other nodes and runs confidential handlers against an enclave, but does not run a refresh or a repair of the key; those are driven from a program of one's own, as in the tests. The network shown on the Live page is seven node processes and one enclave on one machine, run by one operator: it shows the protocol working, not the independence the fault tolerance assumes.- A consensus instance that cannot decide keeps trying; nodes that disagree about each other's keys can stall agreement.
- The number of nodes is fixed for the life of a network, and the node set and threshold for the life of a vault key.
- This page: it holds no keys and changes nothing. It shows what the nodes' files say and what a quorum of RPC endpoints say; it verifies no signature and no attestation itself.