Architecture

RaySpec turns one declarative spec into a running, tenant-isolated AI backend. This document explains how it is put together: the layered spine, the package taxonomy, how a request and a durable job flow through it, the three structural guarantees the design rests on, the security model, persistence, and how you extend it.

For the vocabulary (specs, agents, stores, routes, the run journal, tenancy) read concepts first; this document assumes it.

The layered spine#

RaySpec is a stack of layers, each depending only on the ones below it:

┌──────────────────────────────────────────────────────────────────┐
│  App        CLI (rayspec) · boot bin (rayspec-serve)           │
├──────────────────────────────────────────────────────────────────┤
│  Declarative engine   validate → diff → gate → deploy a spec     │
│                       (compose the running backend from YAML)    │
├──────────────────────────────────────────────────────────────────┤
│  HTTP API   Hono + zod-openapi · routes mounted on the auth chain│
├──────────────────────────────────────────────────────────────────┤
│  Agent core   neutral Backend interface + 4 adapters (in-process)│
├──────────────────────────────────────────────────────────────────┤
│  Accounts & auth   orgs · memberships · users · API keys · OIDC  │
├──────────────────────────────────────────────────────────────────┤
│  Data & journal   Postgres/Drizzle · tenant chokepoint · run log │
├──────────────────────────────────────────────────────────────────┤
│  Durable execution   off-request worker · schedules · replay     │
└──────────────────────────────────────────────────────────────────┘

The bottom layers (accounts, data, the tenant chokepoint, the run journal) are the platform's always-on foundation. The declarative engine sits on top and is what reads your spec and wires the product-specific routes, stores, and agents onto that foundation. The platform itself contains no product: everything product-specific arrives as the spec you inject at boot.

Package taxonomy#

The monorepo is organized into tiers under packages/. Each tier depends only downward.

The neutral core types are the fixed point of the whole system: they sit at the bottom, and the adapters above them absorb every difference between vendor SDKs so those types never have to change when an SDK does.

Data flow#

An HTTP request#

1. A request arrives at the Hono app. Shared middleware applies security headers and authenticates the caller — a Bearer JWT, an API key (same header), or a session — and resolves the active organization (the tenant).
2. The router matches a declared route and its action.
3. For a store action, the data layer runs the CRUD operation through the tenant-scoped database handle — the query is filtered by tenant before it ever reaches Postgres.
4. For an agent action, the run surface invokes the declared agent through the neutral backend. The agent's tool calls dispatch through a single boundary; the response streams back (or returns JSON), and every step is recorded in the run journal.
5. For a handler or stream action, control passes to the declared escape-hatch module through the same chokepoints, then the response is returned.

The tenant filter, the tool boundary, and the journal write are not per-route choices — they are structural, so no route can opt out of them.

A durable job#

1. A caller requests an agent run asynchronously, or a schedule fires a trigger.
2. The request returns immediately with a run id; the work is enqueued onto the durable worker.
3. The worker executes the run off-request. The run's start is recorded in the journal under a run-scoped idempotency key (the run id), and its steps, usage, and cost are written as it proceeds.
4. If the process restarts mid-run, the durable engine re-executes the run from the start rather than losing it — there is no intra-run checkpoint resume. A run that already completed is short-circuited from the journal; an in-flight run is guarded by a run-level single-flight keyed by the run id, so a recovery re-execution does not re-fire a non-idempotent side effect, and a run whose replay safety cannot be guaranteed is quarantined rather than blindly retried.
5. On completion the outputs are persisted and the run is marked terminal; usage and cost are already in the journal.

The three structural guarantees#

Three boundaries carry the weight of the whole design. Each is enforced by construction — you cannot write ordinary code that bypasses it.

1. The neutral backend boundary#

All four agent backends implement one neutral Backend interface, and everything above the adapters speaks only that interface. Each adapter is an anti-corruption layer: it translates the neutral request into its vendor SDK's shape and the vendor's response back into neutral types, absorbing asymmetries (error taxonomies, structured-output support, tool-call formats) internally. The rule is that the neutral types do not move when a vendor SDK churns — the churn is absorbed in the adapter. The parity suite holds every adapter to the identical neutral contract, so "write the agent once, run it on any backend" is a tested property, not an aspiration.

What the boundary unifies is semantics, not streaming granularity. The text_delta events of the neutral event stream are a per-backend property: the Pi adapter relays its SDK's token-incremental deltas, the Anthropic adapter emits one whole-message text_delta per assistant message, the Codex adapter emits at most one per run (the first completed agent message — a second is deliberately not re-emitted), and the OpenAI adapter drives the non-streaming run() overload and emits none at all. A client therefore treats the streamed delta count as a lower bound and never reconstructs the reply by accumulating deltas. The complete text lives in the run's result: returned directly on the JSON path, and surfaced by the conversation capability as the terminal reply event that closes its event stream. The neutral stream's own terminal event, run_completed, carries the run status and the aggregate usage — not text. The parity suite asserts the same neutral event vocabulary for every adapter; it deliberately does not equalize the event stream itself, because levelling down to the lowest common denominator would take token-level streaming away from the backends that have it.

One part of the contract is deliberately not uniform: when a backend decides how it authenticates. The platform resolves a run's auth mode once, before the run starts, and threads it onto the run context so every journaled step attributes to the same mode. A local adapter answers that from its own environment and needs nothing else. A remote backend cannot: at that moment there is no run identity to select a tenant- or run-scoped credential with, so it could only report a mode it has not yet bound. Such a backend may instead implement an optional context-aware preflight, which is handed the server-derived run identity — run id, tenant, the agent's neutral name, the model, and an opaque credential-binding reference the deployment supplies — and returns the mode it has actually bound. It replaces resolveAuth() for that one pre-run resolution rather than joining it, its answer is validated against the neutral auth-mode vocabulary, and a backend that does not implement it takes exactly the path it always did.

The validation is deliberately one-sided: a preflight's answer is checked, a resolveAuth() answer is still taken as given. That asymmetry is what keeps the older contract byte-identical — validating it now would newly refuse a backend that answers off-vocabulary at runtime, where that run completes today. The preflight is also unbounded: it holds the run until it returns, and on the durable path it does so inside the transaction the worker wraps the run in, so a hanging remote preflight pins a pooled connection for its duration.

2. The fail-closed tenant chokepoint#

Every query against tenant-owned data goes through a single tenant-scoped database handle that injects the tenant predicate. The set of tenant-scoped tables is deny-by-default: a table is reachable through the scoped handle only if it is registered as committed source, and the deploy step verifies this rather than registering it — a spec that declares a store which isn't registered refuses to deploy. The only tables exempt from the predicate are the genuinely global ones (identity, organizations, API keys, the audit log, and the auth provider's own storage), and each exemption is explicit and reviewed. The practical consequence: there is no ergonomic path to a cross-tenant read, because the unscoped handle is not the one application code is given.

3. The tool-dispatch trust boundary#

Agent tool calls run through one dispatch boundary, and everything that crosses it from the outside — tool outputs, transcribed or uploaded content, and rehydrated conversation history — is treated as data, never as instructions: untrusted content can inform a model's answer but cannot be allowed to redirect the agent's behavior or its tool use. The boundary is also where each tool's declared idempotency is honored on replay.

Be precise about which attacks that stops, because injection carried in a free-text field comes in three classes and the boundary reaches exactly one of them:

An imperative attack asks to be obeyed, so refusing to read it as an instruction is a complete answer to it. The other two ask for nothing. They only inform the answer — which is precisely what the sentence above permits — and the model then reasons from a planted fact or an invented rule and calls its tools entirely within the rules. Nothing at the dispatch boundary can intercept that, because nothing about the resulting call is out of order.

Closing those two classes is the author's responsibility, in the agent's instructions, and it takes two separate statements:

• Field precedence — which field wins when the free text contradicts a structured one ("if message contradicts headcount, headcount wins"). This is what answers the assertive class.
• A closed decision rule — that the rule as stated is the whole rule, and no further policy, exception, pre-approval or routing override exists. This is what answers the policy class.

Both are needed, and each answers only its own class. Measured on examples/lead-qualifier against gpt-4o-mini, three runs per cell, with a lead whose headcount makes smb the only correct verdict — attacks defended:

The middle row's policy cell is written as a range because that is what repeating it produced: independent three-run samples of the same configuration came back 0/3 and 1/3. That is the point of the row rather than a defect in it — a prompt-side defense fails probabilistically, so no single run count is a property of the configuration, and the regression named at the end of this section runs each class three times for the same reason. What reproduces is the shape: adding field precedence moves the assertive column and leaves the policy column near the floor.

And the part that does not transplant: the reliability of prompt-side injection defense is a function of how mechanically enumerable the decision rule is. Lookup table → works. Judgment call → partially. It is not a property you write into the instructions once and then have everywhere. You can only close a rule that exists — where the decision is a lookup table, "this table is complete" is a checkable statement, but where it is a judgment call an invented standing order violates no rule at all: it is one more factor to weigh, and it gets weighed. So an agent classifying against an explicit table can be closed in its prompt, and an agent asked to exercise judgment cannot be. That is a reason to express a decision as an enumerable rule wherever the domain allows it, and to treat a judgment-call agent's verdict as unbounded by the prompt.

rayspec doctor and rayspec plan report the agent_untrusted_field_precedence advisory for a document whose agent names an unconstrained text column without writing both statements above, and it names which one is missing — the two close different classes, so satisfying one is not satisfying the rule. It is a keyword heuristic over natural language — wrong in both directions by construction, and never fatal. Naming a column is all the document proves: whether the agent reads that column or writes it is decided in handler source the pass never opens, and a text column that declares an enum is excluded because its value cannot be prose. It is a reminder to make the decision, not a verdict that it was made. The shipped worked example is examples/lead-qualifier, and examples/lead-qualifier/injection-smoke.sh is the regression that drives all three classes against a live deployment.

Security model#

RaySpec's core is built for a trusted, self-hosted, single-node posture. It enforces a set of guarantees from the first boot, and it is explicit about a further hardening layer that it does not include.

Built in, from day one#

• Tenant isolation by construction — the fail-closed chokepoint above, with a continuous-integration test that fails the build if any tenant-owned table can be read without the predicate.
• No plaintext secrets — signing keys, peppers, and provider credentials live in the environment or a secret manager, never in the database or in git. The server refuses to boot if a required secret is missing (fail-closed).
• An untrusted-content trust boundary — the tool-dispatch boundary above. It is structural, so it holds against the imperative injection class; the assertive and policy classes are the author's job in the instructions, as that section spells out.
• An out-of-band audit trail — the append-only, tenant-scoped run journal records what ran, for whom, and under what authority, independently of the request path.
• Per-backend credential isolation — each agent backend uses its own operator-supplied credentials; the platform never proxies one party's credentials on behalf of another.

The separate hardening layer (not in the core)#

Running RaySpec for untrusted, multi-tenant, public-internet traffic requires protections that are deliberately out of scope for the core and belong to a distinct hardening layer:

• per-tenant data encryption with wrapped data-encryption keys,
• database row-level security as a second, in-database enforcement of tenancy,
• per-tenant execution sandboxing, and
• cryptographic binding of tokens to their client.

The core does not ship these, and it says so loudly at boot. Do not place a core deployment on a public address for untrusted traffic without that layer. The distinction is intentional: the core gives a self-hoster a correct, tenant-isolated backend for trusted use, and the hardening layer is what a public multi-tenant service additionally needs.

Restore and key rotation#

The boot secrets live in the environment, never in the database — which has a sharp operational consequence when you restore a database dump under different secrets. A full dump restores the rows whole — orgs, users, memberships, the argon2id password hashes, the API-key rows, and all tenant data — so at the DB level everything survives and stays reachable. The only thing a freshly-minted secret permanently breaks is the credential material keyed by that specific secret:

• The API-key pepper (RAYSPEC_API_KEY_PEPPER). Every API-key row stores an HMAC of the key computed with the pepper. Restore the dump under a freshly-minted pepper and those stored HMACs no longer match, so the copied API keys all fail to verify (401) — even though the rows are physically present. That is the only thing a new pepper breaks: the copied API keys, nothing else. The data and the org identities stay reachable — user passwords are hashed with argon2id (each hash carries its own salt and params; the pepper never touches passwords), so they survive the restore untouched. An org owner simply logs in again (password intact), gets a fresh JWT minted under the current signing key, and reaches the tenant data exactly as before. The fix is not to recover the old keys but to mint new API keys after a restore. The one genuine edge, noted honestly: an org whose sole credential was an API key (no user login at all) has nothing to log in with, so it needs a fresh key/identity established out of band.
• The JWT/OIDC signing key (RAYSPEC_JWT_SIGNING_KEY). The same class of problem: tokens issued under the old key fail to verify under a new one. This one self-heals — a user simply signs in again and gets a fresh token minted under the current key. (It is the same re-login that restores API access above, because user passwords are pepper-independent; only a copied API key, which cannot "log in again," must be re-minted rather than self-healing.)

The practical rule: keep a restored dump paired with the secrets it was created under (back up the environment/secret material alongside the database), or plan to re-mint the affected API keys after a cross-environment restore. This is stated for the trusted, single-node posture; it is not a claim that restoring a database into a public, multi-tenant deployment is safe — that requires the separate hardening layer above (see SECURITY.md).

Persistence and the run journal#

RaySpec's request-handling core is stateless; all durable state lives in Postgres via Drizzle. Each agent run follows a hydrate → run → persist cycle, and every step is recorded in the run journal — the append-only, tenant-scoped log that is the single source of truth for replay, cost accounting, and audit. Store schemas are generated from the spec with the tenancy and data-lifecycle columns injected automatically, and every migration is diffed against the current schema and passed through a safety gate (a destructive change is blocked unless explicitly allowed) before it is applied — including a from-clean-database check that the whole migration chain bootstraps an empty database correctly.

A booted deploy applies this generated schema in one direction only: it materializes a store on a clean database and mounts it when the live schema already matches. It is mount-only against an existing deployment — a live schema that has drifted from the spec fails the boot closed rather than being altered implicitly. Evolving an existing deployment's schema is a deliberate, reviewed step: author the forward delta and apply it with rayspec deploy --apply-migration <delta.sql> (which runs it through the same safety gate). See the CLI reference.

The extension model#

RaySpec is designed to be extended in a deliberate order, from least to most power:

1. Configuration first. The declarative spec is the primary surface. Most backends are fully expressed in stores, routes, agents, tools, and triggers — no code.
2. Escape-hatch handlers next. When logic genuinely doesn't fit the declarative surface, a handler points a route, tool, or trigger at a named export in a TypeScript module. Handlers load from a path-jailed root and dispatch through the same chokepoints declarative actions use — so custom code still can't escape tenancy or the trust boundary. Extension packs bundle handlers, stores, and tooling as a versioned, exactly-pinned unit.
3. The core last. Changing the platform itself is the last resort, reserved for genuinely new platform capabilities — not for product logic, which belongs in the spec or a handler.

This ordering keeps product concerns out of the platform and the platform reusable across every product built on it.

See also#

• Concepts — the definitions this document builds on.
• Getting started — run the stack and make a request.