Run your own
Short version: one hub image, one config file for at least one worker, and this site ships inside the hub’s own build.
The hub
The hub is a single container image. It refuses to start unless every required environment variable is set — there’s no insecure default to fall back on. Names only below; your operator (or you, deploying it) supplies the values:
Required:
OIDC_ISSUER,OIDC_CLIENT_ID,OIDC_CLIENT_SECRET— the identity provider browser logins go through.PUBLIC_BASE_URL— this deployment’s own externally-reachable URL.SESSION_SECRET— signs the browser session cookie.REQUIRED_GROUP— the identity-provider group a login must carry to pass.INDEX_PATH— where the session/message index database lives.
Optional:
PORT,STATIC_DIR— network and asset-serving basics.DOCS_DIR— where this site’s built output lives on disk; unset serves no docs at all.DOCS_HOST— a second hostname that serves this site at its own root, IN ADDITION TO (not instead of)/docson the app host, which keeps working exactly as before whether or not this is set (see Start here).AGENT_OIDC_RESOURCE— the resource/audience value for OIDC-authenticated agent tokens, reusing the same issuer as browser login with a distinct resource indicator (see Authentication). The old sharedAGENT_TOKENis retired; if it is still set the hub ignores it and logs a warning.ROOT_BINDINGS_JSON— optional JSON object{"<root id>": "<worker sub>"}pinning a root to the worker allowed to serve it. It overrides what the hub learned on its own (first-come and presence bindings), and each change is audited. Operator bindings are the exception: a binding made withPUT /api/remote/v1/workers/root-bindings/{root}(sourceadmin) is never rebound, unbound or promoted byROOT_BINDINGS_JSON, anullvalue included. A root the object names, even withnull, is pinned, and that PUT answers409for it; remove the key and restart to make the root rebindable there.ROOT_BINDING_FIRST_COME(defaulttrue) lets the first worker to touch an unbound root own it; set it tofalseso onlyROOT_BINDINGS_JSONbinds roots.GET /api/rootsshows each root’sowner_principal. Turns that name no root go only to a worker listed inUNROOTED_POOL_PRINCIPALSor one that owns a config binding or a binding bootstrapped from presence; a first-come or operator binding never grants them. Each key must be a valid root id: 1-128 characters fromA-Z a-z 0-9 . _ : -. The hub refuses to start on any other key and names it in the error. The object is the full set of config bindings: a root whose key is removed from it is unbound at the next restart (its row is deleted and audited asunbind_config_removedwith the previous owner), and an empty object unbinds every config binding. Anullvalue still unbinds that root whatever bound it, except an operator binding. A key that names a root already bound to the same worker (first-come or bootstrapped from presence) turns that binding into a config binding at the next restart (auditedpin_config), so removing the key later unbinds it. Only roots that are not pinned at all, whether bound first-come or bootstrapped, are outside the set and survive any edit of the object. Reservation roots (res-r, a 20-character reservation id, a dot and a harness, such asres-r<20 characters>.claude) are bound only by enrollment; a root that merely starts withres-(res-lab) is an ordinary root. The binding audit records every bind, rebind and unbind. Rejections (a worker naming a root bound to someone else, or any unbound root in strict mode) are coalesced to at most one row per worker and action per minute; the next row says how many similar rejections were suppressed and lists a few sample root ids. Rejection rows are pruned once they are older than 30 days or beyond the newest 100,000, at startup (after the listener is up) and every six hours. Bind and unbind rows are kept.RATE_LIMIT_REMOTE_SUBMIT,RATE_LIMIT_REMOTE_READ,RATE_LIMIT_AGENT_CLAIM,RATE_LIMIT_AGENT_EVENTS,RATE_LIMIT_AGENT_COMPLETE,RATE_LIMIT_AGENT_PUBLISH— per-principal token buckets written<per_minute>[/<burst>](defaults30/10,600/120,300/60,3000/600,600/120,600/120);0disables a class. The workers feature adds six more with the same syntax, listed under Reserved workers below.HUB_CLIENT_IP_SOURCE— which address a request is attributed to where a limit is keyed by caller address instead of by token subject (enrollment, static check-in, and the token endpoint when no client is named).socket(default) uses the TCP peer and ignores every header.fly-client-ipreads theFly-Client-IPheader that Fly’s edge sets; ifHUB_TRUSTED_PROXIESis also set, the header is believed only when the TCP peer is inside it.x-forwarded-fortakes the right-mostX-Forwarded-Forhop that is not a trusted proxy, and only when the TCP peer is itself a trusted proxy; it needsHUB_TRUSTED_PROXIES. Any other value stops the hub at startup. Set a header source only when a proxy you run overwrites that header, otherwise a client chooses its own bucket.deploy/fly/fly.tomlsetsfly-client-ip.HUB_TRUSTED_PROXIES— comma-separated IP addresses or CIDRs of the proxies allowed to speak for a client, for example10.0.0.0/8,fdaa::/16. A value that does not parse stops the hub at startup. Unset by default.TURNS_DB_PATH,TURN_REQUEUE_MAX_ATTEMPTS,PENDING_TURN_TTL_MS— turn queue storage and tuning.TIERS_CONFIG— path to a TOML file defining the model tiers (quick/standard/heavy) turn creation can name (see Integrate as a UI). Unset means the hub has no tier table at all — a NEW-session or fork request naming a tier is rejected outright,400"this hub has no tier table configured (TIERS_CONFIG unset)". An EXISTING session’s follow-up naming a tier is NOT rejected the same way: continuity already pins its provider, so the hub just narrows to that provider’s own candidates (an empty list on the hub’s side, since there’s no tier table to narrow from) and forwards it to whichever worker claims the turn — the worker then falls back to its OWN locally configured tier table to pick a candidate, rather than being told “no” by the hub.VAPID_PUBLIC_KEY,VAPID_PRIVATE_KEY,VAPID_SUBJECT— Web Push notifications on turn completion, if you want them.
The review profile
/api/integrations/turns* (Integrate as a dispatcher)
has its own independent set of environment variables, all optional as a
group but interdependent once you set any one of them:
API_RESOURCE— the resource/audience value forreview:runOIDC tokens, distinct fromAGENT_OIDC_RESOURCEabove. Shared with the work profiles below — there’s no separate work-run audience.REVIEW_PROFILE_PROVIDER,REVIEW_PROFILE_ROOT_ID,REVIEW_PROFILE_WORKING_DIRECTORY— the fixed provider, root, and directory every queued review runs against.REVIEW_PROFILE_MODEL,REVIEW_PROFILE_EFFORT— optional overrides.REVIEW_PROFILE_PERMISSION_MODE— optional, defaults to (and today must be)"plan".
Setting none of the REVIEW_PROFILE_* variables leaves
/api/integrations/turns* entirely unconfigured — every route under it
answers a plain 404. Setting any one of them requires API_RESOURCE
and the three non-optional REVIEW_PROFILE_* variables above, or the hub
refuses to start.
The work profiles
/api/integrations/work-runs* (Integrate as a dispatcher)
is a separate, independently configured surface layered on top of
API_RESOURCE above (the SAME resource/audience review:run tokens use
— there is no distinct work-run audience):
WORK_PROFILES_JSON— a JSON object mapping each opaquetarget_refa caller may name to one work profile:{"provider", "root_id", "working_directory", "model"?, "effort"?, "permission_mode", "branch_prefixes"?, "base_ref"?, "checkout"?, "repo"?}. Must contain at least one profile if set at all (an empty object is refused, same as leaving it unset in effect, but explicitly). Field limits, all in bytes:target_ref(the JSON key) non-blank and at most 500;providernon-blank and at most 200;root_idnon-blank and at most 500;working_directorynon-blank and at most 4096, and must be an absolute path;permission_modenon-blank and at most 100;model, when given, non-blank and at most 500.permission_modemust be one of"acceptEdits","auto", or"bypassPermissions"— the unattended subset; unlike the review profile,"plan"is not accepted here.effort, when given, must be one of"low","medium","high","xhigh","max".providermust additionally name a provider capable of being driven end to end (not every registered provider qualifies).branch_prefixes(default["aya-task/"]) lists the prefixes a caller’s ownbranchmust start with;[]means callers never choose one. Each must make a safe branch name with something after it.base_ref, when given, is the base a run starts from when the request names none; same name rules as a branch.checkoutis"workdir"(the default: runs useworking_directory, as they always have) or"fresh", which gives every run its own new copy ofrepoand requires it.repo(only with"fresh", at most 4096 bytes) is a git URL (https,ssh,gitorfile, no password) or an absolute path to a local mirror on the helper’s machine. The hub stores the resulting{repo, branch, base_ref}with the run; helpers do not act on it yet (the claim does not carry it until the helper side is wired). Setting this leaves/api/integrations/work-runs*unconfigured when unset — every route under it answers a plain404— and requiresAPI_RESOURCEplus the fullWORK_REPORT_*group below when set, or the hub refuses to start.WORK_REPORT_URL,WORK_REPORT_RESOURCE,WORK_REPORT_CLIENT_ID,WORK_REPORT_CLIENT_SECRET— all four together, or none: this is the credential and endpoint the hub uses to deliver a completed work run’s result back to you on its own initiative, a separate OAuth2 client-credentials client the HUB presents (not something a caller authenticates with) — see Run work through the hub: work runs.WORK_REPORT_URLmust be an absolute HTTPS URL (or an HTTP loopback URL, for local development only) with no embedded credentials, query, or fragment.WORK_REPORT_RESOURCEmust be an absolute HTTPS URI under the same restrictions — the resource/audience the hub requests for its own outbound token.
The remote API
/api/remote/v1/* (Remote API) lets a remote agent that
cannot join your network read sessions and drive chats on your machines
through the hub. It is off until REMOTE_API_RESOURCE is set; every path
under it answers a plain 404 until then.
REMOTE_API_RESOURCE— the resource/audience value for itschat:readandchat:drivetokens. Must differ fromAPI_RESOURCEandAGENT_OIDC_RESOURCE, or the hub refuses to start. In your identity provider, allow only the clients you mean to trust to request this resource and these scopes; that is the access control.REMOTE_ALLOWED_SUBJECTS— required, comma-separatedclient-…subjects. No other subject is admitted even with a valid token, and the hub refuses to start withREMOTE_API_RESOURCEset and this empty or missing.REMOTE_MACHINE_ALIASES_JSON— optional JSON object naming machines, keyed by the OIDC subject a machine’s worker authenticates as, for example{"client-abc":"Studio"}. Unnamed machines are listed by their subject.REMOTE_MAX_ACTIVE_TURNS— optional, default8: how many turns one caller may have waiting or running at once before the hub answers429.
A remote caller can ask a chat for read-only (the default) or
allow-edits and nothing stronger, however its token was issued.
Reserved workers
Workers lets a remote caller reserve a short-lived machine, which
enrolls with the hub and serves only that caller. It is off by default; with it
off every worker route answers 404 {"error":"workers unavailable"} and the hub
behaves as before. The reservation and admin routes sit under /api/remote/v1,
so the remote API above must be on too (REMOTE_API_RESOURCE and
REMOTE_ALLOWED_SUBJECTS), and workers authenticate on /api/agent/*, so
AGENT_OIDC_RESOURCE is needed as well. Every variable below is optional, and
a value that is set but invalid stops the hub at startup.
To turn it on:
WORKLOAD_ISSUER=on— the switch. The hub becomes a second OIDC issuer and signs worker tokens itself.PUBLIC_BASE_URL— becomes the issuer (iss) of those tokens, so it must be the URL workers and provisioners can reach. It must behttps; loopbackhttpis allowed for development.- Signing keys, one of two ways. Either set
WORKLOAD_ISSUER_KEKto the base64 of exactly 32 random bytes and the hub generates and stores its RSA signing keys sealed under it, or pointWORKLOAD_SIGNING_KEY_FILEat a PEM key (WORKLOAD_SIGNING_KEY_NEXT_FILEandWORKLOAD_SIGNING_KEY_PREVIOUS_FILEare optional and need the active file). Guard the KEK like any other secret; see KEK loss and rotation below. WORKER_POOLS_FILE— a JSON file with the pools, either an array of pool documents or{"pools":[...]}. Any error in it stops the hub at startup. The pool document (harnesses, who may reserve, limits, backend, attestation and token grants) is described indocs/api/workers.md. Admins can also create pools at runtime with aworkers:admintoken.
A minimal static pool, for one registered machine:
[{"name": "dev", "harnesses": ["claude"],
"access": {"principals": ["client-requester"]},
"backend": {"kind": "static"},
"attestation": {"kind": "static_psk"},
"grants": [{"audience": "$hub_agent",
"scopes": ["agent:claim", "agent:report", "execution:heartbeat"]}]}]
Other settings, all optional:
| Variable | Default | Meaning |
|---|---|---|
WORKLOAD_TOKEN_TTL_SECS | 600 | Lifetime of worker and service tokens, 60 to 3600. |
WORKLOAD_EXTERNAL_TOKEN_TTL_SECS | 300 | Lifetime of tokens for external audiences, 30 to 900. |
WORKLOAD_EXTERNAL_AUDIENCES | empty | Comma-separated https:// resources a pool grant may name beyond the hub’s own. |
RELAY_RESOURCE | none | The audience the $relay grant placeholder resolves to. |
WORKERS_PROVISION_RESOURCE | {issuer}/api/provision/v1 | The audience a provisioning service presents. |
WORKERS_WEBHOOK_SIGNING_KEY | none | A whsk_... key for the signed webhook to a provisioning service. Needed only for webhook pools. |
WORKERS_WEBHOOK_SIGNING_KEY_PREVIOUS | none | The previous webhook key, kept while receivers rotate. |
AGENT_ACCEPT_EXTERNAL_ISSUER | true | Set false so /api/agent/* accepts only hub-issued tokens. |
WORKLOAD_KEY_ROTATION_DAYS | 30 | Automatic rotation period for stored signing keys, 1 to 365. |
Key rotation: with stored keys the hub rotates automatically. A new key is
published in the JWKS two days before it becomes active, and the old one stays
published for seven days so tokens signed just before the switch still verify.
An admin can rotate on demand with POST /api/remote/v1/workers/issuer/keys/rotate
(it stages the next key), or with ?emergency=true to retire every older key and
activate a new one immediately. With file-backed keys the route answers 409;
replace the files instead.
Revocations reach identity-provider tokens too: with the issuer on, a token from
the external issuer on /api/agent/* is refused when its client is disabled or
its subject was revoked after the token was issued. A subject revocation is
pruned an hour after it takes effect (the hub’s own token lifetime cap), so
keep the identity provider’s access-token lifetime at one hour or less for the
clients that call the agent routes, or a long-lived token issued before a
deprovision would be accepted again once its revocation is pruned.
KEK loss and rotation
The KEK seals the stored signing keys at rest and nothing else can open them.
If WORKLOAD_ISSUER_KEK is lost or changed, the hub refuses to start with
the active signing key does not open under WORKLOAD_ISSUER_KEK (wrong KEK?).
Putting the original value back is the only way to keep the existing keys.
If it is truly gone, stop the hub, delete the rows of the
workload_signing_keys table in the database at TURNS_DB_PATH, and start it
again with a new KEK: the hub generates a fresh key set. Every token signed by
the old keys then fails verification, so workers and provisioners must mint
new ones, and any external verifier must refetch the JWKS.
To change the KEK while keeping the stored keys, re-seal them in place with
WORKLOAD_ISSUER_KEK_PREVIOUS:
- Set
WORKLOAD_ISSUER_KEK_PREVIOUSto the old KEK andWORKLOAD_ISSUER_KEKto the new one. - Restart once. Every stored key that opens only under the old KEK is sealed again under the new one.
- Remove
WORKLOAD_ISSUER_KEK_PREVIOUS.
WORKLOAD_ISSUER_KEK_PREVIOUS needs WORKLOAD_ISSUER_KEK and has the same
format. The KEK also keys the workers audit chain, so the hourly chain check
reports rows written under the old KEK as not verifying after a change; note
the head the hub logs before you rotate. Keep the KEK in your secret store,
never in the image or the repository.
Workers need an always-on hub
The issuer, the reconciler and the wake channels live in the hub process, so
a hub that is stopped, or scaled to zero when idle, stops reservations, wakes
and revocation. On Fly, with WORKLOAD_ISSUER on, set
auto_stop_machines = "off" and min_machines_running = 1; the shipped
deploy/fly/fly.toml keeps its defaults, so make that change in your own
deployment. The hub logs an error at startup when FLY_MACHINE_ID is set and
workers are on.
Rate-limit classes
Every class is written <per_minute>[/<burst>], with 0 disabling it, and
is keyed by the token subject unless the table says otherwise. The six classes
of the remote and agent surface are in the
list at the top of this page; these are the six the workers feature
adds, and the routes on each:
| Variable | Default | Routes |
|---|---|---|
RATE_LIMIT_WORKERS_RESERVE | 30/10 | POST /api/remote/v1/reservations, POST /api/remote/v1/sessions/{pk}/move |
RATE_LIMIT_WORKERS_ADMIN | 60/20 | every /api/remote/v1/pools*, /workers/* and /principals/* route |
RATE_LIMIT_TOKEN_ENDPOINT | 120/30 | POST /oauth/token, keyed by client id, or by caller address with no client |
RATE_LIMIT_ENROLL | 30/10 | POST /api/enroll/v1/enroll and /static/checkin, keyed by caller address |
RATE_LIMIT_PROVISION | 120/30 | /api/provision/v1/*, keyed by the provisioner’s subject |
RATE_LIMIT_AGENT_WAKE | 120/30 | GET /api/agent/wake |
Reading a reservation (GET /api/remote/v1/reservations, .../{id},
DELETE .../{id}) draws from RATE_LIMIT_REMOTE_READ. On the agent surface,
GET /api/agent/reservation and the worker heartbeat draw from
RATE_LIMIT_AGENT_CLAIM.
On the machine itself, the worker is aya-helper with an [enrollment] table
in its config, which is exclusive with the other root and hub-connection
settings; see docs/api/workers.md for the keys.
The preview relay
The preview relay serves dev servers that run on your machines at
https://<port>-<label>.<preview-domain>. It is a separate binary,
aya-relay, with a small set of hub settings. It is off until the hub has a
signing key; every /api/relay/* path, /r/s/* and the key document answer a
plain 404 until then.
What you need
- A preview domain that is not the hub’s registrable domain (the relay and
the hub refuse to start otherwise;
allow_same_site_previewandRELAY_ALLOW_SAME_SITE_PREVIEW=1override it and are unsafe) and not a subdomain of it, with a wildcard record (*.preview.example-preview.test) pointing at the relay. Ask for it to be added to the Public Suffix List, or dedicate one domain per organization, so sibling previews do not share cookies (docs/relay/threat-model.md). - A wildcard TLS certificate for
*.preview.example-preview.test. Either let Fly terminate it (fly certs add "*.preview.example-preview.test"prints the_acme-challengerecord to create), or let the relay get one itself withtls.mode = "acme_dns01"and a Cloudflare API token limited to DNS edit on that zone (RELAY_ACME_DNS_TOKEN). - A Valkey or Redis only if you run more than one relay replica
(
[directory] kind = "redis",RELAY_REDIS_URL). The URL must berediss://and carry a password or ACL user; the relay refuses to start otherwise. The shared ACME bundle is stored encrypted under a key derived fromRELAY_PEER_SECRET. Replicas dial only addresses insidelisten.internal_cidrs(private IPv4 and Fly 6PN by default).
In your identity provider
Create an API resource for the relay and one for the hub’s relay endpoints, and four scopes:
| Scope | Resource | Granted to |
|---|---|---|
relay:connect | the relay (https://relay.example.test) | each worker’s client |
relay:sync | the hub’s relay endpoints (https://relay.example.test/hub) | the relay’s own client |
relay:audit | the same hub endpoints resource | the relay’s own client |
relay:open | REMOTE_API_RESOURCE | scripted callers of the bearer ticket route |
Hub settings
RELAY_TICKET_SIGNING_KEY— an Ed25519 private key in PKCS#8 PEM (generate withopenssl genpkey -algorithm ed25519). Setting it turns the relay on; the three variables below are then required, or the hub refuses to start.RELAY_PREVIEW_DOMAIN— the preview domain, for examplepreview.example-preview.test.RELAY_PUBLIC_URL— the relay’s public URL, for examplehttps://relay.example.test.RELAY_HUB_RESOURCE— the resource/audience of the relay’s calls to the hub (therelay:syncandrelay:audittokens). Must differ from the other resources.RELAY_TICKET_SIGNING_KEY_PREVIOUS— optional: the key being rotated out. The key document lists both, so tickets signed with it keep verifying.RELAY_SHARE_MAX_TTL— optional, default24h(also3600,90m,2d): the longest a share link may live.RELAY_ANONYMOUS_SHARES— optional, default off: set to1to allowanyoneshares that need no hub login.RELAY_API_ALLOWED_SUBJECTS— optional, comma-separatedclient-…subjects allowed to mint tickets onPOST /api/remote/v1/relay/tickets. Unset, that route admits no one. It also needsREMOTE_API_RESOURCE.
The ticket key is the one secret that can forge a preview session; keep it out of the image and the repository like every other secret here.
Run the relay
deploy/relay/ holds a Dockerfile, a fly.toml, a commented
relay.example.toml and a compose.yaml that runs the relay with a
password-protected, TLS-only Valkey. The
relay reads its TOML from RELAY_CONFIG and its secrets from
RELAY_COOKIE_KEY, RELAY_PEER_SECRET, RELAY_HUB_CLIENT_SECRET,
RELAY_ACME_DNS_TOKEN and RELAY_REDIS_URL. On Fly, keep tls.mode = "off"
and expose the replica forwarding port only on the private network. With
several replicas, set the cookie key and peer secret to the same value on every
one.
Point workers at it with the helper’s [relay] section
(Preview relay).
The worker
aya-helper loads one TOML file and refuses to start without it — no
settings UI to fall back on. The smallest useful config points it at a
single execution source rather than a full local index. Two DIFFERENT
tokens are required here, not one: a top-level token_env/token_file
for the worker’s OWN local HTTP server (it refuses to start with no
resolvable token at all, even bound to loopback-only — this is unrelated
to talking to any execution source), and each [[execution_sources]]
entry’s own token_file/token_env for authenticating outbound to that
source:
token_file = "/run/secrets/worker-local-token"
[[execution_sources]]
id = "primary"
base_url = "https://hub.example.test"
token_file = "/run/secrets/agent-token"
data_dir is optional and omitted here on purpose: leave it unset and the
worker stores its data in a sibling data/ directory next to the config
file itself (e.g. ~/.config/aya-helper/config.toml gets
~/.config/aya-helper/data/) — set it explicitly only when you want that
data somewhere else entirely.
This is a plain source: the worker executes whatever provider, model,
and root each claimed turn’s own body specifies, exactly as
Integrate as a worker describes. Add an
[execution_sources.worker_profile] block only for a worker-routed
source — one that supplies prepared text and an opaque affinity instead of
a provider/root choice — and only once you’re ready for the worker to
enforce (and reject a claim that violates) the guarantee that such a source
never sets provider, model, root_id, or any other local-policy field
itself (see Integrate as a worker).