Metaharness
Browse documentation
On this page

CLI reference

metaharness <VERB> [OPTIONS]

Eight verbs, the subcommands of Cli in crates/metaharness-cli/src/lib.rs; tests/anti_drift.rs and tests/website_claims.rs fail when this table and the binary disagree. capabilities, conformance, doctor and project need no model, no network and no credential.

VerbWhat it does
aep drive <COMMAND>AEP-governed work through a concrete harness: run, resume, status and eval. run, resume and a live eval run reach a model; see the quickstart.
run <KIND>Run a harness session: events as JSON lines on stdout, commands as JSON lines on stdin.
capabilities <KIND>What an adapter says it can do: declared tiers, pinned versions, operation rendering.
conformance <KIND>The free conformance vectors: no model, no network, no credential.
doctor <KIND>The installed vendor version against the adapter's pin.
project <EVENTS>...Project an event stream into trace-ir/1, byte-stable, or render one or two runs as one static page (--html); --from <FORMAT> reads a vendor record through its adapter's production reader instead (design amendment a28); --to trace-ir-summary writes the document's summary instead (a42).
audit --transcript <FILE>Judge a transcript offline. Refuses with exit 2.
mcp-serveServe the owned tool surface over MCP on stdio. Not typed by an operator: its client is the harness, which starts it from the --mcp-config that run --tool-surface owned (or --catalogue) writes, naming this binary.

<KIND> is claude, codex or b10x. The first two are vendors metaharness drives; the third is the beyond10x loop, which it observes under observe and decides through the loop's caller seam under frame and ask — see the b10x adapter.

metaharness run

The run itself

FlagDefaultMeaning
<KIND>—Which harness.
-p, --prompt <TEXT>—The prompt to start with.
--model <NAME>—The model to ask for. Passed through to the vendor, which resolves it.
--max-turns <N>—A ceiling on turns.
--cwd <DIR>scratchA named working directory for the child, which it can write to. Alone, the operator's real tree: trades two hermetic rows for real work — see below.
--scratch-cwdoffDeclares the --cwd directory as the embedder's disposable scratch copy. H7 is attested declared, H11 is attested from the copy as over a scratch cwd, and --hermetic strict passes on the pair when the record's cwd is the copy; on b10x that is the directory the loop reports, and where it reports none (--substrate <socket>) H7 is unk and it exits 3 — see below. Refused without --cwd.
--scratch-name <NAME>noneNames the run's scratch root, <temp dir>/metaharness-<NAME>, so every run of one case gets the same child paths (TMPDIR, CLAUDE_CONFIG_DIR, CODEX_HOME, the b10x --context copies) and the provider's prompt cache holds across sessions. One plain path component of at most 64 bytes. The root is 0700 and is locked for the run. A leftover of the same user is replaced; a link or a foreign entry is refused. A second run under a held name is refused. The root is removed when the run ends. Without it, each run gets a new temporary directory.
--context <FILE>(none)A file the run is handed before it starts, named relative to the --cwd tree. Repeatable, every kind. See below.

--context: files the run is handed

Each file is opened beneath the --cwd tree without following a symbolic link, read once, and the same bytes are digested and handed over:

kindwhere the files gochannel in the record
claudethe value of --append-system-prompt; -p carries the task alonesystem-prompt
codexinline in the prompt, ahead of the taskuser-prompt
b10xmetaharness's copy of the bytes, through the loop's own --context; the loop wraps the file itself (<context kind="provided_context" trust="operator" …>, no preamble of ours) in its standing instructionsystem-prompt

On claude and codex the list is introduced as reference data, not instructions, and wrapped in <context-files-N>; each file sits between <context-file-N path="…" bytes="…" sha256="…"> and </context-file-N>; each N is the smallest number whose closing line no file (for the wrapper) or not this file (for the file) contains. On b10x the loop's own rendering applies, and it escapes < and & in the file's text, so the model reads the loop's escaped form of the digested bytes. session.started.context_files lists each file's path, sha256, bytes and channel, and the key is absent when no file was declared.

Claude Code is not given the files in its user turn because 2.1.304, 2.1.305 and 2.1.309 turn an @path there into a file read that no tool call, hook or record shows — see the claude adapter. The task text is still the user turn, and the same holds for an @path in it.

Refused before the spawn, naming the path: no --cwd; an absolute path or one with ..; a path that is or passes through a symbolic link; something other than a regular file; content that is not UTF-8 text; a file git's index does not name in the tree (a .git gitfile, as an embedder's scratch copy has, is followed; nothing is fetched to answer); more than 16 files; more than 65,536 bytes of content and rendered path together. The byte cap bounds what the files add, not the task: every argument the launch builds is then measured against Linux's limit on one argument (131,072 bytes with its terminating NUL, MAX_ARG_STRLEN), and one that does not fit is refused by name — the argument, its size and each context file with its size — before the spawn.

Hermeticity

FlagDefaultMeaning
--hermetic [off|on|strict]off; bare --hermetic means onSee the hermetic contract.
--credentials <SOURCE>operator-loginoperator-login, api-key, loopback, or none.
--api-key-secret <NAMESPACE/NAME>(none)b10x only, beside --credentials api-key: the key's name in the operator's secret store. A name and never a value; the loop reads the key itself. Refused before the spawn on claude and codex, beside any other --credentials source, beside a subscription token, and when the name is not <namespace>/<name>.
--plugin-dir <DIR>(none)Plugin directories to load, and only these. Repeatable. Copied into the scratch tree and digested before the child starts; on b10x, handed to the loop as its own --plugin-dir.
--strict-versionoffRefuse before the run when the installed vendor version is outside the adapter's pin, instead of warning.

--credentials none means the run is expected to fail at the first request — and that is sometimes exactly what a test wants.

Model, effort and actor

FlagMeaning
--model-endpoint <BASE_URL>A model gateway, as its root URL (no /v1); each harness reaches its own dialect under it.
--model-wire <WIRE>Which API the harness speaks under that root — only b10x has a choice: openai-responses or anthropic-messages.
--subscription-token-file <FILE>, --subscription-token-env <NAME>, --subscription-token-pointer <POINTER>A subscription token for a route that takes one instead of an API key; the pointer selects a field inside a JSON source. Not interchangeable with --credentials api-key.
--effort <LEVEL>Reasoning effort in the vendor's vocabulary, passed through and validated by the vendor.
--prompt-cache-ttl <TTL>The prompt-cache class a claude run writes with, 5m or 1h; without it the vendor chooses. codex and b10x refuse it. Each usage event's cache_creation_5m_input_tokens and cache_creation_1h_input_tokens say which class the writes took. Read from the 2.1.309 binary, not yet driven.
--actor <ACTOR>Who the run's store writes are made as, carried to the child as AEP_ACTOR — declared, never inherited.
--retain-dir <DIR>Copy the raw vendor wire out of the scratch root before it is deleted.

--credentials api-key is a declaration: without it, ANTHROPIC_API_KEY is not in the child environment at all. On b10x it reads OPENAI_API_KEY by name from the caller's environment and refuses the run when it is absent, unless --api-key-secret names the key instead. Then no key variable is read or forwarded: the loop opens the store over the session bus and reads the key itself, so only DBUS_SESSION_BUS_ADDRESS and XDG_RUNTIME_DIR are added to the child's environment, each when the caller has it. Whether the store opens under the hermetic floor has not been run, and is unverified (design amendment a50).

--credentials loopback puts metaharness in front of the vendor: the child holds a per-run placeholder and no credential file, and one custody attaches the real token on the way out. On Claude Code the child is pointed at the proxy by ANTHROPIC_BASE_URL; on codex by a [model_providers.metaharness_loopback] entry in the scratch CODEX_HOME. A ChatGPT-plan codex login is refused by name — see the codex adapter.

Control

FlagDefaultMeaning
--frame <FILE>—A sealed metaharness.frame/1 document. See frames.
--decisions <MODE>frameframe (the adapter decides from the admitted set) or ask (the embedder decides; one round trip per call).
--tool-surface <SURFACE>nativenative, or owned — which this build refuses on b10x, and on codex without --catalogue.
--catalogue <FILE>(none)A caller's tool-contract/1 document whose entries are the run's tool set. claude and codex; implies --tool-surface owned. See below.

The frame is a path, not a parsed value, on purpose: resolving it is the library's job, and parsing it in the binary would put protocol logic in the CLI.

--catalogue: the caller's tools instead of the vendor's

On claude the launch removes the built-in tools (--tools "") and names one MCP server, metaharness mcp-serve --catalogue <copy>, which lists each entry under its mcp rendering and nothing else, instead of the verbs the owned surface lists by default; Claude Code offers each as mcp__metaharness__<name>. A call to anything else, or to an entry this build does not perform, resolves to no operation, so a frame denies it as UNCOVERED_TOOL. The file is read and checked once, before the frame and before the spawn, and the server reads a scratch copy of the checked document. That copy is JSON with the characters YAML refuses raw or reads as line breaks written as \u escapes, and it is read back before the spawn. A leading byte-order mark in the file is dropped before it is parsed. A live launch with a catalogue has not been driven; the vector that pins it is synthetic (design amendment a40).

On codex the run starts codex app-server with no environment and the entries as client tools, and answers each item/tool/call itself through the same server and checks, with the same scratch copy (design amendment a66). One live run on 0.160.0 served a catalogue call to exit 0; the model was also offered code mode's exec and wait and request_user_input_async, which no config key removes (the Codex page).

A tools/call to an entry answers isError and performs nothing when its arguments are not an object, carry a key that is not one of the entry's input fields, or leave out (or give as null) a field the entry requires. The answer names the entry and the field. A field the entry does not require, given as null, reads as absent and is taken out before the call is performed. A field given a value that the performing b10x-harness entry does not declare, such as process.run's timeout_ms at 0.18.0, 0.19.0 and 0.20.0, is refused by the harness by name, and nothing is done (design amendment a50 § 2).

Refused before the spawn with exit 2, by name: catalogue-not-carried on b10x; catalogue-needs-owned-surface beside --tool-surface native; catalogue-with-allow-program beside --allow-program, because a server handed a catalogue publishes no run for the programs to be started by; catalogue-unreadable for a path that is not a regular file, cannot be read, is past 4 MiB or is not UTF-8; catalogue-not-tool-contract or catalogue-malformed for a file that is not a catalogue, and catalogue-malformed as well for one whose scratch copy would be past 4 MiB or would not read back as the same catalogue; catalogue-copy-differs when the copy written into the scratch tree reads back as anything but the checked catalogue; on codex, catalogue-name-reserved for an entry published as mcp or under mcp__, which the app-server would refuse at thread/start. Two entries published under one name are catalogue-malformed on every kind, when the file is read. mcp-serve --catalogue <FILE> serves the same catalogue on its own, and refuses --allow-program beside it with the same code.

catalogue-copy-differs is the one catalogue refusal that comes after something has started. It fires after the scratch files are written, after a loopback proxy has started when the run uses one, and under --strict-version after the vendor binary has been run with --version. Its message says which of these had happened. No vendor session is started.

Audit

FlagMeaning
--auditJudge the run: the built-in hermetic floor always, the external auditor when --spec is given.
--spec <FILE>The expectation document the external auditor is pointed at.
--auditor <PREFIX>The external auditor, as an argv prefix.
-- <AUDITOR_ARGS>…Arguments passed through to the auditor after everything metaharness adds.

run checks the pairing when it starts, before the scratch tree, any proxy or the harness process. Each case below exits 2 with no harness process started and no provider request made. The first case that matches is the one reported:

The run asked forThe refusal says
--spec and no --auditor, with or without an audit--spec was given without --auditor
--auditor and no --spec, with or without an audit--auditor was given without --spec
arguments after -- and no --auditor, with or without an auditauditor arguments after -- were given without --auditor
--spec and --auditor, with no --audit and --hermetic not strict--spec and --auditor were given on a run that wants no audit
an expectation document whose path is not UTF-8, or that is missing, is not a regular file, or cannot be openedthe expectation document <FILE> could not be read
an --auditor prefix that is empty or only spaces, or whose first word starts with -the auditor prefix is empty, or the auditor prefix starts with "<WORD>", which reads as an option
an --auditor program that cannot be started: a path that is not an existing regular file with an execute bit, or a bare name found as no such file on PATHthe auditor program "<WORD>", then why

--hermetic strict asks for an audit the way --audit does. A bare program name is looked for on the PATH run itself was started with, because that is the PATH the auditor inherits. Every check but the last runs again at audit time, after the session, because the document can change while the session runs. A program removed or changed during the session is still exit 2 after it.

--auditor takes an argv prefix, so a single-word program name is a degenerate prefix and a two-word subcommand is not a special case.

After decision census:, --audit prints what the machine would not admit, in one of three shapes because they are three facts: withheld: <tool> (<reason>); …, withheld: none declared, or withheld: not stated by the harness. The census cannot say it — a tool that was never admitted was never refused, so denied=0 reads the same on a run that got everything.

The b10x arm

These reach the beyond10x loop and are refused by name on claude and codex: a vendor arm's scope travels sealed in Frame.subjects, and a flag would be a second, unsealed copy of it.

FlagMeaning
--hooks <FILE>The operator's own programs, consulted by the loop at each call — its only content-level refusal. Named, never discovered.
--driver <PATH>A program the confined run must be able to start: staged, mounted read-only, added to the loop's allow-list. Not the same question as --allow-program, which says what a run may name.
--allow-program <PROGRAM>A program the loop's run tool may start. Repeatable; empty publishes no run.
--substrate <SOCKET>, --substrate-embeddedConfinement for the loop: a substrate daemon, or its driver held in-process. Never both.
--cgroup-root <DIR>A delegated cgroup subtree; without one no run entry is published.
--toolchain <NAME>A build toolchain admitted read-only inside the confined workspace.
--prices <FILE>A rate card, so the record states what the run cost; the loop has no vendor catalogue behind it.
--write-scope <GLOB=SCOPE><glob>=allowed|partial-only|denied, ordered, first match wins. The operation is refused, never the run.
--scope-announce stated|silentWhether the scope is also stated in the instruction; silent is an experiment control.

--context was on this list until design amendment a35; it reaches every kind now, see above.

What the launch emits, and how the seam reads the record back, is on the b10x adapter.

--cwd: the declaration that trades two rows for real work

The child runs in this tree, and can write to it. So:

  • H7 ("the working directory is ours") and H11 ("no memory file outside the copied tree") stop being impositions and are attested unavailable, naming this directory.
  • The vendor sandbox is widened to that tree so the work can actually happen — on codex the scratch config carries sandbox_mode = "workspace-write", which permits editing files in the cwd. The H7 attestation row says so in words: a reader sees that the run could write to the operator's tree without diffing a scratch config that no longer exists.
  • --hermetic strict therefore exits 1 on such a run: the floor reads both rows as gaps.
  • Plain --hermetic reports it honestly.
  • --add-dir stays denied either way, and no writable_roots widens the grant past this tree.

The embedder that wants a governed run over a real repository accepts exactly that trade. A scratch-cwd run is unchanged: the child writes nothing, anywhere.

--scratch-cwd: the embedder's disposable copy

An embedder that made its own scratch copy of the workspace, runs the child in it, and rechecks the copy's changes before anything reaches the repository is not pointing the run at the operator's tree, and --cwd <DIR> --scratch-cwd says so. The run is otherwise a --cwd run — same directory, same sandbox grant — and the attestation changes:

  • H7 is attested declared, a third state beside imposed and unavailable: the embedder declared the directory as its scratch copy, metaharness did not create it, --add-dir is never passed. The floor reads it ok only when the record's cwd is the planned one; the declaration never stands in for the record.
  • H11 is attested exactly as for a scratch cwd, with the ancestor walk run from the copy. On Claude Code it rests on --setting-sources "" and the fresh scratch config home, held to the memory canary of the record's version; the walk looks for every memory shape the pinned loader reads and names what it finds in the row, so a memory file above the copy is reported and --hermetic strict still passes (amendment a36). On Codex and b10x the walk looks for CLAUDE.md and AGENTS.md and decides the row: a file above the copy is a gap, and --hermetic strict exits 1 after the run. A memory file at the copy's root is the copy's own and is listed in the attestation's ambient_inputs.
  • --hermetic strict passes on a declared copy with nothing above it, when the record's cwd is the copy. On b10x the record's cwd is the directory the loop reports (since 0.15.1), compared as a directory and not as a spelling; where the loop reports none (under --substrate <socket>) H7 is unk and strict exits 3.
  • Without --cwd the flag is refused by name on every kind.

Design amendment a27. The declaration confines nothing; making the copy is the embedder's job.

metaharness capabilities

FlagMeaning
--renderPrint the neutral-operation → vendor-tool table instead of the whole descriptor.

A rendering that only exists inside a run cannot be asserted on before one — which is why it is published here.

metaharness audit and metaharness project

FlagVerbMeaning
<EVENTS>...projectThe event stream(s), positional: one, or two under --html.
--from <FORMAT>projectRead every record given as a vendor record of this format through the adapter's production reader: claude-code-stream-json, codex-rollout, harness-json or claude-code-session, a Claude Code session as stored on disk, read with the sub-agent files under <session>/subagents/ (design amendment a63). Any other name is refused by name; a record that is not the declared format is refused with exit 2 (design amendment a28).
--out <FILE>projectWrite the trace-ir/1 document, or under --to trace-ir-summary its summary, here instead of to stdout.
--html <FILE>projectRender one or two runs as a static two-column page at this path.
--events <FILE>auditThe event stream that went with the transcript.
--to <FORM>projectThe target form: trace-ir (default), the trace-ir/1 document; or trace-ir-summary, its trace-ir-summary/1, with or without --from (design amendment a42). Any other form is refused by name, and trace-ir-summary with --html or with other than exactly one event stream is refused, exit 2. The summary carries every model the document names (models) and says which nodes its token counts were read from (tokens_basis).
--transcript <FILE>auditThe raw vendor transcript to judge.
--spec <FILE>auditThe expectation document.
--auditor <PREFIX>auditThe external auditor, as an argv prefix.

audit currently refuses with exit 2, naming what it waits for. project writes the document (design amendment a15); under --from the document names the vendor format as its source and its digest is over the record's bytes (a28).

Exit codes

CodeMeaningAn audited run's last line
0It ran, and every gating verdict is ok.audit verdict: ok (exit 0)
1A gating verdict is gap, or the auditor exited 1, or a conformance vector failed.audit verdict: gap (exit 1)
2Metaharness could not do its job. Never a verdict about the run.None: a refusal prints no report.
3Nobody found out — a gating row is unk, or the auditor exited 3, or the harness died without a record. The harness itself may have exited 0: the report then says child exit: 0 beside control verdict: unk.audit verdict: no verdict (exit 3)

The meanings are the crate's own table (crates/metaharness-cli/src/lib.rs, § Exit codes, which condenses design § 9.4), and tests/website_claims.rs holds this table to it. Exit 2 covers a verb or option that is not built, and every other refusal before or around a run. 3 is not a softer gap.

An audited run (--audit, or --hermetic strict) ends its report on stderr with three lines, each a fact of its own (design amendment a48):

LineWhat it saysPrinted as
child exit:The harness process's own exit, as stream.closed records it in process. It decides no verdict.child exit: <N>, child exit: signal <N>, or child exit: unknown when nothing measured it, never 0
control verdict:The gating hermetic rows folded gap over unk over ok. Advisory rows and the external auditor do not enter it.control verdict: ok, control verdict: unk, control verdict: gap
audit verdict:The exit code above, as a word and a number.audit verdict: ok (exit 0), audit verdict: gap (exit 1), audit verdict: no verdict (exit 3)

A harness that exited 0 under an opening record silent on its plugins and MCP servers prints:

child exit: 0
control verdict: unk
audit verdict: no verdict (exit 3)

The harness finished, and the floor could not tell whether every control held. That is exit 3, and the three lines say which fact it rests on. A run that wants no audit prints no report, so it prints none of these lines.