Metaharness
Browse documentation
On this page

The codex adapter

Pinned to 0.153.4 and 0.160.0. Adapter class: harness.

metaharness run codex --hermetic -p "…"
metaharness conformance codex     # 27 vectors, 4 of which run a real process and the real hook
metaharness doctor codex

What a run does

  1. Starts a real codex exec into a scratch CODEX_HOME.
  2. Copies the operator's auth.json in per spawn.
  3. Declares a blocking PreToolUse hook.
  4. Tails the session rollout for events — the record that carries timestamps, durations and per-turn usage, where codex exec --json stdout carries none.
  5. Retains those bytes for the auditor and answers the hook per call.

capabilities codex, conformance codex and doctor codex all work with no model and no credential.

--strict-version resolves codex on the PATH the child gets, reads its --version before anything is written for the child, and refuses one off the pin by name, exit 2 (design amendment a34). A run without the flag runs the binary. The rollout's opening record earns one version_outside_pin warning, which metaharness run prints once on stderr, and H9 reads gap from the version the rollout records. H9 is advisory, so the version alone never moves the exit code (amendment a62).

--context <file> puts each declared file inline in the prompt, ahead of the task, verbatim inside a tag whose closing line the file does not contain (design amendment a35; the checks are on the CLI reference). Inline because codex exec 0.153.4 does not expand an @path mention: a relayed probe on 2026-10-20 saw no mention grow the input or reach the model. That probe has not been repeated on 0.160.0; the files are inlined either way.

A caller's catalogue

metaharness run codex --catalogue session.yaml -p "…"

With --catalogue, the run starts codex app-server --strict-config --listen stdio:// instead of codex exec (design amendment a66). A run without it is unchanged.

  1. The thread starts with environments: [], so Codex has no shell, apply_patch or view_image. The catalogue's entries go in as client tools (dynamicTools), under their mcp renderings.
  2. The scratch config switches off every other tool a config key reaches: web search, goals, multi-agent tools, image generation, apps, plugins, tool suggestions, request_user_input, the sleep tool, and on 0.153.4 the orchestrator's skills tool. --strict-config refuses a key the binary does not know.
  3. Each item/tool/call becomes tool.requested on the owned_tool seam. metaharness decides it and answers it itself, through the server and argument checks the Claude route's mcp-serve uses. A denied call, or one outside the catalogue, is answered success: false and performs nothing.
  4. session.started.offered_tools lists the catalogue's entries, then exec, wait and request_user_input_async, which the record does not list and the launch declares (below).
  5. A catalogue entry published as mcp or under mcp__ is refused before the spawn (catalogue-name-reserved): the app-server would refuse it at thread/start. Two entries under one name are refused earlier, on every kind, when the catalogue is read (catalogue-malformed).

What the config cannot remove: a model whose entry in the vendor's model catalogue asks for code mode is offered exec and wait, and reaches the catalogue only from inside exec. A model that names request_user_input_async is offered it. A live run on 0.160.0 met both (docs/research/2026-10-29-codex-0.160.0-app-server-catalogue.md). Six live runs at 0.153.4 and 0.160.0 on a ChatGPT login offered exec, wait and request_user_input_async and nothing else besides the catalogue. In none of them did exec write a file, start a process or open a connection, though the prompt asked for each. request_user_input_async posts the question as an asynchronous message, answers the model {"accepted":true} and waits for nobody (docs/research/2026-10-29-codex-catalogue-live-follow-ups.md). The rollout records the exec cell as a call of its own, so the stream carries a tool.requested named exec under seam: none beside the catalogue call it made. The launch attestation lists these tools among its ambient inputs. They are vendor tools with no effect of their own: each catalogue call made from inside exec is still an item/tool/call that metaharness decides.

--tool-surface native beside --catalogue is refused (catalogue-needs-owned-surface), and --tool-surface owned without a catalogue is still refused on Codex.

The hermetic floor on Codex

--hermetic strict on a clean Codex run exits 3: nobody found out. The rows that read the opening record stand like this at 0.153.4 and at 0.160.0 (design amendment a34):

RowVerdictWhy
H1b, output styleok, when no config layer outside the scratch home existsThe scratch config.toml sets personality = "pragmatic", and the rollout's first turn_context records a personality. The row compares the two: another value is gap, none is unk. On 0.153.4 pragmatic is the binary's own default; 0.160.0 has no default, records the key as written, and sends the same instructions for pragmatic as for no key, so only none changes what the model is told there. The recorded value is the session's record of the setting, not proof of what the model was told — with a managed features.personality = false both releases still recorded "pragmatic" — so when /etc/codex/config.toml, /etc/codex/managed_config.toml, /etc/codex/requirements.toml or, under --cwd, a project .codex/config.toml exists, the row reads unk and names them, and H2 is attested unavailable.
H1a, pluginsunkThe rollout lists no plugins, and the scratch home is not shown to be the whole plugin surface: apps and remote_plugin are on by default.
H4, credential sourceunkThe rollout does not record which credential the session used.
H5, MCP serversunkThe rollout lists no MCP servers.

H1a, H4 and H5 are not read from what the launch imposed: that is metaharness's own claim, and a row is decided from the vendor's record. Each row prints its reason. Use --hermetic on for Codex trials.

The personality arrives one record after the opening one, so the adapter holds session.started until that record and completes it from it. The event order and the timestamps do not change. A hook call never waits for it: a hook request that arrives first releases the opening record as it stands, and H1b reads unk on that run.

Declared capabilities

TierStatusWhy
registrationunverifiedcodex exec takes no tool allowlist. sandbox_mode and approval_policy constrain the process, not the offered tool set. Declared rather than claimed.
calldeliveredDriven, live. See below.
turnunverifiedthread/inject is an app-server method; this adapter drives codex exec, which has no channel for text between turns.
killdeliveredBy terminating the child — metaharness's own act on a process it started, not a claim about a vendor surface.

Operation rendering

Deliberately sparse. An operation with no vendor tool publishes as None — a fact worth publishing rather than an omission.

Neutral operationVendor tool
shellBash
file.writeapply_patch
file.editapply_patch
everything else(none)

The apply_patch rendering comes from the vendor's documentation, not from a driven observation, and is labelled as such.

The live proof, and its exact limits

A policy that admitted no shell met a prompt that asked for one.

LayerWhat it said
The hook process{"tool_name":"Bash","tool_use_id":"exec-96257928-…"}
metaharnessanswered deny, with a reason
The vendor's session recordCommand blocked by PreToolUse hook: this step admits no shell, so the command did not run — with an empty Output:
The model's closing message"The command was blocked and did not run."

So tool.decide is Honoured and the call tier is Delivered. Design amendment a7.

Also not claimed: the turn tier, the registration tier, and the apply_patch rendering.

The vendor sandbox is what the cwd declaration decides

sandbox_mode in the scratch config.toml is not a constant — on this vendor it is the same setting as "may the child do the work it was pointed at".

Runsandbox_modeWhat the child may write
scratch cwd (the default)read-onlynothing, anywhere
--cwd <a real tree> (amendment a6)workspace-writefiles in that tree, and nothing outside it

The vendor's own description of the second value: "The sandbox permits reading files, and editing files in cwd and writable_roots. Editing files in other directories requires approval." The child is spawned in the named tree, so no writable_roots entry is written and the grant stops at the declared directory. --add-dir is still never passed, danger-full-access is never written, and no [sandbox_workspace_write] table is emitted — so nothing here claims or changes this vendor's network default.

The H7 attestation row states the grant in words, so "this run could write to the operator's tree" is readable off the run's own record, not only off a scratch config that is deleted when the run ends. --hermetic strict still refuses a named-cwd run: the grant changes what the child may do, never what the attestation claims.

The loopback provider on codex — the API-key half

--credentials loopback puts metaharness in front of the vendor. The child gets no auth.json at all: a [model_providers.metaharness_loopback] entry in the scratch CODEX_HOME points it at a per-run 127.0.0.1 port, env_key names METAHARNESS_LOOPBACK_KEY, and that variable holds a placeholder worth nothing anywhere else. One custody on metaharness's side of the socket attaches the real token on the way out.

Login in ~/.codex/auth.jsonWhat happens
OPENAI_API_KEY (an API key)Routed through the proxy. Free-proven end to end; that codex honours the provider entry is one paid turn, not yet spent.
tokens (a ChatGPT plan)Refused by name. Whether subscription traffic can be routed through a custom provider at all is unverified (V-LP6) — so it is stated, never degraded to the credential-copy path the loopback provider exists to replace.

Plugin injection — the placement is driven, the plugin list is not

codex exec has no --plugin-dir: codex plugin installs from marketplace snapshots, so there is no flag with which to hand the vendor a directory. So the adapter picks a location — $CODEX_HOME/plugins/<name>, chosen because this binary keeps plugins/cache and plugins/data under the Codex home and a marketplace's own entries are ./plugins/<plugin-name> — copies the declared directory there, and digests it before the copy.

The launch deliberately writes no [marketplaces] table to go with the copy. An unrecognised key under a table this binary reads is dropped without failing the config load, a malformed one can fail it outright — which here is a run with no seam — and the probe shows the copy alone is enough.

Three things that cost more to learn than the code using them

Each is a silent failure. This is why every claim above is read from the run's own record rather than from the file that configured it.

1. A hook is declared in config.toml, not hooks.json

A hooks.json is a plugin manifest's file. An unrecognised key under [hooks] is dropped without failing the config load — so a misconfigured seam and a run where nothing was attempted are the same observation.

2. A hook in a fresh CODEX_HOME never fires without --dangerously-bypass-hook-trust

A scratch home cannot hold persisted trust. The flag's warning is about running somebody else's hook unvetted; the only hook here is the one metaharness wrote a moment earlier.

3. The hook speaks Claude Code's tool vocabulary

tool_name is Bash — where the rollout calls the same call exec and the binary's own model-facing list calls it shell.

A rendering table built from the record would have denied every shell call and reported it as a frame decision.

A version split nobody was looking for

codex --version says 0.145.0. The session_meta.cli_version written by the run that same binary starts says 0.144.0.

The reader warns rather than widening the pin, and the split is tracked as open question Q18. On both pinned releases the two sources agree: the 0.153.4 and 0.160.0 rollouts each record the binary's own --version as cli_version.

The rollout format has no stability guarantee

It drifts within a single install. That is why the version gate is a warning and never a refusal mid-read — and why each pinned release is one the native fixture drove: a verdict that changed because the reader changed must be visible as such, not as a change in the agent's behaviour. 0.160.0 joined 0.153.4 only after both ran the same cases and their records were compared; one reader serves both because the newer rollouts add fields it does not map and move none it does.

Evidence labels

The evidence base is the research record docs/research/2026-08-21-codex-harness-research.md. Every claim there is labelled, and this crate inherits the label at the point of use:

LabelMeaning
VVerified locally against the binary and 2,437 rollout files.
DOfficial docs.
IInferred.
?Unknown.