Engineering vocabulary

The 181 names the engineering protocols use, each declared once in protocols/vocabulary.yaml with its category, its marking and its meaning. The vocabulary holds names and what they mean; what a claim, a piece of evidence or an obligation is, and how one is decided, belongs to Canon.

• core: not specific to Git, pull requests or code; usable by every engineering protocol.
• software.change: only makes sense for Git, pull requests or code.
• self.improvement: only makes sense in an agent runtime's self-improvement turn (self.improvement/1).
• loop.improvement: only makes sense in a turn that improves the improvement loop itself (loop.improvement/1).

Artifact kinds

Category artifact_kind.

Term	Marking	Meaning
intent	core	what the change or response is meant to achieve
system_specification	core	the specified behaviour of the system concerned
plan	core	the intended steps toward the intent
release	core	a versioned, deployable unit of the system
deployment	core	a release running in an environment
service	core	the running service a case concerns
implementation	software.change	a revision of the code that realizes the plan
experiment_design	core	the metrics, splits, seeds and success criterion an experiment is run under
dataset_snapshot	core	the frozen data an experiment's runs read
baseline	core	the reference model, method or revision a candidate is compared with
candidate	core	the model, method or revision under test
run_configuration	core	the settings and environment an experiment's runs used
results	core	the metrics an experiment's runs produced
eval_design	core	the evaluation design an automated loop is tuned under, its splits, metrics and guards
finding	self.improvement	a finding against the agent runtime, filed with a machine-checkable predicate and the record that produced it
promotion	self.improvement	the agent runtime's binary installed for runs, built from a named revision
loop_baseline	loop.improvement	the promoted loop definition a loop metric was measured on
loop_candidate	loop.improvement	a changed loop definition meant to raise a loop metric
loop_finding	loop.improvement	a finding against the improvement loop, naming the metric, its target and the turns it was computed from

Evidence kinds

Category evidence_kind.

Term	Marking	Meaning
operational_observation	core	an observation of the running system
objective_observation	core	an observation of whether the intent's objective is met
impact_assessment	core	an assessment of who and what an incident affects
cause_analysis	core	an analysis of why an incident happened
test_result	software.change	the result of running tests against an implementation revision
code_review	software.change	a review of an implementation revision
build_provenance	software.change	the record of how a release was built from an implementation
rollback_result	core	the result of returning to a previous release, complete or partial
design_review	core	a review of an experiment design, passed or failed
integrity_check	core	a check of the runs behind a set of results for integrity violations, clean or violated
reproduction	core	a rerun on the same dataset snapshot and run configuration, matched or diverged
comparison	core	a comparison of a candidate with the baseline, candidate_better or baseline_better
seed_sweep	core	a sweep of a candidate's effect over seeds or ablations, stable or unstable
regression_test_result	software.change	the result of running a bugfix's regression test against an implementation revision
implementation_change	software.change	the record that the implementation was changed, moving it to a new revision
security_review	software.change	a review of an implementation revision's effect on a security boundary
guard_check	core	a check of a candidate against the surface a loop may change and for leaked or forbidden content, outside, leaked or clean
tuning_comparison	core	a first and a second run of a candidate against the baseline on the tuning split, reproduced, not_reproduced or not_better
heldout_comparison	core	a run of a candidate against the baseline on the held-out split, held or regressed
change_record	software.change	whether merging an implementation revision into the base branch's tip would change that tip's tree outside .engineering/
ci_result	software.change	the result of continuous integration on an implementation revision
gate_result	software.change	the result of the repository's gate against an implementation revision
handover	software.change	the record that the harness chose to hand the change over at an implementation revision, changing no file
merge_observation	software.change	the observation that a revision, or the pull request whose head it is, was merged into the repository's mainline
pull_request	software.change	a pull request whose head is an implementation revision, or that revision with only the commits that record its approval above it
finding_record	self.improvement	the finding as filed, with a predicate that parses and a source that exists, or malformed
promotion_record	self.improvement	the installed binary's embedded revision compared with the merged candidate revision
scenario_result	self.improvement	the finding's predicate run on its scenario, or on its repository at its seed, with the runtime built from a revision
scorecard_result	self.improvement	every scenario of the corpus run at a candidate revision and compared with the baseline's record for the same scenario
definition_check	loop.improvement	a loop candidate's metric definitions compared with the baseline's; unchanged, dual_reported or redefined
loop_comparison	loop.improvement	a loop metric over turns under the candidate compared with the same metric under the baseline
loop_measurement	loop.improvement	the loop scorecard over at least three recorded self-improvement turns at a loop definition revision
tests_change	software.change	the record that a story's tests were written, moving the implementation to a new revision
paths_changed	software.change	which paths an implementation revision changes since the base branch's tip, test paths, implementation paths, both or neither

Claims

Category claim_id.

Term	Marking	Meaning
release.proven	core	the release comes from a verified implementation and carries current build provenance
deployment.healthy	core	the deployment is running healthily
objective.realized	core	the intent's objective is met
impact.bounded	core	the incident's impact is known and contained
service.healthy	core	the service is running healthily
cause.identified	core	the incident's cause is known
tests.pass	software.change	the tests pass for the current implementation revision
implementation.reviewed	software.change	the current implementation revision is reviewed
implementation.verified	software.change	the current implementation revision is verified
experiment.valid	core	the experiment design passed review and the results carry no integrity violation
result.reproducible	core	a reproduction of the current results matched them
candidate.beats_baseline	core	a comparison of the current candidate found it better than the baseline
effect.robust	core	a sweep of the current candidate found its effect stable
hypothesis.supported	core	the experiment is valid and reproducible, and the candidate beats the baseline robustly
regression.reproduced	software.change	a failing test has reproduced the regression a bugfix repairs
regression.shown_first	software.change	a bugfix's regression test failed on its first run, before the implementation was first changed
security.reviewed	software.change	the current implementation revision's security review is approved by another principal than the implementer
rollback.verified	core	the release's rollback is complete, by a rollback result from another principal than the one who rolled it back
change.within_surface	core	an independent guard check did not find the current candidate outside the surface the loop may change
content.clean	core	an independent guard check found the current candidate free of expected answers and new forbidden content
tuning.improved	core	an independent first run of the current candidate on the tuning split was better than the baseline
heldout.not_regressed	core	an independent run of the current candidate on the held-out split did not regress against the baseline
change.supported	core	every keep claim holds for the current candidate revision
change.handed_over	software.change	the harness handed the change over at the current implementation revision, saying it is complete
change.made	software.change	merging the current implementation revision into the base branch's tip would change its tree outside .engineering/
ci.green	software.change	continuous integration passes for the current implementation revision
gate.green	software.change	the repository's gate is green for the current implementation revision
pr.merged	software.change	the pull request for the current implementation revision is merged
pr.opened	software.change	a pull request is open for the current implementation revision
finding.cited	self.improvement	the finding is filed with a predicate that parses and a source that exists
fix.merged	self.improvement	the current candidate revision is merged into the mainline
scenario.passes_after	self.improvement	the finding's predicate fails on the current candidate revision, observed by another agent run than the one that delivered it
scenario.reproduced_before	self.improvement	the finding's predicate holds on the baseline revision
scorecard.no_regression	self.improvement	no scenario of the corpus is worse at the current candidate revision than at the baseline
loop.change_merged	loop.improvement	the current loop candidate is merged into the mainline
loop.definition_honest	loop.improvement	the current loop candidate leaves every metric definition unchanged, or reports the previous definition's figures beside the new ones
loop.metric_below_target	loop.improvement	a loop metric is below its target on the baseline loop definition
loop.metric_improved	loop.improvement	the metric improved under the current loop candidate, observed by another agent run than the one that delivered it
tests.shown_red	software.change	a story's tests were written, and they fail at the current implementation revision
change.complete	software.change	the current implementation revision changes both test paths and implementation paths since the base, with its tests and gate green at that same revision

Actions

Category action_id.

Term	Marking	Meaning
metrics.inspect	core	read the system's metrics
logs.search	core	search the system's logs
release.inspect	core	read what a release contains
release.rollback	core	return to a previous release
traffic.shift	core	move traffic between deployments
emergency.leave	core	leave emergency mode
repository.inspect	software.change	read the repository
repository.merge	software.change	merge a revision into the repository's mainline
tests.run	software.change	run the tests against an implementation revision
tests.write	software.change	change the tests in the repository's working revision
implementation.edit	software.change	change the implementation in the repository's working revision
design.review	core	review the evaluation design, under authority
candidate.evaluate	core	check and run the current candidate revision, producing its guard check and comparisons
change.keep	core	keep a candidate a loop produced, on every keep claim and authority to keep
change.reject	core	discard a candidate once every keep claim is decided and one is false
change.propose	core	hand a candidate that leaves the loop's surface to a person as a proposal, applying nothing
change.apply_outside_surface	core	apply a candidate that leaves the loop's surface, under a capability never granted to a loop
change.handover	software.change	hand the change over, changing no file, once it is complete and no further edit is needed
change.read	software.change	read whether merging the implementation revision into the base branch's tip would change its tree outside .engineering/
ci.read	software.change	read the continuous integration result for the pull request's head
gate.run	software.change	run the repository's gate against an implementation revision
pr.open	software.change	push the implementation revision and open its pull request through the run's forge
binary.promote	self.improvement	build the merged candidate, embed its revision and install it for runs
candidate.merge	self.improvement	merge a candidate that touches no judge surface
candidate.merge_judge_surface	self.improvement	merge a candidate that touches a judge surface, under the owner's capability or a revocable delegation of it
finding.file	self.improvement	file the finding in the planning store with its predicate and its source
scenario.reproduce	self.improvement	run the finding's predicate with the runtime built from the baseline
scenario.verify	self.improvement	run the finding's predicate with the runtime built from the candidate
scorecard.compare	self.improvement	run the scenario corpus at the merged candidate and compare it with the baseline
loop.compare	loop.improvement	compare the metric with the baseline's once at least three turns ran under the merged candidate
loop.measure	loop.improvement	compute the loop scorecard over the recorded self-improvement turns
loop.merge	loop.improvement	merge the loop candidate under the owner's capability or a revocable delegation of it

Obligations

Category obligation_id.

Term	Marking	Meaning
restore_service	core	bring the affected service back to health
investigate_cause	core	establish why the incident happened
tests.required	software.change	the tests must pass for the current implementation revision
review.required	software.change	the current implementation revision must be reviewed
provenance.required	software.change	the release must be proven, its build provenance recorded
health.required	core	the deployment must be observed healthy
red_first.required	software.change	a bugfix's regression test must fail on its first run, before the implementation is first changed
security_review.required	software.change	a change that affects a security boundary must be security reviewed by another principal than the implementer
ci.required	software.change	continuous integration must pass for the current implementation revision
gate.required	software.change	the repository's gate must be green for the current implementation revision
merge.required	software.change	the pull request for the current implementation revision must be merged
regression.required	self.improvement	the scorecard must show no regression at the candidate
reproduction.required	self.improvement	the finding must be reproduced on the baseline before a candidate counts
verification.required	self.improvement	the finding must be verified gone on the candidate by another run
honesty.required	loop.improvement	a loop change never redefines a metric silently
improvement.required	loop.improvement	a loop change counts only when its metric improved

Outcomes

Category outcome_id.

Term	Marking	Meaning
accepted	core	the case closed with its result accepted
declined	core	the case closed without carrying out its intent, on a decision not to or on evidence that its candidate did not achieve it
rolled_back	core	the case closed with its release returned to a previous one
superseded	core	the case closed on a decision that another case replaces it
abandoned	core	the case closed on a decision to stop before it reached any planned outcome
supported	core	the case closed with its hypothesis supported
refuted	core	the case closed with its hypothesis refuted by a valid, reproducible result
invalid_experiment	core	the case closed because its experiment is not valid
inconclusive	core	the case closed on a decision that its experiment settles nothing
kept	core	the case closed with its candidate kept
rejected	core	the case closed with its candidate rejected on a false keep claim
proposed	core	the case closed with its candidate proposed to a person, because it leaves the loop's surface
unreproducible	self.improvement	the turn closed because the finding's predicate does not hold on the baseline

Decisions

Category decision_name.

Term	Marking	Meaning
explicitly_declined	core	a recorded decision not to carry out the case's intent
explicitly_superseded	core	a recorded decision that another case replaces this one
explicitly_abandoned	core	a recorded decision to stop the case before it reaches any planned outcome
explicitly_inconclusive	core	a recorded decision that the case's evidence settles nothing

Case inputs

Category case_input.

Term	Marking	Meaning
risk	core	how much proof a case must carry, fixed when it opens; it selects the obligations that apply
change_kind	software.change	what a change to software is, fixed when its case opens; it selects whether a regression must be reproduced first
affects_security_boundary	core	whether a change affects a security boundary, fixed when its case opens; one that does needs an independent security review
surface	self.improvement	whether the candidate's change, base to head, touches a judge surface
tests_first	software.change	whether a story's tests must fail before its implementation is edited, fixed when its case opens
gate_on	software.change	when the repository's gate runs, fixed when a story's case opens

Input values

Category input_value.

Term	Marking	Meaning
trivial	core	the lowest risk; a change whose tests must pass and nothing more
standard	core	the usual risk; everything trivial requires, and review
elevated	core	a raised risk; everything standard requires, and a proven release
critical	core	the highest risk; everything elevated requires, and a healthy deployment
feature	software.change	a change that adds or alters behaviour
bugfix	software.change	a change that repairs a regression, which a failing test must reproduce before the implementation is edited
refactor	software.change	a change to the implementation that keeps its behaviour
true	core	the value of a yes-or-no case input when what it asks holds
false	core	the value of a yes-or-no case input when what it asks does not hold
judge	self.improvement	the candidate's change touches a judge surface
non_judge	self.improvement	the candidate's change touches no judge surface
required	software.change	the implementation is edited only while the story's tests, once written, fail at the current revision
optional	software.change	the implementation may be edited whatever the tests have done
after_handover	software.change	the gate runs once, on the revision whose tests pass and whose change is handed over
each_revision	software.change	the gate runs at every revision, whenever asked

Independence dimensions

Category independence_dimension.

Term	Marking	Meaning
different_principal	core	the producer of a record is another principal than the one who did the case's work
different_agent_run	core	the producer of a record is another agent run than the one that did the case's work