Simulation Framework
Full-Run Simulation Coverage
Gantry gained scenario data for stopped runs, resume incidents, and merge-family outcomes.
4 milestones 0 tasks
5 plan 18 execute 32 review
5h 47m total
This run made Gantry's end-to-end behavior testable as scenario data. The cut began with stop identity, then used the real engine, real git, and the existing harness parser to run scripted scenarios against the same surfaces a live run uses.
The finished work is still visible as a compact contract rather than as a large feature branch. Stop surfaces expose typed identity, simulation tests drive full runs through `SimulationRunner`, the heavy tier records where the expensive scenarios live, and the blind-spot report names the stop identities the corpus covers or leaves outside the corpus.
How this walkthrough is structured
Feature
What did the run build and ship?
The current tree still carries typed stop identity, `SimulationRunner`, stop-corpus scenarios, merge and resume scenarios, and the stop identity blind-spot report.
Build
How did Gantry structure the work?
The cut put stop identity before simulation, then let corpus work stress the framework at the boundaries where scenario data had to prove stopped-run evidence and resume behavior.
Milestone 1
Stop Surface
`StopIdentityPayload` still carries route, class, operator kind, and merge kind. The live tree also keeps operator pause recognition tied to the typed operator identity, so an old boolean is no longer the only way that surface is detected. The reference vocabulary and public consumers continue to receive the same closed stop identity values. This milestone's change is narrow in git, but it is the contract the scenario corpus asserts.
This milestone was asked to make stopped runs assertable without matching authored messages. The boundary was small enough for a fresh agent because it focused on the stop payload already carried through events and public consumers. It stayed green, which gave the later simulation work a typed surface to inspect instead of a prose string to parse. The run facts show no retry here, so the stress had not yet moved from identity shape into scenario behavior.
Milestone 2
Simulation Runner
`SimulationRunner` is still the home for full-run scenario tests. Scenarios can script agent stages, write declared outputs, apply fixture diffs, run git, simulate harness failures, assert stops, inspect files, and drive rerun or resume follow-up acts. The root test policy still names the minimal green simulation as the default-gate representative and moves only the costliest scenarios into the heavy tier. The old smoke test role has been folded into that runner contract.
This milestone carried the reusable test framework after the stop surface existed. The brief kept the model data-shaped: a mock plan, pattern invocation, scripted agent behavior, fixtures, settings, and expected outcome. It stayed green, which fits a framework slice whose job was to give later workers a runner rather than a finished corpus. The runner could be held by one agent because it reused the production engine, git, and harness parser instead of creating a second interpreter.
Milestone 3
Stop Corpus
The simulation corpus now includes build-pattern recovery, handover, absent output, invalid output, empty campaign, agent failure, and callee stop scenarios. The current code keeps stop evidence derivation working for stops that do not carry their own facts, while output-binding stops can still provide explicit facts. Scenario assertions read recorded stop evidence for present files, absent declared outputs, engine facts, and pattern notes. Some broad source-plan ambitions are spread through later test additions, but the stop-corpus contribution remains visible in the runner tests.
This milestone took the runner into the build-pattern stop cases, where the first real stress appeared. The ledger records a red run-scoped gate at the milestone boundary and a clean retry. The repair note ties the failure to stop evidence derivation and lint failures, not to a different decomposition. That is the useful boundary evidence: the cut was right to isolate non-merge stops before merge and resume, because this is where generated stop evidence had to be corrected.
Milestone 4
Merge Resume Corpus
The current corpus covers textual merge conflict resume, semantic merge-gate give-up, dirty home checkout, blocked executor resume, stale terminal stop replay, raw output replay, and foreign stash handling. `docs/analysis/scenario-coverage-inventory.md` names covered, partial, and covered-outside-corpus rows for the incident set. `scenario_coverage_report.rs` renders uncovered stop pairs and merge sub-kinds from that inventory and the live vocabulary, while preserving partial coverage as evidence. The report is test-only, so it documents blind spots without becoming a release build input.
This milestone added the merge-family and resume incident cases after the runner and non-merge stop corpus had already exposed the evidence path. It also hit a red run-scoped gate and then completed on retry, which places the hard edge at the widest scenario matrix rather than at the framework itself. The brief paired provable incidents with a blind-spot report, so a fresh agent could add honest scenarios while listing failures the mock agent should not fake. The merge stage later failed once before the run landed, which kept final integration under the same proof boundary as the scenarios.