Map: Fanning Gantry's Pipeline Over a List
A job type that runs one fresh worker per enumerated item — one at a time, or several at once.
Gantry normally takes one ambitious plan and builds it through a single pipeline. But some work is not one plan — it is the same task repeated over a list: run a check over every file, draft a summary for every item in a corpus, port every module in a directory. This job gave gantry a Map job type for exactly that. A small specification names a command that prints the work list, one row per item, along with templates for the worker's prompt, the pass/fail check, and the output path — each of which an item fills in with its own named fields. Gantry runs the command, treats each printed row as one unit of work, hands each unit to a fresh worker agent, and judges the result the same way for every unit. A unit that fails is recorded and the run keeps going; only a failure that looks systemic — the same trouble across the board rather than one bad item — stops everything. The list can be worked through one item at a time, or with several workers in flight at once, with an optional background reviewer that samples finished items as the run proceeds.
Build
The build was cut into a sequential core and a concurrency layer, so the genuinely hard part — running many workers at once without corrupting shared state — arrived as a delta over a pipeline already proven one unit at a time.
The first milestone built the whole feature for a single item at a time, and its briefs state why the pieces were cut where they were. Enumeration and field substitution were carved out as pure functions — no agent, no check, no side effects — precisely so the driver that followed could stay thin orchestration over primitives already tested on their own. That boundary is where the sequential core's difficulty concentrated. The enumerator-and-substitution sprint is the one that had a worker fail and get retried, after which a review redrew the remaining work before it landed. The deterministic heart of a Map run — turn a command's rows into units without reordering or reinterpreting them, and fill each unit's fields into the templates — was the piece that fought back, which is a fair place for the difficulty to sit.
The second milestone was shaped as a delta rather than a rewrite: the concurrent worker pool runs the exact same per-unit path the first milestone built, and the single-worker default was required to stay behaviourally identical to the sequential run. That framing let the pool be judged against an existing, trusted baseline instead of against a specification alone. It was also the sprint that came under the most stress. Its brief names shared-state correctness across in-flight units as the whole difficulty, and the gate found it: the pool's first attempt went red, took a repair, and a review re-planned the rest before it passed.
One cross-milestone consequence is worth naming because the run recorded it as it happened. The first milestone delivered the worker preamble through a single shared run-wide prompt file. A background reviewer running a different prompt at the same time as the workers cannot point at that one file without racing it — so the milestone's own closing note forced the next milestone to give the checkpoint agent its own prompt delivery. That is a boundary discovered under load in one milestone and paid down in the next, not a flaw in the earlier one.
Feature
Gantry's engine could build a single plan and nothing else; this job left a Map path that drives an enumerated list of items through the same checked pipeline, and it survives at today's HEAD, reorganized into a larger module.
Before this job, gantry could build one plan through one pipeline and no more. There was no way to point it at a list of independent items and have it run the same task, judged the same way, over each. To do that you would have scripted a loop outside gantry and given up the ledger, the per-item checking, and the safe halt that gantry gives a single build.
What the job left is a complete Map path. A map-spec parser reads the enumerate command, the prompt, check, and artifact templates, and an optional checkpoint section. An enumerator turns the command's output into ordered units and substitutes each unit's fields into those templates, giving every unit a disjoint output path so no two collide. A per-unit gate with skip-and-heal logic decides whether a unit needs building; the driver logs a failed unit and carries on past it; and a systemic-failure detector halts the run when the trouble is not confined to one item. Layered on top is a bounded worker pool driven by a jobs-width flag, a background QA agent that samples recent artifacts on a cadence, and live monitoring with a headless output stream for concurrent runs.
Standing in the tree today, the design holds, though it has been reorganized since it landed. The map specification, the per-unit model, the gate, and the money-free end-to-end demonstration test are all still present. The two files that no longer exist at HEAD were not removed as capabilities: the original single build-driver file has become a build-module directory whose fan-out driver now carries the concurrency and a checkpoint submodule, and the standalone checkpoint file folded into that same module. The worker-pool flag, the sampled checkpoint with its own separate prompt delivery, and the enumerate-driven fan-out are all live in the current code. Most of the lines the job introduced are still in place; the capability was absorbed into a larger build module rather than left as the loose set of files it first arrived as.