From Run Logs to a Reusable Fleet Corpus
A collector that archives finished Gantry runs, and an extractor that turns them into normalized datasets.
Every Gantry run leaves durable files behind — logs, journals, records of what each agent spent. The analysis tools that studied those runs read the live run directories in place, so nothing was preserved once a directory moved or changed, and every analysis re-derived its own inputs. This job built a pipeline in two stages. First, a collector walks the finished runs, agrees with the existing analysis about which are complete and safe to keep, and writes a compressed archive of each into a separate private repository held apart from Gantry's own source. Second, an extractor reads those archives and produces normalized per-run and per-session records, plus aggregate statistics over the whole set. Splitting collection from extraction means the opinion-laden normalization step reruns from a stable archived copy rather than from live runs that may have moved on underneath it.
Build
The plan cut along the direction data flows — collect raw archives, then extract datasets from them alone — and made the shared eligibility check the first thing built; the stress landed on bootstrapping the standalone data repository, which the gate sent back before it settled.
The decomposition followed the direction data flows: one milestone to collect raw archives, a second to extract normalized datasets, and the extractor was required to read from archives only. That constraint let the whole extraction milestone be built and tested with no live runs in reach — its inputs were files a collector had already written. Inside collection, the opening sprint was the shared eligibility check: rather than write a second definition of which runs are safe to preserve, it reused the one the existing snapshot analysis already applied, so the collector and the analysis could never disagree about what counts. Grounding that first let every later collection sprint assume a settled answer to whether a given run should be kept at all.
The boundary that fought back was the repository bootstrap — first-run creation of the independent fleet-data repository, the ignore policy that stops the parent checkout from tracking it, and repeat invocation that must not destructively rewrite an existing store. The gate sent that sprint back twice and it went to investigation before it held. Standing up a nested repository, where first run and later reruns must each do exactly the right thing, is the part of the collector with the most distinct states to get right, and the failures locate the difficulty there rather than anywhere else in the milestone.
The other load-bearing event was a re-plan after the manifest sprint. Computing a content hash over each run's durable inputs, recording stable run identity, and deterministically superseding a run whose inputs later changed was reviewed, and what that review learned redrew the remaining collector-operations work before it was built. The extraction milestone, by contrast, stayed green end to end — a sign that decoupling it from live runs had removed its difficulty in advance rather than deferring it.
Feature
Gantry's fleet analysis once read live run directories with nothing kept between runs; this job left a durable private archive of finished runs and a rerunnable extractor that rebuilds normalized datasets from it, and both commands are still present at today's HEAD.
Before this job, the fleet analysis tools opened a run's orchestration directory in place. Nothing durable held a finished run once its directory moved or changed, and there was no reproducible normalized dataset to import — each analysis re-derived what it needed from whatever was still on disk.
The design that answers that is two commands. The collector initializes a standalone fleet-data repository, discovers finished runs from configured roots without mutating them, and writes one compressed raw archive per eligible run, recording each in a manifest keyed by a content hash — so an unchanged run collected again does no work, and a changed one is superseded predictably. The extractor consumes those archives on their own and emits stable per-job and per-session records as JSONL, alongside regenerated aggregate statistics, with the repository's README documenting the raw, normalized, aggregate, and masking tiers for a future importer. The eligibility check lives in its own importable module shared by both the collector and the older snapshot flow.
Looking at the tree today, both scripts are present and documented, and most of the lines the job introduced are still there. The corpus these very summaries are read from is the one this pipeline produces. The design has grown rather than been replaced: sibling modules for aggregate statistics and JSON handling, and further extractor and dashboard tests, now sit around the same two commands, while the brief lines the job first added to Gantry's top-level README were later rewritten away. The eligibility module, collector, and extractor remain the central pieces they were built to be.