The Other Half of the Corpus
The last post closed on a score: 81% of Glean’s benchmark, fully offline, and a question about whether that was good enough to skip enterprise search. I never answered it, because something quieter overtook it. Coming back to the corpus after a stretch of other work, I found the notes had kept growing while the index had not, and nothing in gapvec had noticed the gap. Its answers arrived as confident as ever, built from a version of the record that no longer existed. I retired it, along with the graph-based context assembler I had built beside it, and started asking what else I had been assuming instead of watching. The largest assumption turned out to be a directory I had never opened: thousands of session logs, filed as exhaust. Could the half of my history I never read answer questions the half I curated could not?
The Benchmark Measured My Imagination
A benchmark assembled from invented questions measures a system against your imagination of your own work, and every question in that project was invented. I wrote fifteen of them across five categories (factual, people, temporal, synthesis, needle) to probe failure modes I already suspected were there. It worked on its own terms: the harness found temporal blindness and needle blindness, and three targeted fixes closed 40% of the gap.
What it could not find was anything I had not thought to look for. I was guessing at my query distribution the entire time, and the real one was accumulating on disk the whole while, in the session logs my harness writes as a side effect of every conversation. Every question I had actually asked, in the words I actually used, with whatever I did next visible when the answer came back wrong.
I had also written them with an interest in how the answer came out.
Two Layers
Your own history is a corpus in two layers, and I had been treating only one of them as a corpus at all.
Artifacts are what you produce: notes, research, decisions, the multi-year meeting record from the last post. Everyone recognizes this layer as a knowledge base, because it looks like one.
Transcripts are the record of how you worked: agent session logs, thousands of them, written automatically and, in my case, never read. I treated this layer as exhaust, a retention setting configured once and a directory never opened, and I suspect that is the common case.
The same three moves apply to both layers: index it, make it queryable, and model its decay.
| Applied to | You get |
|---|---|
| Artifacts | A knowledge tool tuned to what you know, staleness aware, flagging what to retire as facts age |
| Transcripts | Your AI use tuned to you: real query distribution, recurring asks, repeated corrections, unanswered asks |
One history, two layers. Index, query and decay applied to the curated artifacts give a staleness-aware knowledge index that flags aged facts to retire; applied to the thousands of session logs I had never read, they give varve.
What surprised me is how much of the second row I could act on the same week. A real query distribution is what you tune retrieval against. Recurring asks are candidates to cache or templatize. Repeated corrections are preferences I had been restating by hand and could encode once. Unanswered asks are knowledge gaps, already phrased in my own words, without running a survey.
The reason this seems worth writing down: anyone running an agent harness is accumulating the same corpus, and I suspect most of them file it under logs the way I did. I do not know of a richer record of how a person actually works with these tools, though I have exactly one data point, so treat that as a hypothesis too. What I can say is that the corpus is local, and everything below assumes it stays that way.
Counts Before Content Was the Wrong Bet
I built the transcript side first, and I built it for the wrong question.
The initial framing was cost and behavior: token burn by project, spend over time, where corrections cluster. That shape of question needs counts and metadata, so counts and metadata is what the store kept. It answered its questions well. Then I asked it what my workload actually consisted of, and it had nothing useful to say.
Three reasons, which I think generalize past my particular mistake:
Optimizing the canonical store for shareability strips the text you need. I had reasoned that a store holding no content is a store I never have to worry about. That is true, and it is also a store that cannot answer a single question about what I do. Content has to be a first-class local asset. Confidentiality comes from keeping it local, gitignored, and snapshotted off-repo under encryption, with a redacted re-cut when something needs publishing. It does not come from never keeping the text in the first place.
Path-based attribution is content-blind. Grouping sessions by which directory they ran in tells you where work happened, not what it was about. Two sessions in the same repository can be a customer question and a refactor. Attribution has to be semantic (topic, entity, shape of ask) and derived from content, which means you need the content.
Counts-first hides the workload. “What do I ask, clustered by shape and topic, and how well is each cluster served” has to be a first-class view. As an afterthought bolted onto a cost schema, it stays unanswerable.
The store ended up as one lake with two projections, cost analytics and workload analytics. Getting the second required rebuilding the store around content; the first carried over unchanged.
The Instrument
The rebuild is called varve, after the annual sediment layers that make a lake bed readable as a chronology. It is a local record lake over agent sessions: Parquet as the canonical format, DuckDB as the engine, adapters per runtime that are close to pure SQL. No resident processes, no listeners, no egress. Ingest and backfill are the only writers, and analysis opens the lake read-only.
varve is one machine’s worth of plumbing. Near-pure-SQL adapters are the only writers into the Parquet lake, the DuckDB views over it are rebuildable because the engine is disposable, and analysis opens the lake read-only. Nothing listens and nothing leaves.
The design is plain on purpose: no server, no vector database in the first version, no daemon, just files and a binary. A full backfill of years of sessions takes a few seconds. Reconciling its cost numbers against an independent accounting tool agreed within 0.2%, which is most of why I trust the rest of what it reports.
The first question I put to the lake was whether a stricter context-recycle discipline in my harness would pay for itself. The answer was no, and not for the reason I expected: the strict discipline fights the shape of my actual work. My median session peaks well beyond either candidate threshold, so the recycling I was weighing would fire constantly against sessions that were always going to run long.
Repricing every historic call under both thresholds put the recoverable spend, an upper bound that assumes recycling costs nothing, at roughly double under the strict threshold as under the loose one. Scaled onto a round $10,000 of model spend, that is about $3,100 recoverable under strict and $1,450 under loose. I am deliberately not publishing my actual totals; the worked example and the chart carry the real proportions on a round hypothetical.
The same data pointed at a better lever, which is whether a long session dispatches its work or runs everything inline. I had been debating the wrong knob, and the record said so.
Repricing every historic call put the recoverable spend at roughly double under the strict discipline as under the loose one, an upper bound that assumes recycling is free. The scale underneath is why that number never lands: both candidate thresholds sit far short of where my sessions actually peak, so the strict discipline would fire constantly against sessions that were always going to run long. Real proportions on a round hypothetical; the actual totals stay unpublished.
Tuning the Harness
The recycle threshold is one knob, and most of the others are the same shape: a setting I chose once, on intuition, and never revisited against evidence.
The harness is a stack of those decisions. Which skills exist and what their descriptions say. Which model class each kind of task gets dispatched to. What runs inline versus in a subagent. Which commands sit on the permission allowlist. What gets written into a memory file instead of restated by hand. Every one of them leaves a trace in the transcript layer, so every one of them is a query rather than a preference.
What the record can settle:
- Skills that never fire. A skill is only reachable if its description matches how I actually phrase the request. Comparing invocations against the sessions where the skill should have applied names the descriptions that are wrong.
- Corrections I keep repeating. A preference restated by hand across twenty sessions belongs in a skill or a memory file. The transcript layer counts the restatements; until now I noticed them anecdotally, which means I noticed the annoying ones and not the frequent ones.
- Model tier against task shape. Dispatch rules assign a model class per kind of work. Whether the expensive class is drawing the hard work is answerable from the record.
- Permission friction. Repeated prompts on the same command class are a missing allowlist entry, and they are trivially countable.
- Dead tool surface. Tools and MCP servers wired up and never called still spend context on every request that carries their descriptions.
I have not run most of these. The reason I expect them to be worth running is the recycle threshold: that answer was not the one I would have given from memory, and I have no particular reason to think my intuitions about the other knobs are better calibrated than that one was.
The Artifact Half
The artifact layer did not go back into a retrieval tool. It went into a local index built around a maintenance organ instead of a search box, because the failure that killed gapvec was never a ranking failure.
That successor is not open source and will not be: it is welded to the corpus it indexes, which includes customer material. The architecture travels, and part three is where I write it down with the numbers attached.
What Comes Next
varve is public. Not the original repository, which grew against a lake holding customer material and carries history that no amount of tidying makes safe. varve is a fresh cut: the schema, the adapters, and the query library, and none of the operator data.
The part I am actually curious about is the mined query set. Every benchmark I own is still authored, not mined, including the 108 questions in part three: better sets than the original fifteen, and the same species of set. Mining the real distribution out of the transcript layer is the first thing I want from it, and the only way I can think of to settle whether tuning retrieval against questions I provably ask beats tuning it against good guesses. I expect it does. The first post is a standing reminder of what my expectations are worth before the harness has run, so I am going to go measure it.