Get Lamp
In The Merge Request as {Friction, Leverage} I spent a lot of time looking at how code review is straining under AI-generated volume and ended uncertain about what comes next. Since then I’ve been reading about a concept that keeps surfacing in the discourse around autonomous software production, and I wanted to trace what I’ve found and where it led us.
Dark software factories
In January 2026, Dan Shapiro published a five-level framework for AI-assisted software development, modeled on the NHTSA’s driving automation levels. The term “dark factory” comes from manufacturing. FANUC runs lights-out plants in Japan where robots build robots with no humans on the floor. No humans means no lights. Shapiro’s Level 5 applies the same idea to software: a black box that turns specifications into deployed code, with no human writing or reviewing anything.
StrongDM’s AI division is the most visible team operating this way. Simon Willison wrote about them in February. They operate on three rules: code must not be written by humans, code must not be reviewed by humans, and spend at least $1,000 in tokens per engineer per day. To make this work they built a Digital Twin Universe (behavioral clones of third-party services like Okta, Jira, and Slack) so agents can run thousands of test scenarios without touching production. They use holdout scenario validation, where BDD-style acceptance criteria are hidden from the coding agent, to prevent what the merge request piece described as the cycle of self-deception: the same model writing both code and tests can be wrong in the same way twice.
BCG Platinion framed this as industry transformation. Their line that sticks with me: “The defining shift is not the absence of humans; it is the relocation of human effort.” JJ Masse drew a useful distinction between dark factories and vibe coding: a factory is built with intent, every machine has a specification and analyzed failure modes. Vibe coding is the opposite. The gap between the two isn’t degree, it’s kind.
I find the dark factory idea credible for a real category of software. Internal tools, CRUD services, integration glue, infrastructure automation: cases where the specification is precise enough and the blast radius of a mistake is bounded. StrongDM’s approach looks like genuine engineering discipline, not marketing. I expect more teams to work this way.
The question I keep circling
What I’m less sure about is how far this extends. There’s a body of research on AI-generated contributions that suggests a specific failure mode worth paying attention to.
Watanabe et al. (2025) found that 83.8% of agent-generated PRs are accepted. Nakashima et al. (2025) broke down the rejection categories for the rest and found a telling one: “not aligned with project interest.” Not broken code. Not failing tests. Code that works but conflicts with architectural direction the agent had no way to know about. SWE-Bench Pro tells a similar story: roughly half of frontier model failures are semantic, misunderstanding intent rather than failing to produce functional code.
These numbers describe the gap between correctness and alignment. A dark factory can verify correctness through testing, formal methods, property-based checking. Alignment requires knowing what a maintainer thinks about error handling philosophy, which abstractions the team considers load-bearing, why a previous proposal was rejected, whether a technical direction has shifted since last quarter. That knowledge lives in people, not in test suites.
Highly regulated industries are one obvious case where fully autonomous production doesn’t apply. Aviation software under DO-178C, medical devices under IEC 62304, defense and nuclear systems: these have regulatory frameworks requiring documented human review and traceability. The question for regulated software isn’t whether humans stay involved, it’s how to make their involvement more effective as agent velocity increases.
But I’m curious about a broader category too. Any project with multiple maintainers who carry architectural intent (open source projects, platform teams, long-lived internal systems) seems to have this property. The people reviewing contributions hold context that isn’t in the codebase. It’s in years of design decisions, rejected proposals, and learned preferences. That context is the thing agents lack, and it’s what the rejection data points at.
What we’ve been exploring
We’ve been working on this for a few months. The result is synthesist, a specification graph manager for AI-augmented projects. The core idea is something we’re calling disposition graphs: a structured, temporal model that gives AI agents a queryable representation of what stakeholders will accept, grounded in observable evidence.
The context a maintainer carries in their head becomes something an agent can query first: a stance on one topic, held by one person, read off evidence that stays on the record. When the stance moves, the new reading supersedes the old one and the prior record stays queryable.
A disposition is an assessed stance: what implementation choices a specific person will accept on a specific topic. Each disposition has a confidence tier (documented, verified, inferred, speculative) and is grounded in signals, which are immutable records of observable evidence like PR comments, review feedback, and meeting notes. Signals are bi-temporal. They track both when the event happened and when you recorded it, because reading a two-week-old PR comment today is different from reading it the day it was posted.
Dispositions are never deleted, only superseded. When evidence changes an assessment, the old record stays and a new one replaces it with a full supersession chain. You can query what a stakeholder’s stance was on any date, how it evolved, and what evidence supported each position. The temporal model matters because preferences aren’t static. A maintainer who was cautious about a dependency in January might be supportive by March after seeing it stabilize. The graph captures that trajectory.
The tool also enforces a workflow state machine with a hard human checkpoint. Before an agent writes code, it must orient (read the disposition landscape), plan (model work as tasks), and then agree: present the full plan and wait for explicit human approval. The AGREE phase is not advisory. The binary rejects task claims until a human says yes.
Seven phases, one checkpoint. An agent reads the landscape and proposes a plan, then stops: the binary rejects task claims until a human approves. Everything past the gate runs on the approved plan, and a plan that changes under reflection comes back for a fresh yes.
The idea is that an agent queries what collaborators will accept before generating code, not after. We’re calling this “frontrunning preference.” We have a paper describing the formal model, the related work, and the implementation. It’s coming. In the meantime, the tool is open source and we’re actively using it to manage our own open source work, including the development of this site.
What we’re watching
This is early. We don’t have quantitative data on whether disposition graphs reduce rejection rates or revision cycles. That evaluation requires sustained use across real projects, and we’re building that evidence as we go.
I’m curious whether any of the code assistance platforms will find this approach interesting. The forge platforms (GitLab, GitHub, Gitea) already have the raw signal data: years of PR comments, review feedback, issue discussions, merge decisions. A “preference advisory” layer that structures this data for agent consumption feels like a natural extension of what they already do. GitLab’s Knowledge Graph initiative is adjacent. I work at GitLab and I’d love to explore this internally. The problem is one I see every day.
BCG’s framing keeps coming back to me: the shift isn’t the absence of humans, it’s where humans spend their attention. For the software that stays lit, I think that attention is well spent on preference. Helping the people who govern a codebase express what they’ll accept, in a way that agents can query before they start building.