LLM-Mediated Experimentation

A bug in bootc (a container-native OS installer) had been open for over a year. bootc install to-disk --block-setup tpm2-luks hung indefinitely at cryptsetup luksOpen. The community attributed it to TPM2 token behavior and PCR mismatch. Multiple GitHub issues (#421, #476, #477, #561), detailed analysis, proposed fixes. All focused on the TPM2 layer.

The root cause turned out to be a SysV semaphore deadlock between libdevmapper and udevd across the container’s isolated IPC namespace. It had nothing to do with TPM2.

An LLM found it in a single session because it had access to a test bench.

The community’s year of work

Jon Polom, an experienced contributor, had been working this bug without any LLM assistance. He correctly identified the shim version / PCR mismatch problem, proposed two practical fixes (keep the passphrase, check shim versions), advocated for an outer installer architecture, and caught the systemd 258 PCR default drift before anyone else. He was right about all of it.

The challenge was that the symptoms (hang at luksOpen) looked like a TPM2 problem. The hang happened during LUKS activation, which is where TPM2 tokens are consumed. Every debugging path led back to the TPM2 layer because that’s where the observable failure occurred.

What changed: a test bench

I gave the LLM a GCP test bench. Ephemeral VMs with nested KVM, real block devices, a real vTPM, real cryptsetup, real bootc. The LLM could provision VMs, run installs, capture output, inspect kernel stack traces, modify source code, rebuild, and test again. Each VM auto-deleted after 2 hours.

Without the bench, the LLM did what it initially did: read the code, reason about TPM2 tokens, and produce a plausible but wrong analysis. The bench let it fail, observe, and pivot.

Eight runs

Over 8 test runs across 4 fresh VMs in a single session:

Run 1 reproduced the hang. Stock bootc, default podman flags, tpm2-luks block setup. The install hung at cryptsetup luksOpen as reported.

Run 2 tried the community’s proposed fix: reorder luksOpen to happen before TPM2 enrollment. The theory was that the TPM2 token state was interfering with the initial LUKS activation.

Run 3 was the critical moment. The reordered fix still hung. The LLM couldn’t explain it away. If the hang were caused by TPM2 token state, reordering should have fixed it. It didn’t. Something else was blocking luksOpen.

Run 4 captured /proc/PID/stack of the hanging process:

[<0>] __do_semtimedop+0x3a8/0xd50
[<0>] do_semtimedop+0x15e/0x1a0
[<0>] do_syscall_64+0x7e/0x6b0
[<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e

__do_semtimedop. Not a TPM2 call. Not a cryptsetup call. A SysV semaphore operation. The LLM identified this as a libdevmapper udev cookie, a synchronization mechanism between device-mapper and udevd.

Run 5 tested two hypotheses. DM_DISABLE_UDEV=1 (skip udev synchronization for device-mapper) completed the install. --ipc=host (share the host’s IPC namespace) also completed the install. Both confirmed the diagnosis: the semaphore deadlock was caused by IPC namespace isolation between the container and the host’s udevd.

Run 6 ran a definitive 3-test suite on a fresh VM: stock (hang), DM_DISABLE_UDEV=1 (pass), --ipc=host (pass). 100% reproducible.

Runs 7 and 8 built the patched binary from source and verified the fix worked as a code change, not just an environment variable.

The root cause

libdevmapper uses SysV semaphores (“udev cookies”) to synchronize device-mapper operations with udevd. When bootc runs inside a container with the standard podman run --privileged --pid=host invocation, the container has an isolated IPC namespace by default. udevd runs on the host in the host’s IPC namespace.

cryptsetup luksFormat creates a LUKS volume. libdevmapper creates a semaphore in the container’s IPC namespace. cryptsetup luksOpen tries to activate a dm-crypt mapping. libdevmapper waits for udevd to signal completion via the semaphore. udevd cannot see the semaphore because it is in a different IPC namespace. The semaphore is never released. luksOpen blocks forever.

Users who happened to pass --ipc=host to podman would not have hit this bug. Jon likely never hit the semaphore issue because he was probably passing --ipc=host or equivalent namespace flags when he tested. The LLM hit it because it used the default podman invocation from the bootc documentation.

What the human did

I did not debug the bug. I set the direction (“fix the LUKS hang”), provided context from Jon’s analysis, reviewed risk (“are we confident this is safe during installation?”), approved external actions (posting the issue and PR), caught quality issues (“the line wrapping looks weird in the commit message”), and made architectural calls (“scope the fix to the Tpm2Luks code path, not globally”).

The LLM did the experimentation, iteration, and discovery. The human did judgment, review, and stakeholder management.

The bench is the breakthrough

The critical moment was Run 3. When the “correct” fix still hung, the LLM couldn’t explain it away. A human might have spent hours debugging the TPM2 token path, because the hypothesis was strong and the symptoms were consistent with it. The LLM checked /proc/PID/stack and found the semaphore in minutes. It was not attached to any theory. It followed the evidence.

8 test runs in one session. Each run: provision VM, install dependencies, build from source, run test, capture output, analyze, modify, repeat. A human doing this manually would spend a day per cycle. The LLM did it in hours, and it did not lose context between runs. Each failure informed the next hypothesis.

Ephemeral infrastructure made this possible. GCP VMs with 2-hour auto-delete. Each test run on a fresh VM. No state leaking between runs. The LLM could trash a VM and spin up a new one in minutes.

The expert was right

Jon’s analysis was correct on every point he investigated. The shim version / PCR mismatch is a real problem. The passphrase and recovery key proposals address real gaps. The outer installer architecture would avoid this entire class of namespace isolation issues. The systemd 258 PCR default change is a real concern.

The LLM’s contribution was finding one additional bug (the semaphore deadlock) that was masking everything else. It found it because it had a bench to fail on and because it approached the problem from a different angle: brute-force iteration on a test bench versus deep domain knowledge. The two are complementary, not competitive.

Logging everything

The first attempt at this bug (v1) failed partly because we did not log our test runs. When Jon asked what we had tried, we could not give precise answers. For v2, every run was logged with exact commands, full output, versions, and analysis. This log became the evidence we shared with maintainers in #2089.

The log also made the PR review straightforward. The maintainer could see exactly what was tested, in what order, with what results. No ambiguity about whether the fix was verified.

What other bugs could benefit

Any bug where the failure mode is observable but the root cause is hidden behind a plausible-but-wrong hypothesis. The pattern: give the LLM a bench where it can reproduce the failure, try the obvious fix, watch it fail, and then dig deeper without anchoring bias.

The infrastructure pattern is simple: ephemeral VMs (or containers, or test environments) that the LLM can provision, use, and destroy. A test script that reproduces the failure. Logging that captures everything. The LLM handles the iteration. The human handles the judgment.

This connects to the broader question of how we keep AI-assisted work disciplined and to the gap between what pipelines test and what actually breaks. The harness exploration work was the foundation that made this kind of LLM-mediated debugging possible.