Talk three of three · October 2026
Meta models, SysML v2, and one question asked four ways
This talk lands in the week of 26 October 2026.
AI in Systems Engineering · week 9 · Andrew Dunn
Scroll to run it Read on: the plates are printed at their finished state
Plate I
A meta model is the small set of types and relationships laid over a trove of information, and SysML v2 is one built for engineering.
One
The model this talk runs on arrives as a pile of files.
Apollo 11 SysML v2 model, 28 files, 7,249 lines. Airbus, MPL-2.0.
It is the Apollo 11 mission written as an engineering model: the spacecraft, the requirements it had to meet, the actions the crew and the vehicle perform, and the phases of the flight. Nothing in the pile tells a program which of those a given line is.
Two
A meta model does not read the pile. It types it.
A meta model is the small set of types and relationships a trove of information is allowed to use: this is a part, this is a requirement, this is an action, and these may connect to those. SysML v2 is one of them, fixed by a standard before anyone describes a spacecraft, so every tool reads the same vocabulary.
Three
Typed, the pile sorts into counts you can act on.
From the shipped model: 194 part definitions, 296 requirement definitions, 110 action definitions, 16 state definitions, inside 2,544 typed elements.
Talk one ended by sending you to the larder lesson, where an agent was handed a pantry file, a preferences file and a folder of recipes: a list of what is in the kitchen, with nothing in it saying what any line is. Typing the pile is the other move. Write down a guess now: does stuffing the whole pile into the question beat typing it first?
Plate II
Once elements are typed the links between them are just as real, and a parser builds that graph from the syntax without asking a language model to find anything.
One
A parser reads the source text by its grammar, the way a compiler reads code.
It runs over the files once and returns what the syntax says is there: every definition, every usage, and every link one of them declares to another. The result is a knowledge graph, which is the same model written so a program can walk it: elements are nodes, relationships are edges, and each keeps its own name.
Two
The links carry as much of the engineering as the elements do.
From the shipped model: 272 satisfy links, 104 perform links, 14 phase states and 14 transitions, one of them the entry.
A satisfy link says which part answers which requirement. A perform link says which part carries out which action. The mission chain says which phase follows which, and what event ends the one before it. None of that was inferred. It was written down, and the parser read it.
Three
Nobody has to ask a language model where the parts are.
A codebase comparison put parsed graphs ahead of model-built ones; the model-built graph skipped 377 files on one repository.
Asking a model to extract the graph adds a step that can be wrong, and it is wrong in ways nothing downstream can see. Parsing is cheap, it is repeatable, and when the model changes the graph changes with it on the next run.
Plate III
Hold the question and the content fixed, change only the shape they arrive in, and the answer moves.
One
A representation is the shape the same facts arrive in.
Not different facts, and not a different question. The same seventeen files of the model, handed over twice: once as the source text they are written in, once as a table rendered from them. A duel holds everything else still and changes only that.
Two
On the same forty questions the two shapes land 40 points apart.
Forty tasks on this deck, Sonnet 5: the raw slice 0.59 at 76,658 tokens in, the rendered requirements table 0.19 at 7,906. Tokens are the pieces of text a model reads and is billed by.
The table is smaller, tidier, and easier to read. It also throws away most of what the questions ask about, and the score says so. Shape decides which facts are still in the room when the question is read.
Three
The effect holds where the content is identical too.
The same graph selection as table rows 0.12, in TOON, a compact row format, 0.14, Sonnet 5 over forty tasks. Prompt format alone moves GPT-3.5-turbo by up to 40% on a code translation task, and GPT-4 only narrows that spread (He et al. 2024, arXiv:2411.10541).
Take one selection out of the graph and write it two ways and the answer still moves, by less. The shape does work before the model has read a single fact, so it is a thing to measure rather than a thing to argue about.
the source text: 0.59 correct, 76,658 tokens in.
Plate IV
There are four ways to hand a model this graph, and retrieval is a rung rather than the finish line.
One
Four ways to hand over the model, measured on the same forty questions.
Forty tasks on this deck, Sonnet 5: a view 0.17, here a completeness report rendered from the model, the raw slice 0.59, retrieval 0.51, tools 0.90. The whole model 0.68 on GPT-5 mini over forty; Sonnet 5's own whole-model arm ran five tasks, at 0.60.
A view of the model is a report rendered from it, and this one averages 0.17 over the forty. Seventeen files of source text, pasted in whole, average 0.59. That jump is the biggest single gain on the ladder, and it is paid for in tokens every time somebody asks.
Two
Retrieval fetches the part of the graph a question touches, and sends only that.
Retrieval 0.51 at 10,276 tokens in, against the raw slice's 0.59 at 76,658. Sonnet 5, forty tasks.
An index is built over the model's own structure rather than over pages of text, the question is searched against it, and the neighbourhood around the answer comes back. The arm averages about half the points for about a seventh of the slice's tokens. That is real, and it is one pass.
Three
The three obvious ways to improve that pass were each measured, and each came back null.
All three on the first corpus under Sonnet 4: vector search against plain search 0.880 to 0.880; graph traversal at two to three hops 0.493 against search's 0.528, at about 69 times the tokens; planning tools +0.035 on hard tasks, underpowered.
Switch one on and the bar jiggles and settles back where the recording left it. Check your guess from chapter one here: stuffing does beat a bare view, and retrieval on its own beats neither.
Switch a probe on: the retrieval rung jiggles and settles back where the recording left it.
Plate V
The top rung is not a bigger view. It is an agent making one call at a time against the graph and reading only what the question needs.
One
An agent is a model that calls tools over the graph, one call at a time.
It is handed no view. It searches for an element, traces a link, inspects what came back, and renders the answer, choosing each call after the last one returned. The four operations are the same four the parser built the index for in chapter two.
Two
On this deck the agent reaches 0.90, and the smaller model reaches 0.83.
Forty tasks on this deck, the Apollo model: the agent 0.90 on Sonnet 5 at 41,514 tokens in, 0.83 on GPT-5 mini at 28,432. A live instrument moves; both numbers are dated where they are printed.
It reads a fraction of what the whole model costs and answers more of the questions than any single shape of it does. The gain comes from asking four narrow questions instead of one wide one.
Three
Two frozen results sit either side of this one, and neither is the comparison the chapter just made.
Pre-registered Apollo replication, Sonnet 4.5, explanation tasks, both arms agent loops: a view rendered from the graph 0.806 against the agent assembling those facts call by call 0.521, g=0.83, exact p=0.0007. First corpus, Sonnet 4, discovery tasks: an agent over command line search 0.900 against vector retrieval 0.569, p=0.018.
One says a view rendered for the question beats the agent assembling that view itself; the other says the agent beats vector retrieval. The same pre-registration carried a second effect, tool-selection guidance, which had looked real at g=0.71 and came back at g=0.11. It was reported as a failure to replicate, which is what the registration was for.