Talk two of four · 26 October 2026 1 / 34
The ladder and the meter
From ontology to meta-model
AI in Systems Engineering · week 8 · Andrew Dunn
Last week a language model became an agent.
Today: what it knows about the things it handles.
Thirty minutes of talk, then thirty of discussion. Laptops closed until then.
Part 0 · Meta 2 / 34
Background Modeling and simulation, built on software, statistics and platform engineering
Systems engineering Practised under Dr. Rapp
Role Post-sales engineering at GitLab, since leaving public service last year · gitlab.com/andunn
Research Methods for evaluating how language models perform when handed a meta-model, on SysML v2 · gitlab.com/nomograph
Andrew Dunn; the email is on screen.
Modeling and simulation first; systems engineering under Dr. Rapp.
Research: how language models perform when handed a meta-model.
Part 0 · Re-entry 3 / 34
Three weeks, one climb
Week 7 · Hands and feet a language model becomes an agent, on an everyday task
Week 8 · The ladder and the meter what things are, how they relate, and how you score what a language model does with them
Week 9 · Structure over the trove SysML v2, a meta-model for systems, over a real model
Each week: talk → scroll copy (alone, about twenty minutes) → lesson or article
Last week: an agent, a harness, a memory system.
Today: what things are, how they relate, how we score it.
Last week's row said "what the model is shown". Today's is wider.
Next week: SysML v2, a standard meta-model for systems.
Part 0 · Re-entry 4 / 34
Last week, in one slide
Language model, context window, system prompt, it forgets.
Tool, tool call, loop, harness, agent.
A memory system; every change carries a name; the loop stops for a person.
The memory system held lines of text. Nothing said what a line is.
Part 0 · Re-entry 5 / 34
Today’s question
A language model can act. How does it know what things are, and how they relate?
Last week's memory system held text with no stated meaning.
The answer has a name: an ontology.
Eleven words today. The first seven build an ontology, on one kitchen.
Part I · The words 6 / 34
Two lines, two jobs
Two lines: butter is dairy, dairy is perishable.
Is butter perishable? Nobody wrote it. It follows. That's inference.
One rule: every perishable has an opened date. Butter has none. A program catches it.
No language model in either job. Now we build the words for this, on a bigger kitchen.
Part I · The words Word 1 of 11
Class
A class is a kind of thing, not any one thing.
Recipe is a kind. So is Ingredient.
An ontologist reads a class as the set of its members.
Part I · The words Word 2 of 11
Instance
Black bean chili is a Recipe. Black beans is an Ingredient.
Kinds on top; particular things below, each tied to its kind.
Black beans is one entry in the household's list: one thing, not a bag.
OWL's word for them: individuals.
Part I · The words Word 3 of 11
Property
The ontology says a recipe has a minutes value, a number.
Black bean chili carries one: 40.
A value is data. It points at no other thing.
Whether a recipe must have one is a rule; slide 12.
Part I · The words Word 4 of 11
Relation
The ontology says a recipe uses ingredients: a link between kinds.
The facts state one: black bean chili uses black beans.
This thing, the link, that thing. Three parts: a triple.
Part I · The words Word 5 of 11
Kind of
Legume is a kind of Ingredient.
Black beans is a Legume.
So black beans is an Ingredient: it follows from two written lines.
Standards call this a subclass.
Part I · The words Word 6 of 11
Constraint
A constraint: a rule the facts must keep, testable by a program.
Every recipe uses at least one thing, and each is an Ingredient.
Every recipe has exactly one minutes value, a number.
Gruber calls the written statements axioms. Here a program tests them.
Part I · The words Word 7 of 11
Ontology
Gruber, 1993: an explicit specification of a conceptualization.
Conceptualization: the things, kinds and links we take the kitchen to have.
Inside the line, the ontology proper, written down. Below it, facts in its words.
One file can hold both.
Part II · With a language model 14 / 34
The harness, again
Last week's loop, drawn round today's graph.
The facts sit in the memory system: that is what its lines were.
The ontology is one file beside them.
The language model is the same one. We change what we hand it.
Part II · With a language model Use 1 of 5
In the window
The ontology is short, so it rides in every conversation.
Now the language model writes Recipe, uses, Legume: our words.
The facts stay in the memory system until asked for.
Part II · With a language model Use 2 of 5
Prose in, structure out
Same facts twice: a sentence on the card, structure in the graph.
A person reads both. Only the graph lets a program follow links.
Dashed means written, not yet accepted.
Typed links, stored as facts: the caption's knowledge graph.
Part II · With a language model Use 3 of 5
Check
The rules run as a program, the same way every time.
Three facts pass. Minutes fails: about half an hour is not a number.
Chickpeas count as an Ingredient through the written kind-of line.
Checked as written: a missing value fails, an unwritten kind-of line counts for nothing.
Part II · With a language model Use 4 of 5
Repair loop
The failed rule goes back to the language model, by name.
It answers 30. The check passes. The facts join the memory system.
After the last try, our loop stops for a person, as talk 1's did.
Part II · With a language model Use 5 of 5
Follow the links
The language model calls a tool that starts at Legume and follows links.
It keeps recipes under 45 minutes. Two come back.
Each answer arrives with the path that justifies it.
Part II · With a language model 20 / 34
What an evaluation is
Without a fixed key, "it got better" is an impression.
Today: a first look, forty questions on the Apollo 11 SysML v2 model, swapping only what the model is shown.
Next week: designing evaluations yourselves, on a real SysML model.
Part II · With a language model 21 / 34
The meter
Forty questions on the Apollo 11 SysML v2 model (Airbus, MPL-2.0). Score: the share of a question’s fields right, averaged over forty; fully right counted apart.
Forty fixed questions on one systems model, each with a key.
Score: the share of a question's fields right, averaged over forty.
Fully right is counted apart. Same key every arm, Sonnet 5 throughout.
Swap what sits between question and language model; the meter stays.
Part II · With a language model 22 / 34
Measured
Tokens a question: 76,658 · 10,276 · 41,514. Sonnet 5 throughout. The agent’s graph was parsed from the files, not extracted by a language model.
Seventeen files in the window: 0.59, twenty fully right, $0.32 a correct answer.
Nearest chunks: 0.51 on a seventh of the tokens, $0.06 a correct answer.
An agent with tools over the parsed model: 0.90, 34 right, $0.11.
Of these three, retrieval is cheapest per correct; the agent gets most right.
Part III · One file, three jobs 23 / 34
One ontology, three jobs
One file: three kinds, a kind-of line, a link, a property, two rules.
The language model writes in it; a program checks against it.
The memory system keeps facts in its kinds and links.
Every wrong answer: a kind, link or rule the file lacks. Add it there.
Part IV · Up one level Word 8 of 11
Meta-model
Black bean chili is a Recipe. Recipe is a Class.
Two kinds of "is a": one about kitchens, one about the language.
A meta-model: the words a model is written in, and their rules.
Six of today's words are the kitchen's meta-model.
Part IV · Up one level Word 9 of 11
Level
M0: the pot on the stove. The model's chili entry stands for it.
M1 holds kinds and things together; there, the chili is a Recipe.
M2 is the six words M1 is written in.
Each level is written in the words of the one above.
Part IV · Up one level 26 / 34
SysML v2
SysML v2 is a meta-model for systems, its kinds fixed by a standard.
Part, port, connection, constraint, requirement: five of its definition kinds.
Its models sit at M1, like ours; its M0 is spacecraft and crew.
A field with a standard meta-model starts from it. Systems engineering has one.
Part IV · Up one level 27 / 34
The full column
Plain words first: the real thing, the model, the language, the language for languages.
KerML is the core that SysML v2 is built on. MOF is the top, and it describes itself.
Next week we work at M1, on a real SysML v2 model, using M2 as the guide.
Part IV · Up one level 28 / 34
KerML and MOF
SysML v2 extends KerML's meta-model. Both are MOF models.
MOF describes itself: the top of this column.
The kitchen column stops at M2: its six words have no formal definition.
KerML's Semantic Library, written in KerML, is the caption's ontological model.
Part IV · Up one level Word 10 of 11
Definition
part def Astronaut plays the part Recipe played: a kind, in the model.
On a def, ":>" reads kind of: Astronaut is a kind of Person.
The definitions of a SysML model do the kitchen ontology's job.
They sit at M1 beside the things they classify, as Recipe did.
Part IV · Up one level Word 11 of 11
Usage
crew is Astronaut used inside a mission: a usage. The kitchen's "uses".
[1..*] is a multiplicity: at least one. Our rule, in the language's words.
A usage is not an astronaut. The three who flew are M0.
A line names Neil Armstrong. It is still model text, at M1.
Part IV · Up one level 31 / 34
A rule on every level
A requirement is a constraint a stakeholder puts on a design.
In the model: AstronautSafety, a requirement def, at M1.
In the language: requirement def is a kind of constraint def.
Also there: every usage has a definition. crew has Astronaut.
Part V · Your turn 32 / 34
Six chapters, about twenty minutes, alone.
It walks stuffing, retrieval and tools, and the scorer behind slide 21.
Read it before next week; talk three builds on it.
Part V · Your turn 33 / 34
Eight rungs. The bottom four change in minutes and roll back free.
Today's work sits on rungs one, two, four and five: the four arms on one and four, the ontology in every conversation on two, the checker on five.
Climb only when a scored answer says to.
Part V · Your turn 34 / 34
Discussion
andrew.dunn.dev/talks/ladder-and-meter/
andrew.dunn.dev/guides/ladder-and-meter/
andrew.dunn.dev/lessons/ontology/
andrew.dunn.dev/writing/the-flywheel-and-the-meter/
andrew.dunn.dev
In pairs: your field in five kinds, three links, two rules.
One wrong answer a language model gave you: which kind, link or rule would have caught it?
Next week: what a meta-model does, on a real SysML v2 model.
In pairs: your field's kinds, links and rules.
Then one wrong answer a language model gave you, and the missing piece.
Next week: what a meta-model does, on a real SysML v2 model.
The links stay up. Questions.