Talk two of four · 26 October 20261 / 34

The ladder and the meter

From ontology to meta-model

AI in Systems Engineering · week 8 · Andrew Dunn

Part 0 · Meta2 / 34

Andrew Dunn

Building tools, infrastructure and systems that let someone do work they couldn’t do before.

andrew.dunn.dev/activity

andrew@dunn.dev

  1. BackgroundModeling and simulation, built on software, statistics and platform engineering
  2. Systems engineeringPractised under Dr. Rapp
  3. RolePost-sales engineering at GitLab, since leaving public service last year · gitlab.com/andunn
  4. ResearchMethods for evaluating how language models perform when handed a meta-model, on SysML v2 · gitlab.com/nomograph
Part 0 · Re-entry3 / 34

Three weeks, one climb

  1. Week 7 · Hands and feet a language model becomes an agent, on an everyday task
  2. Week 8 · The ladder and the meter what things are, how they relate, and how you score what a language model does with them
  3. Week 9 · Structure over the trove SysML v2, a meta-model for systems, over a real model

Each week: talk → scroll copy (alone, about twenty minutes) → lesson or article

Part 0 · Re-entry4 / 34

Last week, in one slide

Part 0 · Re-entry5 / 34

Today’s question

A language model can act. How does it know what things are, and how they relate?

Part I · The words6 / 34

Two lines, two jobs

Part I · The wordsWord 1 of 11

Class

Part I · The wordsWord 2 of 11

Instance

Part I · The wordsWord 3 of 11

Property

Part I · The wordsWord 4 of 11

Relation

Part I · The wordsWord 5 of 11

Kind of

Part I · The wordsWord 6 of 11

Constraint

Part I · The wordsWord 7 of 11

Ontology

Part II · With a language model14 / 34

The harness, again

Part II · With a language modelUse 1 of 5

In the window

Part II · With a language modelUse 2 of 5

Prose in, structure out

Part II · With a language modelUse 3 of 5

Check

Part II · With a language modelUse 4 of 5

Repair loop

Part II · With a language modelUse 5 of 5

Follow the links

Part II · With a language model20 / 34

What an evaluation is

Part II · With a language model21 / 34

The meter

Forty questions on the Apollo 11 SysML v2 model (Airbus, MPL-2.0). Score: the share of a question’s fields right, averaged over forty; fully right counted apart.

Part II · With a language model22 / 34

Measured

Tokens a question: 76,658 · 10,276 · 41,514. Sonnet 5 throughout. The agent’s graph was parsed from the files, not extracted by a language model.

Part III · One file, three jobs23 / 34

One ontology, three jobs

Part IV · Up one levelWord 8 of 11

Meta-model

Part IV · Up one levelWord 9 of 11

Level

Part IV · Up one level26 / 34

SysML v2

Part IV · Up one level27 / 34

The full column

Part IV · Up one level28 / 34

KerML and MOF

Part IV · Up one levelWord 10 of 11

Definition

Part IV · Up one levelWord 11 of 11

Usage

Part IV · Up one level31 / 34

A rule on every level

Part V · Your turn32 / 34

Read the scroll copy first

Stuffing, retrieval and tools on one systems model, and how each was scored, in six chapters.

andrew.dunn.dev/guides/ladder-and-meter/

QR code for andrew.dunn.dev/guides/ladder-and-meter/
Part V · Your turn33 / 34

The ladder

Eight rungs, cheap to durable. The lowest rung that durably fixes the behavior, picked on evidence.

andrew.dunn.dev/writing/the-flywheel-and-the-meter/#building-a-flywheel-to-traverse-the-ladder

QR code for andrew.dunn.dev/writing/the-flywheel-and-the-meter/
Part V · Your turn34 / 34

Discussion

  • andrew.dunn.dev/talks/ladder-and-meter/
  • andrew.dunn.dev/guides/ladder-and-meter/
  • andrew.dunn.dev/lessons/ontology/
  • andrew.dunn.dev/writing/the-flywheel-and-the-meter/
  • andrew.dunn.dev

In pairs: your field in five kinds, three links, two rules.

One wrong answer a language model gave you: which kind, link or rule would have caught it?

Next week: what a meta-model does, on a real SysML v2 model.