andrew.dunn.dev

kebnf

~3 min
grammarsysmlrustantlrtree-sitter
Source crates.io

The OMG publishes the concrete syntax of SysML v2 and KerML as KeBNF text, not as a grammar that ANTLR4 or tree-sitter can consume: its rules carry metamodel binding that no parser generator understands. kebnf, a single Rust binary, reads that text with its own chumsky parser into one AST, emits both grammar targets from it, and writes the binding it strips to a per-rule mapping file.

parsesrecordsOMG SPECKeBNF sourceKerML, SysML v2ONE ASTkebnfchumsky parserGRAMMARANTLR4walks the ASTGRAMMARtree-sitterpatterns, prefixes inlinedSIDE OUTPUTmapping.jsonmetamodel binding
From KeBNF text to two grammars
  • Parses the OMG’s KeBNF text for KerML and SysML v2 into one AST.
  • Emits an ANTLR4 grammar and a tree-sitter grammar from that AST.
  • Records the metamodel binding it strips to mapping.json.

KeBNF annotates its rules for metamodel binding: the metaclass a rule produces, property assignments, boolean flags, cross-references, and inline semantic actions that tell a conformant tool how to build the abstract syntax tree. None of it has syntactic meaning to a parser generator. kebnf strips it during conversion, classifies each rule by what had to go, and keeps the stripped annotations in the mapping, so a downstream tool can trace an emitted rule back to its metamodel.

The two backends treat the AST differently. KeBNF factors prefix keywords out into shared rules such as DefinitionPrefix, since that is where the binding attaches, and ANTLR4 takes that structure as written. tree-sitter’s GLR tables need the disambiguating keyword visible at the start of each rule, and a factored-out prefix hides it, so the emitter writes each definition and usage rule from a pattern table with its prefixes inlined, and walks the AST only for the remaining rules.

walkpatternrecordsKEBNF, AS PUBLISHEDDefinitionPrefixBasicDefinitionPrefix = isAbstract ?= ‘abstract’ | isVariation ?= ‘variation’DefinitionPrefix : Definition = BasicDefinitionPrefix? DefinitionExtensionKeyword*SIDE OUTPUTmapping.json: Definition, isAbstract ?=, isVariation ?=stripped from this rule, kept for downstream toolsANTLR4, FROM THE ASTdefinitionPrefix: basicDefinitionPrefix? prefixMetadataMember*annotations gone, factoring keptTREE-SITTER, FROM THE PATTERN TABLEpart_definitionseq(repeat($.prefix_metadata_annotation), optional(‘abstract’), ‘part’, ‘def’, …)prefix inlined, keyword seen earlymapping.json describes the KeBNF rule; for pattern-table rules that is not the grammar tree-sitter receives
One rule as KeBNF, ANTLR4 and tree-sitter
  • mapping.json holds the stripped annotations for every rule.
  • ANTLR4 keeps KeBNF’s rule structure, prefix rules included; CI compiles it and runs TestRig over a SysML v2 file.
  • tree-sitter definition and usage rules come from the pattern table, so for those rules the mapping describes the specification, not the emitted grammar; CI only generates the parser, and coverage against tree-sitter-sysml’s test snippets is measured by hand.
  • The README lists the constructs tree-sitter does not parse; TREE-SITTER-FINDINGS.md and AMBIGUITY-RESOLUTIONS.md carry the analysis behind the patterns.

kebnf runs as its own track. sysml parses SysML v2 with tree-sitter-sysml and the benchmark harness measures through that parser; kebnf’s grammars are checked by CI only for generating (an ANTLR4 compile with TestRig on a minimal file, and a tree-sitter generate), with snippet coverage against tree-sitter-sysml measured by hand, and not against large real-world models.