Example · a translation loop, measured
Translating John 3 from the Greek.
What shaped every verse.
A new American English edition — rbs — drafted from the original-language tokens alone. No English Bible in the drafter's prompt; an ONTOLOGY in it instead: referents, senses, rules and a brief, all held to by checks that are themselves measured. This page is the full account. First the loop as it ran on John 3 (sections 01–06); then the ontology itself — its four boxes and ten layers, and how it generates the skills, is the knowledge base, declares the plan and keys the harness (07–12); then the licence per layer; and at the end a detailed comparison with the scholarly committees, the SIL field stack, Greek Room and Clear, unfoldingWord and the Anselm Project Bible — where we are ahead, where they are, and why the difference is the ontology.
The picture
Six bands. The drafter sees the top three; the harness sees them all.
Read it top to bottom: what the drafter may see and what it may not; the MCP tools that serve it; the skill and the plan with its revise pass; the four checks; the harness — and, in the gutter, the three dashed arrows that carry what the harness found back into the checker, the skill and the knowledge base. The right-hand column is where a verse goes afterwards.
01The pieces
Four things touched every verse. None of them was a translation.
The constraint that makes the loop honest is in the first card: the drafter is never shown an English rendering of the passage. Everything that shapes a verse therefore has to come in through the source, the knowledge base, the skill, or the checks — and each of those is something the harness can measure and change.
The Hebrew/Aramaic and Greek tokens as the published tables carry them (WLC + NE1904), each with its parsing and Strong's number, frozen under a content commitment; the STEP lexicon entry each number opens; the scripture knowledge base. No English edition of the passage — ever. A runtime test asserts the WEB's wording is absent from the drafter's prompt.
Five tools on an internal Worker: get_source_tokens, lookup_lexicon, lookup_names, find_similar_approved, run_translation_checks. Each returns the JSON and a one-line-per-item text rendering (`prompt`) that the plan templates straight into the model's prefix — 30k input tokens a chunk fell to 19k when JSON stopped going to the model.
One SKILL.md (translate-from-source) says what the task is, what it covers, what it declines, and which four tools it may call. The orchestration compiles that into a stepped plan — tokens → lexicon → names → similar → draft → checks → revise (only when a check blocks) → final → propose — and records every step as a PROV activity.
Experiments over pre-registered case sets, five arms, deterministic graders first and a judge after, paired-bootstrap lifts, a regression suite that plants six kinds of error into the floor. Its first runs blocked two thirds of a real Bible; what it found rewrote the checker's rules, the skill's brief and the knowledge base, each fix as a test.
02The plan, step by step
Ten steps. Two of them think.
This is the plan the model-source-lexicon-memory arm runs for one chunk of eight verses. Every step is a PROV activity with its inputs named; every output is an entity; a guarded step that does not run is a recorded skip. The two model steps are the only probabilistic ones.
get_source_tokens, lookup_lexicon and lookup_names each return a prompt rendering — one line per token, lexeme or name, cantillation stripped — and the plan templates those into a cached prefix. The model never parses a structure to find a gloss.03The ontology in the text
Two layers of the knowledge base reached the drafter in John 3. Both are visible in the verses.
The scripture knowledge base is an A-box, not a glossary: 3,129 persons, 1,007 places, 5,169 key terms, 50 theological concepts whose definition IS the rendering rule, 16 divine names each with a convention, and — since October 17 — three divine persons the names are forms of. The lexicon tool attaches a concept's rule to the Strong's entry it governs; the names tool resolves a token to a referent and its decided rendering. Here is where each one shows in the chapter.
Names resolved to referents
Each of these was resolved through the knowledge base to its TIPNR IRI (an at:Person, an at:Location, a scr:NamedThing), so the check holds the draft to the referent's rendering rather than to a spelling: the Judas who was Simon's son is one IRI in all four Gospels, and the patriarch is another.
Four btr:NameRendering decisions were enforced, not advised: God, Jesus, Christ, Holy Spirit (decision-names-amer-v1). One unit (G5438 φυλακή “watch of the night”, with its approximate value) also reached the prompt and went unused.
model-src arm over the same chapter, about $0.80 — and has not yet been run on John 3.04The harness plan
Seven steps, in the order they have to happen.
A harness is a plan before it is a tool. This one freezes what it measures, pre-registers what it will measure it against, sets a floor and a ceiling, grades deterministically before it asks a judge, reads the floor and ceiling before it reads the model, pilots before it drives — and only then publishes under an issuer.
05What the harness changed
The harness did not grade the translation. It rewrote the rules the translation was held to.
Each of these is a measurement that disagreed with a text the measurement had no business failing — the WEB, the floor, a Jonah chunk, two verses of John 3 — and a rule that changed as a result, with a test so it cannot drift back. The skill's text, the checker, and the knowledge base are the three things it reached.
06The result
John 3:16, from the Greek.
Twenty-five tokens, twenty-five alignments, one negator carried, θεός rendered by convention and ζωὴ αἰώνιος by the knowledge base's rule. No reference edition was consulted while it was written; the graders compared it with four afterwards.
For God so loved the world that he gave his one and only Son, so that everyone who believes in him should not perish but have eternal life.
source-coverage 25/25 · negation 1 carried · names: θεός → “God” (conventional) · quantities: none in the source · issuer rbs-bible.svc
For God so loved the world that He gave His one and only Son, that everyone who believes in Him shall not perish but have eternal life.
Where they differ is where a rule or a reading decided: rbs does not capitalise deity pronouns, renders ἵνα “so that”, and μὴ ἀπόληται “should not perish”. Where they agree, the same Greek produced the same English twice, independently.
rbs, public domain, one signed descriptor per verse, verifiable against the corpus root — and under an issuer of its own (rbs-bible.svc, chartered by the scripture steward), because a derived edition does not carry the Berean name. John 3 is the first chapter; the rest of the Bible is a decision about spend (≈ $650 at the pilot's rate), not about method.07The ontology, in full
Four boxes, ten layers. The translation is held to all of them.
An ontology here is not documentation. It is the single source from which the skills, the knowledge base, the plan and the harness are generated or validated, so that a rule written once holds in all four places and a change in one is a measured change in the others. Read the four boxes first — what kinds of knowledge the ontology carries — then the ten layers, top to bottom, from the foundations the whole substrate shares to the translation domain and its two A-boxes.
08Ontology → skill
The skills are generated from the ontology. The ontology is not described by the skills.
A SKILL.md is the text a model reads to do one job. Here it is also a contract: which capability it realizes, which classes it covers and excludes, which tools it may call, and — generated from the SHACL shapes — exactly which typed record the run must leave. The same shapes validate the record afterwards. Promise and check cannot disagree because they have one source.
09Ontology → knowledge base
The knowledge base is the A-box. Consistency is bound to the referent, never to a word.
Two graphs, two licences, one spine. The CC BY knowledge base holds every person, place, being, key term, concept, divine name and unit the frozen source needs; the CC BY-SA explorer snapshot holds the events, collectives and memberships Theographic contributed, joined to the first by a crosswalk of cross-references. Both are generated from their sources and checked for drift; the curated parts — concept rules, divine persons, realia — are hand-authored and cited.
10Ontology → plan
The plan is a declared artefact. Its constraints are the invariants.
A plan of steps with typed inputs, outputs, guards and constraints is a P-Plan with EP-Plan constraints — the same vocabulary the foundations layer provides. Declaring the translate plan in it turns the loop's invariants from prose into things a validator checks before a run and a grader reads after one, and makes the plan identical across target languages by construction.
11Ontology → harness
The harness is keyed to the ontology. Every number says which layer it measures.
A harness that reports one score tells you nothing about what to change. This one maps graders to the skill, knowledge-base file or ontology layer they measure, draws its perturbation kinds and difficulty tags and arm kinds from the controlled vocabularies, validates a run's records against the shapes the skills were generated from, and closes the loop: a finding names an ontology element, a release regenerates every projection, and the same pre-registered set accepts or reverts it.
12Open throughout
Open source from the text to the T-box, with the one share-alike graph kept apart.
The edition and everything that produces it are open. The inputs are public domain and attribution-only, so the text and the ontology stay attribution-only too; Theographic's share-alike data lives in its own graph with its own licence file and is consulted, not merged; code is Apache-2.0; every dependency is pinned in a manifest that fails the build when a licence is outside the policy.
13The field
How this differs from the leading translation efforts — in detail, and honestly.
What exists, as each project publishes it (read on 17 October 2026; nothing here is copied from any of them). For each class of effort: what they do, what we do differently, and in both directions what the other is ahead on. The through-line is the same in every row: where they hold their decisions as documents and their consistency as strings, we hold them as an ontology a machine is measured against and a second project can import.
What they do
The gold standard of the craft: a decided critical text (BHS/BHQ, NA28/UBS5), scholars per book in two or three tiers of review, full lexica and discourse analysis, key-term lists and style sheets maintained by the committee, Paratext's biblical-terms and parallel-passage tools, a preface that states the philosophy. NET publishes 60,000 translator notes. The process is the review; nothing ships without it.
What we do differently
We take the committee's DISCIPLINE and make it machine-checkable: the key-term list is a graph of referents and senses with a rendering per language; the style sheet is a typed brief with a digest; the parallel-passage tool is a grader over groups the benchmark already keeps; every rendering decision is a record with a model, a prompt version and a reviewer. What a committee holds in its members' heads and its minutes, the ontology holds as data a run is measured against. What we lack is exactly their strength: no verse has yet passed a human exegete, and the textual basis is declared (Nestle 1904 as the tables carry it) rather than decided unit by unit. Both are open items with a path (the blind review queue; the 1,759-unit textual ledger).
Who is ahead on what. Ahead of us: human exegetical review, a decided critical text, discourse analysis, readability testing. Behind us: nothing of theirs is reproducible by a second party from the published artefacts; ours is, verse by verse, from frozen commitments.
What they do
Public-domain or open texts made by a small team or one editor with volunteers; the BSB publishes its interlinear tables with eight editions' variant marks, which is the single most useful open artefact in the field — and the frozen source our tokens come from. Light textual criticism; interlinear-driven consistency; minimal automated checking.
What we do differently
We stand on the BSB's tables and freeze them under content commitments; the gloss they carry is our FLOOR (the arm every model arm must beat), never a source the drafter sees. Where the BSB's consistency is interlinear and a team's eye, ours is a referent graph and a measured check; where its variant marks are footnotes, ours are a ledger of typed textual decisions awaiting sign-off. The licence shape is the same (public domain / CC BY) and deliberately so.
Who is ahead on what. Ahead of us: a complete, published, human-read Bible. Behind us: no provenance beyond the team's word; no measured checker.
What they do
The stack that serves actual minority-language translation: an NMT model fine-tuned on a project's own ~8,000 translated verses drafts the next book; AQuA assesses drafts (missing words, semantic similarity to a reference, naturalness outliers) and gives reviewers heat maps; Paratext runs the biblical-terms and consistency checks; FOBAI's process requires team, consultant and community review. SIL's own line: "all drafts contain errors". Serval is open source; project data stays with the project.
What we do differently
Different problem, same values. Serval learns the target language from the team's own verses; we translate from the original into a language the model already knows, so our risk is the opposite one — RECITATION of published editions, which we measure (John 3 overlapped the BSB at 0.56 where BSB and WEB overlap each other at 0.33) and gate. Where AQuA compares a draft to a reference, our checks compare it to the SOURCE TOKENS it came from (coverage, negation, names through the referent graph, quantities) and the references are graders' instruments only. Where Paratext's biblical terms are strings per project, ours are referents and senses shared across every edition. The reviewer dashboard AQuA has is on our list; the measured checker precision and recall we publish is not on theirs.
Who is ahead on what. Ahead of us: deployment to hundreds of real teams, consultant review, community testing — the whole human process. Behind us: the checker's own false-positive and catch rates are not published; evaluation sets are not pre-registered and locked.
What they do
The best open checking machinery in the field: spelling by phonetic similarity and alignment, terminology consistency, omissions, names, quantities, parallel passages, character-set checks (Greek Room); morphology, syntax trees, semantic roles, gold alignments and speaker-quotation spans for the whole Bible under CC BY (Clear). Built for reviewers of human translations.
What we do differently
Clear's data is where our next layers come from: Macula's domain and frame fields for senses (so πνεῦμα "wind" and "Spirit" become two senses, not one number), speaker-quotation spans for the speaker check and the referent pass, gold alignments to score the drafter's self-reported alignments. Greek Room's checks and ours overlap on names, quantities and consistency; the difference is that ours run INSIDE the loop (a blocking failure sends the draft back before anyone sees it) and are themselves scored on a floor, a ceiling and a planted-error suite on every change. Theirs are for a reviewer; ours are for the reviewer and for the model, and are measured.
Who is ahead on what. Ahead of us: syntax trees and gold alignments in the drafter's context and the grader (planned, CC BY). Behind us: no numbers on their own checks' precision; no loop.
What they do
Gateway-language literal and simplified editions plus per-verse notes and per-term articles, so church-based teams can translate with helps; alignment-based checking in translationCore; checking levels 1–3. Everything CC BY-SA.
What we do differently
Translation Words is the closest thing in the field to our key-term and concept layer — about a thousand articles on terms, names and realia — and share-alike, so it goes beside Theographic in the separately licensed graph, consulted and cited, never merged into the CC BY knowledge base or the text. Our concept rules differ in kind: each is a RENDERING RULE for one edition with a rationale and a lexicon citation, bound to Strong's numbers (and soon senses) the checker can hold a draft to; theirs are explanations for a human translator. Both are needed; ours is the one a machine is measured against.
Who is ahead on what. Ahead of us: breadth of helps and a mature church-based checking process. Behind us: nothing checkable by code.
What they do
The closest cousin and in several respects ahead: OSHB + SBLGNT/MorphGNT as the source; 14,877 variation units from open apparatus decided blind by two or three model families; one LLM in nine roles (analysis, draft, segment review, audit, terminology, style, proofread, formal English, blind meaning review); the drafter not told it is Scripture, no book or verse reference; a RECALL GATE (≥ 30 shared words with earlier text fails) and a post-hoc comparison with four published versions against a ref-vs-ref ceiling; per-book consistency passes; eight blind English readers; every verse opens its tokens, reasoning and notes; the text CC BY 4.0. No person edited a verse.
What we do differently
APB proved the blinded drafter and the recall gate work; we adopted both (the gate is live, the acceptance run on John 3 is pending) and the textual ledger is being built on the same open apparatus. Where we go further: APB's consistency is per lemma and its ledger is a published log; ours is per REFERENT with kinship, period and geocoding on the node, and the provenance chain is VERIFIABLE — frozen source commitments → PROV run record → content commitment per verse → a chartered issuer's signature → a Merkle root anyone can check. APB reports its audits; we publish the checker's own precision and recall and refuse a rule change the regression suite prices badly. APB has no human in the loop by design; we keep a blind review queue because a translation without an exegete is a draft, however good. And every term, sense, rule and referent here is an open ontology a second project can import, not a project-internal table.
Who is ahead on what. Ahead of us: a complete Bible; a decided critical text with 14,877 units; multi-family review on every verse; readability testing. Behind us: a verifiable provenance chain; a referent graph with an open T-box; measured checker precision/recall; pre-registered locked evaluation sets; a human review path.
What they do
The general harnesses run experiments over any model and task with custom scorers, trajectory inspection, log viewers and rescoring; the Last Translation Benchmark attaches explicit verification rules to each hard example; RATE evaluates non-literal language with retrieval and pairwise comparison; the learned metrics score similarity to a reference.
What we do differently
We own the runner (the experiment lives in the agent runtime beside the org's grant and the gateway knobs) and take the PATTERNS: rescoring stored runs, trajectory graders, per-case verification rules (as typed acceptance conditions), evaluator validation against reviewers. The one thing we refuse that the general tooling invites is treating similarity to a published English Bible as quality — chrF against the WEB is a recitation DIAGNOSTIC here, reported per edition and never quoted as a score, because a drafter tuned on it learns to recite the WEB.
Who is ahead on what. Ahead of us: log viewers and a decade of UX. Behind us: nothing domain-specific — they supply the harness; the Scripture tests, the referents and the rules are ours to bring.
The differentiated value, in eight sentences
One ontology. Four things generated from it. Every one of them measured.
Frozen sources, a referent graph, skills generated from shapes, a declared plan, deterministic checks with a floor and a ceiling, a harness keyed to the layers it measures, and a licence per layer — this is how every skill on the substrate is built and assessed. The translation is the example where the thing measured is a sentence of Scripture, which makes the power of the ontology, and its limits today, easy to see.