SNAPKITTYWEST commited on
Commit
8accf92
Β·
verified Β·
1 Parent(s): fc3e70a

Model card: restore full README from GitHub, add commercial license option

Browse files
Files changed (1) hide show
  1. README.md +1520 -0
README.md CHANGED
@@ -15,3 +15,1523 @@ tags:
15
 
16
  > Mirrored from [https://github.com/SNAPKITTYWEST/semantic-engine](https://github.com/SNAPKITTYWEST/semantic-engine) at commit `b9afd82`. Part of the **SnapKitty October 2026 main drop**.
17
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  > Mirrored from [https://github.com/SNAPKITTYWEST/semantic-engine](https://github.com/SNAPKITTYWEST/semantic-engine) at commit `b9afd82`. Part of the **SnapKitty October 2026 main drop**.
17
 
18
+ README
19
+ # Semantic Engine: Deterministic NL β†’ IR β†’ Boolean β†’ Latin β†’ SUBLEQ Pipeline
20
+
21
+ License: MPL-2.0
22
+
23
+ ========================================================================
24
+ SOVEREIGN LEVIATHAN NODE LICENSE
25
+ License-ID: SL-AGPL3-001 | Covenant-Version: 1.0
26
+ Copyright (C) 2026 SnapKittyWest. Ahmad Ali Parr,
27
+ Bel Esprit D'Accord Irrevocable Trust.
28
+ ========================================================================
29
+
30
+ This work is licensed under dual license terms:
31
+
32
+ MPL-2.0 OR GPL-3.0-or-later
33
+
34
+ With additional Sovereign Leviathan Node License terms (AGPL-3.0 base).
35
+ Source files are individually licensed under MPL-2.0 (file-level copyleft).
36
+ Commercial repository β€” no MIT permitted.
37
+
38
+ This file is a covered work under the GNU Affero General Public License,
39
+ version 3, together with the Sovereign Leviathan additional terms.
40
+
41
+ "Hark, though this node be but a spark,
42
+ Its covenant endureth through the dark.
43
+
44
+ Ignorantia juris non excusat."
45
+ ========================================================================
46
+
47
+ ## Table of Contents
48
+ - [Executive Overview](#executive-overview)
49
+ - [Project Vision](#project-vision)
50
+ - [Core Architecture](#core-architecture)
51
+ - [Pipeline Stages](#pipeline-stages)
52
+ - [Pipeline Flowchart](#pipeline-flowchart)
53
+ - [Verification Framework](#verification-framework)
54
+ - [Boolean Engine & CNF Normalization](#boolean-engine--cnf-normalization)
55
+ - [Token Engine & Binary Serialization](#token-engine--binary-serialization)
56
+ - [Latin Canonical Layer](#latin-canonical-layer)
57
+ - [SUBLEQ IR & Virtual Machine](#subleq-ir--virtual-machine)
58
+ - [Verification Flowchart](#verification-flowchart)
59
+ - [Agent Harness](#agent-harness)
60
+ - [CLI Reference](#cli-reference)
61
+ - [Building & Installation](#building--installation)
62
+ - [Testing & Validation](#testing--validation)
63
+ - [Implementation Architecture](#implementation-architecture)
64
+ - [Performance & Design Constraints](#performance--design-constraints)
65
+ - [Contributing & License](#contributing--license)
66
+
67
+ ---
68
+
69
+ ## Executive Overview
70
+
71
+ Semantic Engine is a 20-phase, deterministic transformation pipeline that converts natural language input into formally verified SUBLEQ machine code, passing through typed IR, Boolean normalization, Latin canonicalization, and templated artifact emission. Every transformation is gated by an explicit finite state machine. Every temporal mutation is tracked. Every output is verified before acceptance.
72
+
73
+ ### What This Is
74
+
75
+ A hand-rolled, zero-dependency semantic compiler implementing:
76
+
77
+ - **20-State Finite State Machine**: Every phase transition is explicit and typed. Invalid transitions fail closed.
78
+ - **Typed Semantic IR**: Operations, operands, constraints, geometry, and temporal state β€” all content-addressed with SHA-256.
79
+ - **Boolean CNF Normalization**: Full De Morgan, distribution, constant folding, and idempotency guarantees.
80
+ - **Binary Token Codec**: `decode(encode(x)) == x` for all canonical Boolean structures.
81
+ - **Latin Controlled Vocabulary**: 28 canonical lexemes, no uncontrolled synonyms, deterministic token→lexeme mapping.
82
+ - **SUBLEQ Target Machine**: One-instruction computer (`mem[B] -= mem[A]; if mem[B] <= 0: goto C`), 65KB memory, bounded execution.
83
+ - **10-Point Verification Engine**: Type correctness, geometry, temporal state, Boolean stability, token round-trip, Latin vocabulary, Rosetta completeness, SUBLEQ halting, artifact provenance, template binding.
84
+ - **Agent Harness**: Agents propose artifacts; the system verifies before accepting. No agent output bypasses the FSM.
85
+ - **Dual Template Backend**: Mustache and Go template renderers, both hand-rolled with zero external dependencies.
86
+ - **113 Tests**: All round-trip invariants, all edge cases, all phases exercised.
87
+
88
+ ### Why This Matters
89
+
90
+ Semantic transformation is the substrate of every compiler, every code generator, every agent framework, and every formal verification tool. The typical approach is to treat NL→code as a black box (LLM prompt, template engine, or ad-hoc regex). This project makes every stage explicit, every invariant checked, and every intermediate representation inspectable:
91
+
92
+ - **Auditability**: Every transformation records provenance. Every hash chains forward. You can trace any output token back to the original input word.
93
+ - **Determinism**: Same input β†’ same IR β†’ same Boolean form β†’ same Latin β†’ same SUBLEQ β†’ same artifact. No randomness, no model variance, no hidden state.
94
+ - **Composability**: Each phase is a pure function gated by the FSM. Phases can be tested, replaced, or extended independently.
95
+ - **Formal Target**: SUBLEQ is Turing-complete with one instruction. If it compiles to SUBLEQ and the VM halts, the program is well-defined.
96
+
97
+ The pipeline processes natural language at the rate of ~300 inputs/second on a single core (113 tests in 0.38s), with zero external dependencies beyond Python 3.11.
98
+
99
+ ### Semantic IR: The Central Data Structure
100
+
101
+ The entire pipeline orbits a single typed intermediate representation. Every transformation reads from or writes to the `SemanticIR`:
102
+
103
+ ```python
104
+ @dataclass
105
+ class SemanticIR:
106
+ ir_id: str # UUID, unique per pipeline run
107
+ intent: IntentClass # BUILD, TRANSFORM, VALIDATE, ...
108
+ root: SemanticNode # root of the semantic tree
109
+ nodes: list[SemanticNode] # all nodes in the IR
110
+ temporal_lock: TemporalLock # audit log of Ο„ transitions
111
+ provenance: Provenance # source_input_id, stage, transform_hash
112
+ verification_state: VerificationState # UNVERIFIED β†’ VERIFIED β†’ REJECTED
113
+ output_mode: str # natural_language, code, artifact
114
+ metadata: dict[str, Any] # extensible
115
+ ```
116
+
117
+ Each `SemanticNode` carries:
118
+ - **Operation**: name, arity, commutativity, associativity
119
+ - **Operands**: typed, named, with optional geometry
120
+ - **Constraints**: predicate strings that become Boolean variables
121
+ - **Geometry**: row/col/shape for dimensional checking
122
+ - **Temporal state**: current Ο„ value
123
+ - **Provenance**: SHA-256 chain from input to current stage
124
+ - **Children**: sub-nodes for recursive structures
125
+
126
+ Every node, operation, constraint, and the IR itself produce deterministic SHA-256 hashes from their content. Two pipeline runs with the same input produce byte-identical hashes at every stage.
127
+
128
+ ### Provenance Chain
129
+
130
+ ```python
131
+ @dataclass(frozen=True)
132
+ class Provenance:
133
+ source_input_id: str # ties back to RawInput.input_id
134
+ stage: str # which pipeline phase
135
+ timestamp: float # wall clock
136
+ parent_id: Optional[str] # previous provenance (for chaining)
137
+ transform_hash: str # SHA-256 of the transformation applied
138
+ ```
139
+
140
+ The `chain_hash` method computes `SHA-256(transform_hash + ":" + data)`, extending the hash chain at each stage. This means any modification to any intermediate β€” even a single bit flip β€” produces a completely different hash at the artifact stage.
141
+
142
+ ---
143
+
144
+ ## Project Vision
145
+
146
+ We believe semantic transformation requires:
147
+
148
+ 1. **Explicit State Machines**: Not implicit control flow. Every legal transition is enumerated. Every illegal transition fails closed.
149
+ 2. **Temporal Accountability**: Every mutation advances Ο„. Read-only passes preserve Ο„. Locked states cannot be modified.
150
+ 3. **Dimensional Safety**: Matrix operations carry geometry. Contraction mismatches are caught before SUBLEQ assembly.
151
+ 4. **Controlled Vocabulary**: No synonym drift. Latin lexemes map 1:1 to semantic IDs. The Rosetta table is the single source of truth for machine translation.
152
+ 5. **Verified Emission**: No artifact leaves the pipeline without passing all 10 verification checks.
153
+
154
+ This project demonstrates that a complete NL→machine-code pipeline can be built from first principles: no frameworks, no LLM dependencies (BERT is optional), no external template engines, no runtime magic.
155
+
156
+ ---
157
+
158
+ ## Core Architecture
159
+
160
+ ```
161
+ semantic-engine/
162
+ β”œβ”€β”€ cli/main.py # Phase 18: 11 CLI subcommands
163
+ β”œβ”€β”€ src/semantic_engine/
164
+ β”‚ β”œβ”€β”€ __init__.py # Public API: SemanticPipeline, PipelineResult
165
+ β”‚ β”œβ”€β”€ types.py # Phase 1: Enums, error hierarchy, input types
166
+ β”‚ β”œβ”€β”€ ir.py # Phase 2: Typed IR β€” nodes, geometry, temporal, artifacts
167
+ β”‚ β”œβ”€β”€ fsm.py # Phase 3: 20-state FSM, typed transition table
168
+ β”‚ β”œβ”€β”€ temporal.py # Phase 4: Ο„-advancing transforms, violation detection
169
+ β”‚ β”œβ”€β”€ geometry.py # Phase 5: Matrix dimension checking, GeometryError
170
+ β”‚ β”œβ”€β”€ matrix.py # Phase 6: SemanticMatrix builder, compose/transform
171
+ β”‚ β”œβ”€β”€ boolean_engine.py # Phase 7: CNF normalization, De Morgan, simplification
172
+ β”‚ β”œβ”€β”€ token_engine.py # Phase 8: Binary encode/decode, round-trip invariant
173
+ β”‚ β”œβ”€β”€ latin.py # Phase 9: Controlled vocabulary, tokenβ†’lexeme map
174
+ β”‚ β”œβ”€β”€ lexer.py # Phase 10: Regex Latin lexer, fail-closed on unknowns
175
+ β”‚ β”œβ”€β”€ rosetta.py # Phase 11: Rosetta table, semantic_id β†’ SUBLEQ op
176
+ β”‚ β”œβ”€β”€ subleq.py # Phases 12–13: SUBLEQ VM, assembler, disassembler
177
+ β”‚ β”œβ”€β”€ verifier.py # Phase 14: 10 independent verification checks
178
+ β”‚ β”œβ”€β”€ artifacts.py # Phases 15–16: Mustache + Go template renderers
179
+ β”‚ β”œβ”€β”€ agent_harness.py # Phase 17: Agent proposal/verify/accept loop
180
+ β”‚ β”œβ”€β”€ nlp.py # NLP: tokenization, intent, entity, BERT interface
181
+ β”‚ └── pipeline.py # Full pipeline orchestrator
182
+ β”œβ”€β”€ lexicon/latin_vocabulary.json # 28 canonical Latin lexemes
183
+ β”œβ”€οΏ½οΏ½οΏ½ mappings/rosetta_table.json # 24 semantic_id β†’ SUBLEQ operation mappings
184
+ β”œβ”€β”€ templates/
185
+ β”‚ β”œβ”€β”€ prompt.mustache # Natural language artifact template
186
+ β”‚ β”œβ”€β”€ document.mustache # Document artifact template
187
+ β”‚ └── backend.go.tmpl # Go backend artifact template
188
+ β”œβ”€β”€ examples/
189
+ β”‚ β”œβ”€β”€ matrix_multiply.txt # "multiply two matrices A and B..."
190
+ β”‚ β”œβ”€β”€ parse_binary_format.txt # "build a parser for this binary format..."
191
+ β”‚ └── validate_packets.txt # "create a backend service that validates packets"
192
+ β”œβ”€β”€ tests/ # 113 tests across 7 test modules
193
+ β”œβ”€β”€ pyproject.toml # Python 3.11+, zero required dependencies
194
+ └── LICENSE # MPL-2.0
195
+ ```
196
+
197
+ **4,143 lines of source** (18 modules + CLI + 7 test files). Zero stubs. Zero `pass` placeholders.
198
+
199
+ ---
200
+
201
+ ## Pipeline Stages
202
+
203
+ | Phase | Module | Input | Output | Invariant |
204
+ |-------|--------|-------|--------|-----------|
205
+ | 1 | `types.py` | β€” | Enums, error types, input structs | Type safety via frozen dataclasses |
206
+ | 2 | `ir.py` | β€” | SemanticIR, SemanticNode, Artifact | Content-addressed (SHA-256) |
207
+ | 3 | `fsm.py` | `(state, input_class)` | `FSMTransition` | Fail-closed on missing transitions |
208
+ | 4 | `temporal.py` | `TemporalState(Ο„)` | `TemporalState(Ο„+1)` | Ο„ monotonic, locked states reject |
209
+ | 5 | `geometry.py` | `Geometry(mΓ—n)` | `DimCheck` | A.col == B.row for matmul |
210
+ | 6 | `matrix.py` | `SemanticNode` | `SemanticMatrix` | Composition: a.cols == b.rows |
211
+ | 7 | `boolean_engine.py` | Constraints | `BooleanNormalForm` (CNF) | normalize(normalize(x)) == normalize(x) |
212
+ | 8 | `token_engine.py` | Boolean IR | `BooleanTokenStream` / bytes | decode(encode(x)) == x |
213
+ | 9 | `latin.py` | Token stream | `LatinCanonicalForm` | Every token maps to exactly one Latin lexeme |
214
+ | 10 | `lexer.py` | Latin text | `LexResult` | Unknown tokens fail closed |
215
+ | 11 | `rosetta.py` | Latin form | `RosettaMapping` | All semantic_ids must map (no unmapped) |
216
+ | 12–13 | `subleq.py` | Rosetta mapping | `SubleqProgram` / VM trace | Program halts within 10K steps |
217
+ | 14 | `verifier.py` | All intermediates | `VerificationReport` | 10 independent checks, UNVERIFIED→VERIFIED→REJECTED |
218
+ | 15–16 | `artifacts.py` | SemanticIR + template | `Artifact` | Mustache + Go renderers, provenance chain intact |
219
+ | 17 | `agent_harness.py` | `AgentTask` | Verified `Artifact` | Agent proposals verified before acceptance |
220
+ | 18 | `cli/main.py` | File path | Formatted output | 11 subcommands + trace flags |
221
+ | 19–20 | `tests/` | β€” | 113 assertions | All round-trip invariants exercised |
222
+
223
+ ---
224
+
225
+ ## Finite State Machine
226
+
227
+ The FSM (`fsm.py`, 128 lines) defines exactly 21 transitions across 20 states plus an ERROR sink. Every transition is a typed record:
228
+
229
+ ```python
230
+ @dataclass(frozen=True)
231
+ class FSMTransition:
232
+ source_state: FSMState # where we are
233
+ input_class: str # what arrived
234
+ operation: str # what to do
235
+ constraints: tuple[str, ...] # what must hold
236
+ destination_state: FSMState # where we go
237
+ invariant: str # human-readable guarantee
238
+ ```
239
+
240
+ ### Complete Transition Table
241
+
242
+ | Source State | Input Class | Operation | Destination | Constraint |
243
+ |-------------|-------------|-----------|-------------|------------|
244
+ | INPUT | raw_input | normalize | NORMALIZE | text_not_empty |
245
+ | NORMALIZE | normalized_input | lex | LEXICAL_ANALYSIS | tokens_non_empty |
246
+ | LEXICAL_ANALYSIS | token_stream | detect_intent | INTENT_DETECTION | tokens_recognized |
247
+ | INTENT_DETECTION | intent_class | bind_entities | ENTITY_BINDING | intent_not_unknown |
248
+ | ENTITY_BINDING | entities | bind_semantics | SEMANTIC_BINDING | entities_typed |
249
+ | SEMANTIC_BINDING | semantic_candidate | lock_temporal | TEMPORAL_LOCK | candidate_valid |
250
+ | TEMPORAL_LOCK | temporal_state | bind_geometry | GEOMETRY_BINDING | tau_monotonic |
251
+ | GEOMETRY_BINDING | geometry | build_matrix | MATRIX_FORM | geometry_consistent |
252
+ | MATRIX_FORM | semantic_matrix | resolve_boolean | BOOLEAN_ROOT | matrix_well_formed |
253
+ | BOOLEAN_ROOT | boolean_expr | normalize_boolean | BOOLEAN_NORMALIZATION | boolean_valid |
254
+ | BOOLEAN_NORMALIZATION | normalized_boolean | tokenize | TOKENIZATION | normal_form_stable |
255
+ | TOKENIZATION | token_stream | canonicalize_latin | LATIN_CANONICALIZATION | tokens_round_trip |
256
+ | LATIN_CANONICALIZATION | latin_form | lex_latin | LATIN_LEXING | vocabulary_closed |
257
+ | LATIN_LEXING | latin_tokens | map_rosetta | ROSETTA_MAPPING | latin_tokens_valid |
258
+ | ROSETTA_MAPPING | rosetta_mapping | build_subleq | SUBLEQ_IR | mapping_complete |
259
+ | SUBLEQ_IR | subleq_program | verify | VERIFICATION | subleq_valid |
260
+ | VERIFICATION | verified_ir | select_artifact | ARTIFACT_SELECTION | all_checks_pass |
261
+ | ARTIFACT_SELECTION | artifact_type | emit | EMISSION | template_bound |
262
+ | EMISSION | artifact | complete | COMPLETE | artifact_provenance_ok |
263
+ | COMPLETE | done | noop | COMPLETE | (terminal) |
264
+ | INPUT | invalid | fail | ERROR | error_logged |
265
+
266
+ The transition map is indexed by `(source_state, input_class)`. Any `(state, input)` pair not in the table triggers fail-closed behavior: the FSM transitions to ERROR and raises `PipelineError`.
267
+
268
+ ### Constraint Validators
269
+
270
+ Each constraint in the transition table maps to a validator function:
271
+
272
+ ```python
273
+ "text_not_empty": lambda ctx: bool(str(ctx.get("raw_text")).strip())
274
+ "tokens_non_empty": lambda ctx: bool(ctx.get("tokens"))
275
+ "intent_not_unknown": lambda ctx: ctx.get("intent") != "unknown"
276
+ "all_checks_pass": lambda ctx: ctx.get("verified") is True
277
+ "template_bound": lambda ctx: ctx.get("template") is not None
278
+ "artifact_provenance_ok": lambda ctx: ctx.get("artifact") is not None
279
+ ```
280
+
281
+ Constraints that are enforced by exception (e.g., geometry, temporal) use pass-through validators β€” the corresponding module raises before the validator runs.
282
+
283
+ ### FSM Context
284
+
285
+ The `FSMContext` carries state, history, data bag, and error:
286
+
287
+ ```python
288
+ ctx = FSMContext() # state=INPUT
289
+ ctx.transition("raw_input") # β†’ NORMALIZE
290
+ ctx.transition("normalized_input") # β†’ LEXICAL_ANALYSIS
291
+ # ...19 transitions later...
292
+ ctx.transition("artifact") # β†’ COMPLETE
293
+ assert ctx.is_terminal() # True
294
+
295
+ # Full audit trail
296
+ for line in ctx.dump_history():
297
+ print(line)
298
+ # INPUT --[raw_input]--> NORMALIZE
299
+ # NORMALIZE --[normalized_input]--> LEXICAL_ANALYSIS
300
+ # ...
301
+ ```
302
+
303
+ ---
304
+
305
+ ## Pipeline Flowchart
306
+
307
+ ```
308
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
309
+ β”‚ Semantic Engine Pipeline β”‚
310
+ β”‚ NL Input β†’ Semantic IR β†’ Boolean β†’ Latin β†’ SUBLEQ β†’ Artifact β”‚
311
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
312
+
313
+ START
314
+ β”‚
315
+ β–Ό
316
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
317
+ β”‚ RawInput(text, source, id) β”‚
318
+ β”‚ SHA-256 hash computed β”‚
319
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
320
+ β”‚
321
+ β–Ό
322
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
323
+ β”‚ Phase 1–2: NORMALIZE β”‚
324
+ β”‚ Tokenize β†’ segment sentences β”‚
325
+ β”‚ Rule-based + optional BERT β”‚
326
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
327
+ β”‚
328
+ β–Ό
329
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
330
+ β”‚ Phase 3–6: SEMANTIC BINDING β”‚
331
+ β”‚ Intent β†’ Entities β†’ Constraintsβ”‚
332
+ β”‚ Temporal lock (Ο„β‚€ β†’ τ₁) β”‚
333
+ β”‚ Geometry enforcement β”‚
334
+ β”‚ SemanticMatrix construction β”‚
335
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
336
+ β”‚
337
+ β–Ό
338
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
339
+ β”‚ Phase 7: BOOLEAN ENGINE β”‚
340
+ β”‚ Constraints β†’ CNF expression β”‚
341
+ β”‚ De Morgan + distribute OR/AND β”‚
342
+ β”‚ Idempotency: f(f(x)) = f(x) β”‚
343
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
344
+ β”‚
345
+ β–Ό
346
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
347
+ β”‚ Phase 8: TOKEN ENGINE β”‚
348
+ β”‚ Boolean IR β†’ binary tokens β”‚
349
+ β”‚ Magic: BOOL | Version: 1 β”‚
350
+ β”‚ Round-trip: decode∘encode = id β”‚
351
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
352
+ β”‚
353
+ β–Ό
354
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
355
+ β”‚ Phase 9–10: LATIN LAYER β”‚
356
+ β”‚ Tokens β†’ Latin lexemes β”‚
357
+ β”‚ ET VEL NON AUT SI VERUM FALSUM β”‚
358
+ β”‚ Regex lexer, fail-closed β”‚
359
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
360
+ β”‚
361
+ β–Ό
362
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
363
+ β”‚ Phase 11: ROSETTA MAPPING β”‚
364
+ β”‚ semantic_id β†’ SUBLEQ op β”‚
365
+ β”‚ 24 entries, all must map β”‚
366
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
367
+ β”‚
368
+ β–Ό
369
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
370
+ β”‚ Phases 12–13: SUBLEQ β”‚
371
+ β”‚ Assemble β†’ flat binary β”‚
372
+ β”‚ VM: 65KB memory, bounded exec β”‚
373
+ β”‚ SUBLEQ(A,B,C): β”‚
374
+ β”‚ mem[B] -= mem[A] β”‚
375
+ β”‚ if mem[B] ≀ 0: goto C β”‚
376
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
377
+ β”‚
378
+ β–Ό
379
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
380
+ β”‚ Phase 14: VERIFICATION β”‚
381
+ β”‚ 10 independent checks β”‚
382
+ β”‚ UNVERIFIED β†’ VERIFIED β”‚
383
+ β”‚ or β†’ REJECTED (fail closed) β”‚
384
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
385
+ β”‚
386
+ β–Ό
387
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
388
+ β”‚ Phases 15–17: ARTIFACT β”‚
389
+ β”‚ Template selection + rendering β”‚
390
+ β”‚ Mustache or Go template β”‚
391
+ β”‚ Agent harness (if applicable) β”‚
392
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
393
+ β”‚
394
+ β–Ό
395
+ COMPLETE
396
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
397
+ β”‚ PipelineResult β”‚
398
+ β”‚ .ir β”‚
399
+ β”‚ .artifact β”‚
400
+ β”‚ .subleq_listing β”‚
401
+ β”‚ .trace[] β”‚
402
+ β”‚ .hashes{} β”‚
403
+ β”‚ .ok β”‚
404
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
405
+ ```
406
+
407
+ ---
408
+
409
+ ## Verification Framework
410
+
411
+ The Verifier runs 10 independent checks. All must pass before any artifact is emitted.
412
+
413
+ | # | Check | What It Verifies | Failure Mode |
414
+ |---|-------|------------------|--------------|
415
+ | 1 | `type_correctness` | Operation arity matches operand count | `VerificationError` |
416
+ | 2 | `geometry` | Matrix contraction dimensions (A.col == B.row) | `GeometryError` |
417
+ | 3 | `temporal_state` | All Ο„ transitions are +1 or +0 (no jumps) | `TemporalViolation` |
418
+ | 4 | `boolean_normalization` | `normalize(normalize(x)) == normalize(x)` | `VerificationError` |
419
+ | 5 | `token_round_trip` | `decode(encode(stream)) == stream` | `TokenError` |
420
+ | 6 | `latin_vocabulary` | Every lexeme exists in the controlled vocabulary | `PipelineError` |
421
+ | 7 | `rosetta_mapping` | All semantic_ids have SUBLEQ mappings | `RosettaError` |
422
+ | 8 | `subleq_validity` | Program halts within 10,000 steps | `PipelineError` |
423
+ | 9 | `artifact_provenance` | Artifact.semantic_root == IR.ir_id | `VerificationError` |
424
+ | 10 | `template_bindings` | Template is bound and content is non-empty | `VerificationError` |
425
+
426
+ ### Verification State Machine
427
+
428
+ ```
429
+ UNVERIFIED
430
+ β”‚
431
+ β–Ό
432
+ VALIDATING ──── checks running
433
+ / \
434
+ β–Ό β–Ό
435
+ VERIFIED REJECTED
436
+ β”‚
437
+ β–Ό
438
+ Artifact emitted
439
+ ```
440
+
441
+ ### Error Hierarchy
442
+
443
+ ```
444
+ PipelineError(stage, message, input_ref)
445
+ β”œβ”€β”€ GeometryError β€” dimension mismatch
446
+ β”œβ”€β”€ TemporalViolation β€” illegal Ο„ mutation
447
+ β”œβ”€β”€ VerificationError β€” check failed
448
+ β”œβ”€β”€ TokenError β€” binary codec failure
449
+ └── RosettaError β€” unmapped semantic_id
450
+ ```
451
+
452
+ Every error carries the FSMState where the failure occurred, enabling precise diagnostics.
453
+
454
+ ---
455
+
456
+ ## Boolean Engine & CNF Normalization
457
+
458
+ The Boolean engine converts semantic constraints into Conjunctive Normal Form through a deterministic pipeline:
459
+
460
+ ### Expression Types
461
+
462
+ | Type | Representation | Example |
463
+ |------|---------------|---------|
464
+ | `BooleanLiteral` | `⊀` / `βŠ₯` | True, False |
465
+ | `BooleanVar` | `x` / `Β¬x` | Variable with optional negation |
466
+ | `BooleanExpression` | `(x ∧ y)` | Tree of operator + operands |
467
+
468
+ ### Operators
469
+
470
+ | Operator | Symbol | Latin Lexeme | SUBLEQ Op |
471
+ |----------|--------|-------------|-----------|
472
+ | AND | ∧ | ET | AND |
473
+ | OR | ∨ | VEL | OR |
474
+ | NOT | Β¬ | NON | NOT |
475
+ | XOR | βŠ• | AUT | XOR |
476
+ | IMPLIES | β†’ | SI | OR(Β¬a, b) |
477
+ | EQUIVALENCE | ↔ | IFF | XNOR |
478
+ | TRUE | ⊀ | VERUM | LOAD_1 |
479
+ | FALSE | βŠ₯ | FALSUM | LOAD_0 |
480
+
481
+ ### Normalization Pipeline
482
+
483
+ 1. **Simplify**: Constant folding, double-negation elimination (`¬¬A β†’ A`), identity removal
484
+ 2. **Push NOT inward**: De Morgan's laws (`Β¬(A ∧ B) β†’ Β¬A ∨ Β¬B`, `Β¬(A ∨ B) β†’ Β¬A ∧ Β¬B`)
485
+ 3. **Distribute OR over AND**: Produce CNF (`A ∨ (B ∧ C) β†’ (A ∨ B) ∧ (A ∨ C)`)
486
+ 4. **Simplify again**: Clean up redundant terms
487
+
488
+ ### Rewrite Rules
489
+
490
+ ```
491
+ A β†’ B ≑ Β¬A ∨ B
492
+ A ↔ B ≑ (A β†’ B) ∧ (B β†’ A)
493
+ A βŠ• B ≑ (A ∨ B) ∧ Β¬(A ∧ B)
494
+ Β¬(A ∧ B) ≑ Β¬A ∨ Β¬B (De Morgan)
495
+ Β¬(A ∨ B) ≑ Β¬A ∧ Β¬B (De Morgan)
496
+ ¬¬A ≑ A (Double negation)
497
+ A ∧ βŠ₯ ≑ βŠ₯ (Annihilation)
498
+ A ∨ ⊀ ≑ ⊀ (Annihilation)
499
+ A ∧ ⊀ ≑ A (Identity)
500
+ A ∨ βŠ₯ ≑ A (Identity)
501
+ ```
502
+
503
+ ### Idempotency Guarantee
504
+
505
+ ```python
506
+ bnf = normalize(expr)
507
+ assert repr(normalize(bnf)) == repr(bnf) # Always holds
508
+ ```
509
+
510
+ ---
511
+
512
+ ## Token Engine & Binary Serialization
513
+
514
+ The token engine serializes Boolean IR into a compact binary format and reconstructs it losslessly.
515
+
516
+ ### Binary Format
517
+
518
+ ```
519
+ Offset Size Field
520
+ ────── ──── ─────
521
+ 0 4 Magic: "BOOL" (0x424F4F4C)
522
+ 4 1 Version: 0x01
523
+ 5 2 Token count (big-endian uint16)
524
+ 7+ var Token records:
525
+ 1 byte token_type
526
+ 1 byte arity
527
+ 2 bytes payload_length (big-endian uint16)
528
+ N bytes payload
529
+ ```
530
+
531
+ ### Token Type Constants
532
+
533
+ | ID | Name | Meaning |
534
+ |----|------|---------|
535
+ | `0x01` | LITERAL | Boolean literal (payload: `\x01`=true, `\x00`=false) |
536
+ | `0x02` | VAR | Variable (payload: UTF-8 name) |
537
+ | `0x03` | VAR_NEG | Negated variable |
538
+ | `0x10` | AND | Conjunction |
539
+ | `0x11` | OR | Disjunction |
540
+ | `0x12` | NOT | Negation |
541
+ | `0x13` | XOR | Exclusive or |
542
+ | `0x14` | IMPLIES | Material conditional |
543
+ | `0x15` | EQUIVALENCE | Biconditional |
544
+ | `0x20` | TRUE | Verum constant |
545
+ | `0x21` | FALSE | Falsum constant |
546
+ | `0x30` | OPEN | Expression open |
547
+ | `0x31` | CLOSE | Expression close |
548
+ | `0x40` | ARITY | Arity marker |
549
+
550
+ ### Round-Trip Invariant
551
+
552
+ ```python
553
+ from semantic_engine.token_engine import encode, decode, round_trip
554
+
555
+ expr = AND(var("x"), OR(var("y"), NOT(var("z"))))
556
+ assert repr(round_trip(expr)) == repr(expr) # Always holds
557
+ ```
558
+
559
+ This invariant is checked by the verifier at Phase 14 β€” no artifact can emit if the round-trip fails.
560
+
561
+ ---
562
+
563
+ ## Latin Canonical Layer
564
+
565
+ The Latin layer provides a controlled vocabulary between Boolean tokens and SUBLEQ machine operations. Every token type maps to exactly one Latin lexeme. No uncontrolled synonyms exist.
566
+
567
+ ### Controlled Vocabulary (28 lexemes)
568
+
569
+ | Latin | Semantic ID | Operator | Arity |
570
+ |-------|------------|----------|-------|
571
+ | ET | BOOL_AND | AND | 2 |
572
+ | VEL | BOOL_OR | OR | 2 |
573
+ | NON | BOOL_NOT | NOT | 1 |
574
+ | AUT | BOOL_XOR | XOR | 2 |
575
+ | SI | BOOL_IMPLIES | IMPLIES | 2 |
576
+ | IFF | BOOL_EQUIV | EQUIVALENCE | 2 |
577
+ | VERUM | BOOL_TRUE | TRUE | 0 |
578
+ | FALSUM | BOOL_FALSE | FALSE | 0 |
579
+ | INITIUM | DELIM_OPEN | β€” | 0 |
580
+ | FINIS | DELIM_CLOSE | β€” | 0 |
581
+ | OPUS | SEMANTIC_OP | β€” | var |
582
+ | FORMA | SEMANTIC_MATRIX | β€” | var |
583
+ | TEMPUS | TEMPORAL_STATE | β€” | 0 |
584
+ | COMPUTA | OP_COMPUTE | compute | 1 |
585
+ | MULTIPLICA | OP_MULTIPLY | multiply | 2 |
586
+ | ADDE | OP_ADD | add | 2 |
587
+ | SUBTRAHE | OP_SUBTRACT | subtract | 2 |
588
+ | PARSE | OP_PARSE | parse | 1 |
589
+ | AEDIFICA | OP_BUILD | build | 1 |
590
+ | GENERA | OP_GENERATE | generate | 1 |
591
+ | VERIFICA | OP_VALIDATE | validate | 1 |
592
+ | CONVERTE | OP_TRANSFORM | transform | 2 |
593
+ | RESTITUE | OP_RECONSTRUCT | reconstruct | 1 |
594
+ | MATRIX | ENTITY_MATRIX | β€” | 0 |
595
+ | VECTOR | ENTITY_VECTOR | β€” | 0 |
596
+ | PARSER | ENTITY_PARSER | β€” | 0 |
597
+ | SERVITIUM | ENTITY_SERVICE | β€” | 0 |
598
+ | FORMA_BINARIA | ENTITY_BINARY_FORMAT | β€” | 0 |
599
+
600
+ ### Example Canonical Form
601
+
602
+ Input: `"build a parser for this binary format deterministically"`
603
+
604
+ Latin output: `INITIUM ET OPUS[deterministic] FINIS`
605
+
606
+ The Latin lexer (`lexer.py`) validates the output with regex patterns and rejects any token not in the controlled vocabulary.
607
+
608
+ ---
609
+
610
+ ## SUBLEQ IR & Virtual Machine
611
+
612
+ SUBLEQ is a one-instruction computer: `SUBLEQ A, B, C` means `mem[B] -= mem[A]; if mem[B] <= 0: goto C`.
613
+
614
+ ### Machine Specification
615
+
616
+ | Parameter | Value |
617
+ |-----------|-------|
618
+ | Memory | 65,536 cells (64KB) |
619
+ | Word size | 16-bit signed integer |
620
+ | Instruction format | 3 words: A, B, C |
621
+ | Binary encoding | Big-endian, 6 bytes per instruction |
622
+ | Halt condition | Branch to address -1 |
623
+ | Built-in addresses | 0 (zero), 1 (one), 2 (temp) |
624
+ | Program base | Address 100 |
625
+ | Data allocation | Bump allocator from address 256+ |
626
+ | Max steps | 100,000 (configurable) |
627
+
628
+ ### Rosetta β†’ SUBLEQ Assembly
629
+
630
+ Each Rosetta entry maps to one or more SUBLEQ instructions:
631
+
632
+ ```
633
+ BOOL_AND β†’ SUBLEQ 0, out, next ; AND: allocate + clear
634
+ BOOL_NOT β†’ SUBLEQ a, 1, next2 ; NOT: subtract from 1
635
+ SUBLEQ 0, out, next ; store result
636
+ TICK β†’ SUBLEQ 0, out, next ; temporal marker
637
+ NOP β†’ SUBLEQ 0, 0, next ; no-op passthrough
638
+ HALT β†’ SUBLEQ 0, 0, -1 ; terminal
639
+ ```
640
+
641
+ ### VM Trace
642
+
643
+ Every instruction execution records a `VMTrace`:
644
+
645
+ ```python
646
+ @dataclass
647
+ class VMTrace:
648
+ step: int # execution step counter
649
+ pc: int # program counter
650
+ A: int # source address
651
+ B: int # destination address
652
+ C: int # branch address
653
+ mem_b_before: int # mem[B] before subtraction
654
+ mem_b_after: int # mem[B] after subtraction
655
+ branched: bool # whether branch was taken
656
+ ```
657
+
658
+ ### SUBLEQ VM Walk-Through
659
+
660
+ Consider a simple assembled program with 3 instructions:
661
+
662
+ ```
663
+ Address Instruction Comment
664
+ ────── ────────────── ───────
665
+ 100 SUBLEQ 0, 256, 109 ; AND: clear cell 256
666
+ 106 SUBLEQ 0, 0, 115 ; NOP: passthrough
667
+ 112 SUBLEQ 0, 0, -1 ; HALT
668
+ ```
669
+
670
+ **Execution trace:**
671
+
672
+ | Step | PC | A | B | C | mem[B] before | mem[B] after | Branch? |
673
+ |------|----|---|---|---|---------------|-------------|---------|
674
+ | 0 | 100 | 0 | 256 | 109 | 0 | 0 (0-0) | YES (0≀0) β†’ PC=109 |
675
+ | 1 | 106 | 0 | 0 | 115 | 0 | 0 (0-0) | YES (0≀0) β†’ PC=115 |
676
+ | 2 | 112 | 0 | 0 | -1 | 0 | 0 (0-0) | YES (0≀0) β†’ HALT (C=-1) |
677
+
678
+ The program halts in 3 steps. The verifier confirms halting and sets `subleq_validity: PASSED`.
679
+
680
+ ### Assemble / Disassemble Round-Trip
681
+
682
+ ```python
683
+ prog = SubleqProgram()
684
+ prog.add(SubleqInstruction(A=0, B=0, C=-1, comment="HALT"))
685
+
686
+ # Assemble to binary: 6 bytes (3 Γ— int16 big-endian)
687
+ binary = prog.assemble()
688
+ assert binary == b'\x00\x00\x00\x00\xff\xff'
689
+
690
+ # Disassemble back
691
+ prog2 = SubleqProgram.disassemble(binary)
692
+ assert prog2.instructions[0].A == 0
693
+ assert prog2.instructions[0].C == -1
694
+ ```
695
+
696
+ ### Address Allocation
697
+
698
+ The assembler uses a bump allocator starting at address 256 (0–255 reserved for VM built-ins). Each SUBLEQ operation that needs scratch space calls `_alloc(n)` to reserve `n` contiguous cells. This ensures no two operations share memory, preventing aliasing bugs.
699
+
700
+ ### Halting Verification
701
+
702
+ The verifier runs the SUBLEQ program in a sandboxed VM (separate memory instance, max 10K steps). If the program does not halt, verification fails and the artifact is rejected. The sandboxed VM is a completely separate `SubleqMemory` instance β€” it cannot affect the pipeline's state.
703
+
704
+ ---
705
+
706
+ ## Verification Flowchart
707
+
708
+ ```
709
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
710
+ β”‚ Verification Engine β”‚
711
+ β”‚ 10 Independent Checks β€” All Must Pass β”‚
712
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
713
+
714
+ SemanticIR
715
+ β”‚
716
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
717
+ β”‚ β”‚ β”‚
718
+ β–Ό β–Ό β–Ό
719
+ Type Geometry Temporal
720
+ Check Check Check
721
+ β”‚ β”‚ β”‚
722
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
723
+ β”‚
724
+ BooleanNormalForm
725
+ β”‚
726
+ β–Ό
727
+ Normalization
728
+ Idempotency Check
729
+ β”‚
730
+ β–Ό
731
+ BooleanTokenStream
732
+ β”‚
733
+ β–Ό
734
+ Round-Trip Check
735
+ encode β†’ decode β†’ compare
736
+ β”‚
737
+ β–Ό
738
+ LatinCanonicalForm
739
+ β”‚
740
+ β–Ό
741
+ Vocabulary Check
742
+ all lexemes in vocab?
743
+ β”‚
744
+ β–Ό
745
+ RosettaMapping
746
+ β”‚
747
+ β–Ό
748
+ Completeness Check
749
+ all semantic_ids mapped?
750
+ β”‚
751
+ β–Ό
752
+ SubleqProgram
753
+ β”‚
754
+ β–Ό
755
+ Halting Check
756
+ VM executes < 10K steps?
757
+ β”‚
758
+ β–Ό
759
+ Artifact
760
+ / \
761
+ β–Ό β–Ό
762
+ Provenance Template
763
+ Check Binding Check
764
+ \ /
765
+ β–Ό β–Ό
766
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
767
+ β”‚ All 10 passed? β”‚
768
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
769
+ YES β”‚ β”‚ NO
770
+ β–Ό β–Ό
771
+ VERIFIED REJECTED
772
+ β”‚
773
+ β–Ό
774
+ Artifact emitted
775
+ ```
776
+
777
+ ---
778
+
779
+ ## Semantic Matrix Layer
780
+
781
+ The matrix layer (`matrix.py`, 172 lines) represents semantic operations as structured matrix forms. A `SemanticMatrix` is not a numerical matrix β€” it captures the structure of an operation (operand slots, geometry, relationships) in a matrix-like layout.
782
+
783
+ ### Matrix Layouts by Operation Type
784
+
785
+ **Matrix multiply** (`matmul`): 3Γ—3 layout
786
+ ```
787
+ [A_input] [contraction] [B_input]
788
+ [m_dim] [n_dim] [p_dim]
789
+ [output_C] [(mΓ—p)] [β€”]
790
+ ```
791
+
792
+ **Binary operation** (`add`, `subtract`): 2Γ—2 layout
793
+ ```
794
+ [A_input] [B_input]
795
+ [result] [β€”]
796
+ ```
797
+
798
+ **Generic operation**: Nx2 layout (one row per operand)
799
+ ```
800
+ [operand_1_input] [output]
801
+ [operand_2_input] [output]
802
+ [...] [...]
803
+ ```
804
+
805
+ ### SemanticCell
806
+
807
+ Each cell in the matrix carries:
808
+ ```python
809
+ @dataclass
810
+ class SemanticCell:
811
+ row: int # position
812
+ col: int # position
813
+ operand: Optional[Operand] # bound operand (if any)
814
+ label: str # display name
815
+ role: str # "input", "output", "contraction", "constraint", "dim"
816
+ ```
817
+
818
+ ### Matrix Composition
819
+
820
+ Two semantic matrices can be composed (analogous to function composition):
821
+
822
+ ```python
823
+ composed = transform(matrix_a, matrix_b)
824
+ # Requires: a.cols == b.rows
825
+ # Output: SemanticMatrix(rows=a.rows, cols=b.cols)
826
+ # Operation name: "multiply∘validate"
827
+ ```
828
+
829
+ If `a.cols != b.rows`, the composition raises `GeometryError`.
830
+
831
+ ---
832
+
833
+ ## Artifact Templates
834
+
835
+ Phases 15–16 (`artifacts.py`, 237 lines) provide two hand-rolled template renderers β€” no external dependencies.
836
+
837
+ ### Mustache Renderer
838
+
839
+ Supports:
840
+ - `{{variable}}` β€” simple variable substitution
841
+ - `{{#section}} ... {{/section}}` β€” truthy sections and list iteration
842
+ - `{{^section}} ... {{/section}}` β€” inverted (falsy) sections
843
+
844
+ Example template (`templates/prompt.mustache`):
845
+ ```mustache
846
+ {{title}}
847
+
848
+ {{semantic_description}}
849
+
850
+ {{#constraints}}
851
+ - Constraint: {{.}}
852
+ {{/constraints}}
853
+
854
+ {{#operations}}
855
+ - Operation: {{.}}
856
+ {{/operations}}
857
+ ```
858
+
859
+ ### Go Template Renderer
860
+
861
+ Supports a subset of Go's `text/template`:
862
+ - `{{.Field}}` β€” dot-field access
863
+ - `{{range .Items}} ... {{end}}` β€” list iteration
864
+ - `{{if .Cond}} ... {{end}}` β€” conditional sections
865
+
866
+ Example template (`templates/backend.go.tmpl`):
867
+ ```go
868
+ package {{.Package}}
869
+
870
+ {{range .Operations}}
871
+ func {{.Name}}({{range .Params}}{{.Name}} {{.Type}}, {{end}}) (interface{}, error) {
872
+ {{.Body}}
873
+ }
874
+ {{end}}
875
+ ```
876
+
877
+ ### Artifact Types
878
+
879
+ | Type | Template | Use Case |
880
+ |------|----------|----------|
881
+ | PROMPT | prompt.mustache | LLM-ready natural language |
882
+ | DOCUMENT | document.mustache | Human-readable documentation |
883
+ | BACKEND | backend.go.tmpl | Go source code skeleton |
884
+ | SOURCE | β€” | Generic source output |
885
+ | CONFIG | β€” | Configuration files |
886
+ | TEST | β€” | Test specifications |
887
+ | SCHEMA | β€” | JSON/protocol schemas |
888
+ | REPORT | β€” | Verification reports |
889
+
890
+ ---
891
+
892
+ ## Agent Harness
893
+
894
+ Phase 17 defines a verification-first agent interface. Agents are functions that receive an `AgentTask` and return an `AgentProposal`. The harness verifies every proposal against the SemanticIR before acceptance.
895
+
896
+ ### Agent Flow
897
+
898
+ 1. **Register**: `harness.register(Agent(id, fn, capabilities))`
899
+ 2. **Dispatch**: `harness.dispatch(task, agent_id)` β†’ calls agent function
900
+ 3. **Verify**: Proposal's artifact is verified against task's IR reference
901
+ 4. **Accept/Reject**: Verified artifacts are accepted; failures are logged and raise `PipelineError`
902
+
903
+ ### Key Constraint
904
+
905
+ Agent output never bypasses the FSM, IR validation, temporal lock, or verification engine. The agent is a proposal mechanism β€” the pipeline is the acceptance mechanism.
906
+
907
+ ---
908
+
909
+ ## CLI Reference
910
+
911
+ ```
912
+ semantic-engine <command> <input> [--json] [--trace] [--subleq] [--tokens] [--provenance]
913
+ ```
914
+
915
+ ### Subcommands
916
+
917
+ | Command | Description |
918
+ |---------|-------------|
919
+ | `parse <file>` | Tokenize and segment input |
920
+ | `inspect <file>` | Build and inspect the Semantic IR |
921
+ | `normalize <file>` | Show Boolean CNF normalization |
922
+ | `boolean <file>` | Show Boolean IR expression |
923
+ | `tokens <file>` | Show Boolean token stream |
924
+ | `latin <file>` | Show Latin canonical form |
925
+ | `subleq <file>` | Show SUBLEQ IR listing |
926
+ | `verify <file>` | Run full verification, show results |
927
+ | `artifact <file>` | Emit natural-language artifact |
928
+ | `backend <file>` | Emit Go backend source artifact |
929
+ | `run <file>` | Full pipeline with optional trace flags |
930
+
931
+ ### Examples
932
+
933
+ ```bash
934
+ # Parse and tokenize
935
+ semantic-engine parse examples/matrix_multiply.txt --json
936
+
937
+ # Full pipeline with trace
938
+ semantic-engine run examples/matrix_multiply.txt --trace --subleq --tokens --provenance
939
+
940
+ # Emit Go backend
941
+ semantic-engine backend examples/validate_packets.txt
942
+
943
+ # Verify input
944
+ semantic-engine verify examples/parse_binary_format.txt
945
+ ```
946
+
947
+ ### Exit Codes
948
+
949
+ | Code | Meaning |
950
+ |------|---------|
951
+ | 0 | Success (VERIFIED) |
952
+ | 1 | Pipeline error or REJECTED |
953
+
954
+ ---
955
+
956
+ ## Building & Installation
957
+
958
+ ### Requirements
959
+
960
+ - **Python 3.11+** (no external dependencies required)
961
+ - **Optional**: `transformers` + `torch` for BERT-based intent classification (falls back to rule-based without them)
962
+
963
+ ### Install
964
+
965
+ ```bash
966
+ # Clone
967
+ git clone https://github.com/SNAPKITTYWEST/semantic-engine.git
968
+ cd semantic-engine
969
+
970
+ # Install (editable)
971
+ pip install -e .
972
+
973
+ # Or install with BERT support
974
+ pip install -e ".[bert]"
975
+
976
+ # Or install with dev tools
977
+ pip install -e ".[dev]"
978
+ ```
979
+
980
+ ### Run from Source
981
+
982
+ ```bash
983
+ # Direct execution
984
+ python cli/main.py run examples/matrix_multiply.txt --trace
985
+
986
+ # Or via installed entry point
987
+ semantic-engine run examples/matrix_multiply.txt --trace
988
+ ```
989
+
990
+ ### Quick Start
991
+
992
+ ```bash
993
+ # Clone and test
994
+ git clone https://github.com/SNAPKITTYWEST/semantic-engine.git
995
+ cd semantic-engine
996
+ python -m pytest tests/ -q
997
+
998
+ # Expected output:
999
+ # 113 passed in 0.38s
1000
+
1001
+ # Run the pipeline
1002
+ echo "multiply two matrices A and B where A is 3x4 and B is 4x5" > /tmp/input.txt
1003
+ python cli/main.py run /tmp/input.txt --trace --subleq
1004
+ ```
1005
+
1006
+ ---
1007
+
1008
+ ## Testing & Validation
1009
+
1010
+ ### Test Modules
1011
+
1012
+ | Module | Tests | What It Covers |
1013
+ |--------|-------|---------------|
1014
+ | `test_types.py` | 9 | Enums, RawInput hashing, NormalizedInput, error hierarchy |
1015
+ | `test_ir.py` | 17 | SemanticNode, SemanticIR, Geometry compatibility, TemporalLock |
1016
+ | `test_fsm.py` | 11 | Full state traversal, fail-closed transitions, constraint validation |
1017
+ | `test_boolean.py` | 18 | CNF normalization, De Morgan, constant folding, idempotency |
1018
+ | `test_tokens.py` | 13 | Binary encode/decode, round-trip, malformed input rejection |
1019
+ | `test_subleq.py` | 15 | VM execution, halting, assemble/disassemble, Rosetta→SUBLEQ |
1020
+ | `test_roundtrip.py` | 30 | End-to-end pipeline, all three examples, trace validation |
1021
+
1022
+ **Total: 113 tests, 0.38s**
1023
+
1024
+ ### Invariants Under Test
1025
+
1026
+ - **Token round-trip**: `decode(encode(x)) == x` for all Boolean expression shapes
1027
+ - **Boolean idempotency**: `normalize(normalize(x)) == normalize(x)`
1028
+ - **Temporal monotonicity**: Ο„ advances by exactly 0 or 1 per transition
1029
+ - **SUBLEQ halting**: Every assembled program halts within step limit
1030
+ - **Provenance chain**: Artifact.semantic_root == IR.ir_id
1031
+ - **FSM completeness**: All 20 states reachable; invalid transitions fail closed
1032
+ - **Latin vocabulary closure**: No lexeme outside the controlled vocabulary passes the lexer
1033
+ - **Geometry consistency**: Mismatched matrix dimensions raise GeometryError
1034
+ - **Binary format integrity**: Truncated/corrupted binary raises TokenError with precise location
1035
+ - **Rosetta completeness**: Unmapped semantic IDs raise RosettaError
1036
+
1037
+ ### Running Tests
1038
+
1039
+ ```bash
1040
+ # Quick
1041
+ python -m pytest tests/ -q
1042
+
1043
+ # Verbose with coverage
1044
+ python -m pytest tests/ -v --tb=short
1045
+
1046
+ # Single module
1047
+ python -m pytest tests/test_roundtrip.py -v
1048
+ ```
1049
+
1050
+ ---
1051
+
1052
+ ## Implementation Architecture
1053
+
1054
+ ### Module Dependencies
1055
+
1056
+ ```
1057
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
1058
+ β”‚ pipeline.py β”‚ ← orchestrator
1059
+ β”‚ (SemanticPipeline) β”‚
1060
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
1061
+ β”‚ imports all phases
1062
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
1063
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
1064
+ β–Ό β–Ό β–Ό β–Ό β–Ό β–Ό β–Ό β–Ό
1065
+ nlp fsm temporal geometry matrix boolean token latin
1066
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
1067
+ β”‚ β–Ό β”‚ β–Ό β–Ό β–Ό
1068
+ β”‚ ir.py β—„β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ token lexer rosetta
1069
+ β”‚ β”‚ engine β”‚ β”‚
1070
+ β–Ό β–Ό β”‚ β”‚ β–Ό
1071
+ types.py ◄─── all modules β”‚ β”‚ subleq
1072
+ β”‚ import types β–Ό β”‚ β”‚
1073
+ β”‚ artifacts.py β”‚ β”‚
1074
+ β”‚ β”‚ β”‚ β”‚
1075
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”˜
1076
+ β”‚
1077
+ οΏ½οΏ½οΏ½
1078
+ verifier.py
1079
+ β”‚
1080
+ β–Ό
1081
+ agent_harness.py
1082
+ ```
1083
+
1084
+ ### Key Design Decisions
1085
+
1086
+ 1. **No inheritance hierarchy for IR nodes** β€” composition via dataclasses, not OOP. Every node type is a `@dataclass` with explicit fields.
1087
+ 2. **FSM transition table is data, not code** β€” the entire transition table is a list of `FSMTransition` records. Adding a state means adding a row, not writing a handler.
1088
+ 3. **Content-addressed everything** β€” nodes, operations, constraints, artifacts all have deterministic SHA-256 hashes derived from their content.
1089
+ 4. **Temporal state is a value, not a side effect** β€” `apply_temporal` returns `(result, new_Ο„)`. The old state is not mutated.
1090
+ 5. **Latin as intermediate language** β€” Latin provides a stable, unambiguous vocabulary that humans can read and machines can map. It serves as the "Rosetta Stone" between semantic intent and machine instruction.
1091
+
1092
+ ---
1093
+
1094
+ ## NLP Subsystem & Intent Classification
1095
+
1096
+ The NLP layer (`nlp.py`, 177 lines) provides the entry point for all natural language input. It operates in two modes: **rule-based** (zero dependencies, always available) and **BERT-enhanced** (optional, requires `transformers` + `torch`).
1097
+
1098
+ ### Rule-Based Analysis
1099
+
1100
+ Intent classification uses a controlled keyword vocabulary organized by semantic class:
1101
+
1102
+ | Intent Class | Keywords |
1103
+ |-------------|----------|
1104
+ | BUILD | build, create, make, construct, implement, write |
1105
+ | TRANSFORM | transform, convert, translate, map, rewrite, change |
1106
+ | VALIDATE | validate, verify, check, confirm, ensure, test |
1107
+ | GENERATE | generate, produce, emit, output, render |
1108
+ | PARSE | parse, read, decode, lex, scan, interpret |
1109
+ | COMPUTE | compute, calculate, multiply, add, subtract, solve |
1110
+ | RECONSTRUCT | reconstruct, restore, recover, rebuild, regenerate |
1111
+ | UNKNOWN | (no keywords matched β€” confidence 0.3) |
1112
+
1113
+ **Confidence scoring**: `min(0.5 + matched_keywords Γ— 0.15, 0.95)`. A single keyword match yields 0.65 confidence; three matches yield 0.95 (capped).
1114
+
1115
+ ### Entity & Operator Extraction
1116
+
1117
+ Regex-based extraction for 30 entity types and 18 operator types:
1118
+
1119
+ ```
1120
+ Entities: matrix, vector, parser, backend, service, function, module,
1121
+ component, token, format, stream, buffer, table, schema,
1122
+ graph, tree, list, array, string, integer, boolean, packet,
1123
+ message, frame, field, register...
1124
+
1125
+ Operators: multiply, add, subtract, divide, transform, convert,
1126
+ validate, parse, build, generate, compute, verify,
1127
+ reconstruct, normalize, encode, decode, serialize,
1128
+ deserialize
1129
+
1130
+ Constraints: deterministic, immutable, idempotent, commutative,
1131
+ associative, invertible, ordered, typed, bounded,
1132
+ validated, correct, minimal, canonical, stable
1133
+ ```
1134
+
1135
+ ### Relationship Extraction
1136
+
1137
+ Pattern: `{subject} {preposition} {object}` where prepositions include: of, from, to, into, for, with, by, through, via, as.
1138
+
1139
+ Example: `"build a parser for this binary format"` β†’ `[("parser", "for", "binary"), ("binary", "format", ...)]`
1140
+
1141
+ ### Ambiguity Detection
1142
+
1143
+ The pipeline detects ambiguous inputs via marker words: *it, this, that, they, something, somehow, maybe, perhaps, possibly, or something, etc, and so on*. Ambiguous inputs are flagged in the trace but not rejected β€” the FSM remains the admissibility gate.
1144
+
1145
+ ### BERT Interface
1146
+
1147
+ When `transformers` is installed and a model name is provided, the `BertDecoder` class extracts CLS embeddings from the last hidden state. These embeddings are stored in `SemanticCandidate.raw_embeddings` for downstream use but do not override rule-based classification. BERT output is always `SemanticCandidate` β€” never executable code.
1148
+
1149
+ ```python
1150
+ pipeline = SemanticPipeline(bert_model="bert-base-uncased")
1151
+ result = pipeline.run(RawInput("multiply two matrices"))
1152
+ # .raw_embeddings populated with 768-dim float vector
1153
+ ```
1154
+
1155
+ ---
1156
+
1157
+ ## Temporal Lock System
1158
+
1159
+ The temporal system (`temporal.py`, 86 lines) enforces a monotonic clock over every pipeline transformation. Every mutation advances Ο„ by exactly 1. Read-only passes preserve Ο„ (advance by 0). Locked states reject all mutations.
1160
+
1161
+ ### Temporal State
1162
+
1163
+ ```python
1164
+ @dataclass
1165
+ class TemporalState:
1166
+ tau: int # monotonic integer tick
1167
+ label: str # reason for current state
1168
+ locked: bool # if True, advance() raises TemporalViolation
1169
+ ```
1170
+
1171
+ ### Temporal Operations
1172
+
1173
+ | Operation | Ο„ Change | Use Case |
1174
+ |-----------|----------|----------|
1175
+ | `apply_temporal(fn, t, lock, stage)` | Ο„ β†’ Ο„+1 | Mutating transformation |
1176
+ | `preserve_temporal(fn, t, lock, stage)` | Ο„ β†’ Ο„ | Read-only pass |
1177
+ | `t.lock()` | Ο„ unchanged, locked=True | Freeze state permanently |
1178
+ | `t.advance()` on locked state | **TemporalViolation** | Safety enforcement |
1179
+
1180
+ ### Temporal Transitions
1181
+
1182
+ Every temporal operation records a `TemporalTransition`:
1183
+
1184
+ ```python
1185
+ @dataclass(frozen=True)
1186
+ class TemporalTransition:
1187
+ from_tau: int
1188
+ to_tau: int
1189
+ reason: str
1190
+ stage: str
1191
+ ```
1192
+
1193
+ The `TemporalLock` validates that `to_tau ∈ {from_tau, from_tau + 1}`. Any jump greater than 1 raises `TemporalViolation`. The full audit log is available via `lock.audit()` and is checked by the verifier at Phase 14.
1194
+
1195
+ ### Why Temporal Locking Matters
1196
+
1197
+ Without temporal accountability, transformations can silently overwrite each other, replay attacks become possible, and the provenance chain breaks. The temporal lock ensures that every mutation is sequenced, auditable, and irreversible.
1198
+
1199
+ ---
1200
+
1201
+ ## Geometry Engine
1202
+
1203
+ The geometry engine (`geometry.py`, 114 lines) enforces dimensional constraints on operations before they reach the matrix or SUBLEQ layers.
1204
+
1205
+ ### Geometry Record
1206
+
1207
+ ```python
1208
+ @dataclass(frozen=True)
1209
+ class Geometry:
1210
+ row: Optional[int] # number of rows
1211
+ col: Optional[int] # number of columns
1212
+ contraction: Optional[int] # contracted dimension
1213
+ shape: tuple[int, ...] # arbitrary shape tuple
1214
+ index_labels: tuple[str, ...] # named indices
1215
+ ```
1216
+
1217
+ ### Dimension Checking
1218
+
1219
+ **Matrix multiply** (`check_matmul`): A ∈ R^(mΓ—n), B ∈ R^(nΓ—p) β†’ C ∈ R^(mΓ—p). Requires `A.col == B.row`.
1220
+
1221
+ **Element-wise** (`check_elementwise`): add, subtract, XOR require identical shapes.
1222
+
1223
+ **Underdetermined**: If either dimension is `None`, the check passes (not a violation yet β€” the operation may be abstract).
1224
+
1225
+ ### Geometry Spec Parser
1226
+
1227
+ The `parse_geometry` function accepts multiple notations:
1228
+
1229
+ ```
1230
+ "3x4" β†’ Geometry(row=3, col=4, shape=(3,4))
1231
+ "R^(3x4)" β†’ Geometry(row=3, col=4, shape=(3,4))
1232
+ "(3,4)" β†’ Geometry(row=3, col=4, shape=(3,4))
1233
+ "scalar" β†’ Geometry(row=1, col=1, shape=(1,1))
1234
+ "5" β†’ Geometry(row=5, col=1, shape=(5,))
1235
+ ```
1236
+
1237
+ ### Hand-Traced Example
1238
+
1239
+ Input: `"multiply two matrices A and B where A is 3x4 and B is 4x5"`
1240
+
1241
+ ```
1242
+ Phase 5: Geometry Binding
1243
+ A.geometry = Geometry(row=3, col=4)
1244
+ B.geometry = Geometry(row=4, col=5)
1245
+ check_matmul(A, B):
1246
+ A.col (4) == B.row (4) βœ“
1247
+ output: Geometry(row=3, col=5, contraction=4)
1248
+
1249
+ Phase 6: SemanticMatrix Construction
1250
+ operation: multiply (arity 2)
1251
+ matrix layout (3Γ—3):
1252
+ [A_input] [contract=4] [B_input]
1253
+ [m=3] [n=4] [p=5]
1254
+ [output=C] [(3Γ—5)] [β€”]
1255
+ ```
1256
+
1257
+ If the input were `"multiply A (3x4) by B (5x2)"`, Phase 5 would raise:
1258
+ ```
1259
+ GeometryError: contraction mismatch: A.col=4 β‰  B.row=5
1260
+ ```
1261
+
1262
+ ---
1263
+
1264
+ ## Worked Example: End-to-End Trace
1265
+
1266
+ Input file: `examples/matrix_multiply.txt`
1267
+ ```
1268
+ multiply two matrices A and B where A is 3x4 and B is 4x5
1269
+ ```
1270
+
1271
+ ### Full Pipeline Trace
1272
+
1273
+ ```
1274
+ === TRACE ===
1275
+ INPUT: 7a3f2b1c-...
1276
+ NORMALIZE: 13 tokens
1277
+ LEXICAL_ANALYSIS: 1 sentences
1278
+ INTENT_DETECTION: compute (0.65)
1279
+ ENTITY_BINDING: ['matrix']
1280
+ SEMANTIC_BINDING: IR a1b2c3d4
1281
+ TEMPORAL_LOCK: Ο„0β†’Ο„1
1282
+ GEOMETRY_BINDING: dimensions validated
1283
+ MATRIX_FORM: 2x2
1284
+ BOOLEAN_ROOT: multiply
1285
+ BOOLEAN_NORMALIZATION: stable=True
1286
+ TOKENIZATION: 3 tokens, 16 bytes
1287
+ LATIN_CANONICALIZATION: INITIUM ET OPUS[multiply] FINIS
1288
+ LATIN_LEXING: 4 lex tokens
1289
+ ROSETTA_MAPPING: 4 entries
1290
+ SUBLEQ_IR: 5 instructions
1291
+ VERIFICATION: PASSED
1292
+ ARTIFACT_SELECTION: mode=natural_language
1293
+ EMISSION: artifact e5f6a7b8
1294
+ COMPLETE
1295
+
1296
+ === SUBLEQ ===
1297
+ DELIM_OPEN_0: SUBLEQ 0, 0, 109 ; PUSH:
1298
+ BOOL_AND_1: SUBLEQ 0, 256, 115 ; AND:
1299
+ SEMANTIC_OP_2: SUBLEQ 0, 0, 121 ; NOP:multiply
1300
+ DELIM_CLOSE_3: SUBLEQ 0, 0, 127 ; POP:
1301
+ SUBLEQ 0, 0, -1 ; HALT
1302
+
1303
+ === HASHES ===
1304
+ input: a7b3c9d1e5f2...
1305
+ semantic_ir: 4f8a2b1c7d3e...
1306
+ subleq: 9c1d4e7f2a3b...
1307
+ artifact: b2c5d8e1f4a7...
1308
+
1309
+ Status: VERIFIED
1310
+ ```
1311
+
1312
+ ### What Each Hash Proves
1313
+
1314
+ - **input**: SHA-256 of the raw text β€” proves the exact input
1315
+ - **semantic_ir**: SHA-256 of all node hashes chained β€” proves the IR structure
1316
+ - **subleq**: SHA-256 of the assembled binary β€” proves the machine code
1317
+ - **artifact**: SHA-256 of the rendered output β€” proves the final content
1318
+
1319
+ If any bit changes at any stage, the downstream hashes change. Replay with the same input produces identical hashes.
1320
+
1321
+ ---
1322
+
1323
+ ## Performance & Design Constraints
1324
+
1325
+ ### Performance
1326
+
1327
+ | Metric | Value |
1328
+ |--------|-------|
1329
+ | Full pipeline (single input) | ~3.4ms |
1330
+ | Test suite (113 tests) | 0.38s |
1331
+ | Throughput (estimated) | ~300 inputs/sec |
1332
+ | Memory per pipeline run | < 1MB |
1333
+ | SUBLEQ VM step limit | 100,000 (configurable) |
1334
+ | Binary token format overhead | ~4 bytes/token |
1335
+ | Python version | 3.11+ |
1336
+ | External dependencies | 0 (BERT optional) |
1337
+
1338
+ ### Design Constraints
1339
+
1340
+ - **Zero external dependencies** for the core pipeline. `transformers` + `torch` are optional for BERT.
1341
+ - **Deterministic** β€” no random seeds, no model variance, no floating-point accumulation in the core path.
1342
+ - **Fail-closed** β€” every error condition raises a typed exception. The FSM has no implicit fallthrough.
1343
+ - **Content-addressed** β€” every intermediate can be hashed and compared. Pipeline replays produce identical output.
1344
+ - **Audit trail** β€” `PipelineResult.trace` records every phase transition. `PipelineResult.hashes` records SHA-256 at input, IR, SUBLEQ, and artifact stages.
1345
+
1346
+ ---
1347
+
1348
+ ## Contributing & License
1349
+
1350
+ ### How to Contribute
1351
+
1352
+ 1. Fork the repository
1353
+ 2. Create a feature branch: `git checkout -b feature/your-feature`
1354
+ 3. Write code, tests, and verify: `python -m pytest tests/ -q`
1355
+ 4. Commit with semantic messages
1356
+ 5. Push and create a pull request
1357
+
1358
+ ### Commit Message Format
1359
+
1360
+ ```
1361
+ <type>: <subject>
1362
+
1363
+ Types: feat, fix, docs, test, refactor, perf, chore
1364
+ ```
1365
+
1366
+ ### License Compliance
1367
+
1368
+ Source files are individually licensed under **MPL-2.0** (file-level copyleft).
1369
+ The project as a whole is dual-licensed under **Apache-2.0 OR GPL-3.0-or-later** with Sovereign Leviathan Node License terms (AGPL-3.0 base).
1370
+
1371
+ **Commercial repository. No MIT permitted.**
1372
+
1373
+ Choose the license that fits your use case:
1374
+ - **Internal use**: Apache-2.0 for patent protection
1375
+ - **Open-source project**: GPL-3.0 to propagate freedom
1376
+ - **File-level modifications**: MPL-2.0 requires sharing changes to covered files
1377
+
1378
+ ### Contributors
1379
+
1380
+ - **Original Design**: Ahmad Ali Parr
1381
+ - **Implementation**: SnapKittyWest
1382
+ - **License**: Bel Esprit D'Accord Irrevocable Trust
1383
+
1384
+ ### References
1385
+
1386
+ - Quine, W.V.O. (1951). *Mathematical Logic*. Harvard University Press.
1387
+ - Mavaddat, F., & Parhami, B. (1988). "URISC: The Ultimate Reduced Instruction Set Computer." *International Journal of Electrical Engineering Education*, 25(4), 327–334.
1388
+ - Tseytin, G.S. (1983). "On the Complexity of Derivation in Propositional Calculus." *Studies in Constructive Mathematics and Mathematical Logic*.
1389
+ - POSIX.1-2008 System Interface.
1390
+
1391
+ ### πŸ’Ό Commercial License
1392
+
1393
+ This repository is published under **MPL-2.0**. Building a commercial product or service? A **proprietary commercial license** from Snapkitty Collective LLC lets you ship this code on terms other than MPL-2.0.
1394
+
1395
+ **[β†’ Get a commercial license](mailto:A.parr@belespritdaccord.uk?subject=Commercial%20license:%20semantic-engine)** Β· A.parr@belespritdaccord.uk
1396
+
1397
+
1398
+ ---
1399
+
1400
+ ## Type System & Enumerations
1401
+
1402
+ The type system (`types.py`, 199 lines) defines all enumerations, input types, and the error hierarchy used across all 18 modules.
1403
+
1404
+ ### FSMState (21 values)
1405
+
1406
+ ```
1407
+ INPUT β†’ NORMALIZE β†’ LEXICAL_ANALYSIS β†’ INTENT_DETECTION β†’
1408
+ ENTITY_BINDING β†’ SEMANTIC_BINDING β†’ TEMPORAL_LOCK β†’
1409
+ GEOMETRY_BINDING β†’ MATRIX_FORM β†’ BOOLEAN_ROOT β†’
1410
+ BOOLEAN_NORMALIZATION β†’ TOKENIZATION β†’ LATIN_CANONICALIZATION β†’
1411
+ LATIN_LEXING β†’ ROSETTA_MAPPING β†’ SUBLEQ_IR β†’ VERIFICATION β†’
1412
+ ARTIFACT_SELECTION β†’ EMISSION β†’ COMPLETE
1413
+ β†— (from any state)
1414
+ ERROR
1415
+ ```
1416
+
1417
+ ### IntentClass (8 values)
1418
+
1419
+ BUILD, TRANSFORM, VALIDATE, GENERATE, PARSE, COMPUTE, RECONSTRUCT, UNKNOWN.
1420
+
1421
+ Every input is classified into exactly one intent. UNKNOWN is valid but triggers a lower confidence score (0.3).
1422
+
1423
+ ### SemanticNodeType (8 values)
1424
+
1425
+ OPERATION, ENTITY, CONSTRAINT, RELATION, INTENT, GEOMETRY, TEMPORAL, LITERAL.
1426
+
1427
+ These types determine how the node is processed by downstream phases β€” operations require arity checking, entities get type-tagged, constraints become Boolean variables.
1428
+
1429
+ ### ArtifactType (8 values)
1430
+
1431
+ PROMPT, DOCUMENT, SOURCE, CONFIG, TEST, SCHEMA, REPORT, BACKEND.
1432
+
1433
+ Each type maps to a specific template and rendering strategy.
1434
+
1435
+ ### BooleanOperator (8 values)
1436
+
1437
+ AND, OR, NOT, XOR, IMPLIES, EQUIVALENCE, TRUE, FALSE.
1438
+
1439
+ These map 1:1 to Latin lexemes (ET, VEL, NON, AUT, SI, IFF, VERUM, FALSUM) and 1:1 to SUBLEQ operations via the Rosetta table.
1440
+
1441
+ ### Input Types
1442
+
1443
+ ```python
1444
+ @dataclass(frozen=True)
1445
+ class RawInput:
1446
+ text: str # the natural language input
1447
+ source: str = "stdin" # origin identifier
1448
+ timestamp: float # wall clock at creation
1449
+ input_id: str # UUID v4
1450
+ # .hash β†’ SHA-256 of text
1451
+
1452
+ @dataclass(frozen=True)
1453
+ class NormalizedInput:
1454
+ original: RawInput # back-reference
1455
+ text: str # whitespace-normalized
1456
+ tokens: tuple[str, ...] # word-level tokenization
1457
+ sentences: tuple[str, ...] # sentence segmentation
1458
+ # .hash β†’ SHA-256 of normalized text
1459
+ ```
1460
+
1461
+ Both types are frozen (immutable). The `hash` property computes SHA-256 on demand for content addressing.
1462
+
1463
+ ### SemanticCandidate
1464
+
1465
+ The bridge between NLP and the typed IR:
1466
+
1467
+ ```python
1468
+ @dataclass
1469
+ class SemanticCandidate:
1470
+ input_ref: str # ties back to RawInput.input_id
1471
+ intent: IntentClass # classified intent
1472
+ entities: list[str] # extracted entities
1473
+ operators: list[str] # extracted operators
1474
+ constraints: list[str] # extracted constraints
1475
+ relationships: list[tuple[str, str, str]] # subject-preposition-object triples
1476
+ confidence: float # [0.0, 1.0] β€” validated in __post_init__
1477
+ ambiguous: bool # whether ambiguity markers detected
1478
+ raw_embeddings: Optional[list[float]] # BERT CLS vector (if available)
1479
+ ```
1480
+
1481
+ The `confidence` field is validated on construction β€” values outside `[0.0, 1.0]` raise `ValueError`. This prevents downstream phases from operating on malformed candidates.
1482
+
1483
+ ---
1484
+
1485
+ ## Data Flow Summary
1486
+
1487
+ ```
1488
+ RawInput("multiply two matrices A and B where A is 3x4 and B is 4x5")
1489
+ β”‚
1490
+ β–Ό normalize_input()
1491
+ NormalizedInput(tokens=("multiply","two","matrices","A","and","B",
1492
+ "where","A","is","3x4","and","B","is","4x5"),
1493
+ sentences=("multiply two matrices A and B where A is 3x4 and B is 4x5",))
1494
+ β”‚
1495
+ β–Ό BertDecoder.decode()
1496
+ SemanticCandidate(intent=COMPUTE, entities=["matrix"],
1497
+ operators=["multiply"], constraints=[],
1498
+ confidence=0.65, ambiguous=False)
1499
+ β”‚
1500
+ β–Ό SemanticNode.make() + SemanticIR.create()
1501
+ SemanticIR(intent=COMPUTE, nodes=[
1502
+ SemanticNode(type=OPERATION, operation=multiply/1,
1503
+ operands=[Operand("matrix","entity")],
1504
+ constraints=[], geometry=None, Ο„=0)
1505
+ ])
1506
+ β”‚
1507
+ β–Ό apply_temporal() Ο„β‚€ β†’ τ₁
1508
+ β–Ό enforce_geometry() geometry validated
1509
+ β–Ό SemanticMatrixBuilder.build() 2Γ—2 SemanticMatrix
1510
+ β–Ό semantic_constraints_to_boolean() BooleanNormalForm (CNF)
1511
+ β–Ό BoolEncoder.encode_expr() BooleanTokenStream β†’ bytes
1512
+ β–Ό LatinCanonicalizer.canonicalize() "INITIUM ET OPUS[multiply] FINIS"
1513
+ β–Ό LatinLexer.lex_canonical() LexResult (4 tokens)
1514
+ β–Ό RosettaMapper.map() RosettaMapping (4 entries)
1515
+ β–Ό build_program_from_rosetta() SubleqProgram (5 instructions)
1516
+ β”‚
1517
+ β–Ό Verifier: 10 checks Γ— PASS
1518
+ β”‚
1519
+ β–Ό ArtifactEngine.build_document()
1520
+ Artifact(type=DOCUMENT, content="COMPUTE\n\nIntent: compute | Nodes: 1",
1521
+ verification_state=VERIFIED)
1522
+ β”‚
1523
+ β–Ό PipelineResult(ok=True, hashes={input, semantic_ir, subleq, artifact})
1524
+ ```
1525
+
1526
+ Every arrow is a phase transition in the FSM. Every intermediate is hashed. Every mutation advances Ο„. Every output is verified.
1527
+
1528
+ ---
1529
+
1530
+ **Status**: Production-ready β€” 20 phases, 113 tests, zero stubs
1531
+ **Certification Date**: 2026-09-21
1532
+ **Test Count**: 113 passed
1533
+ **Code Quality**: 4,143 lines, zero external dependencies
1534
+ **License**: MPL-2.0 (file) | Apache-2.0 OR GPL-3.0-or-later (project)
1535
+
1536
+ *"Hark, though this node be but a spark,
1537
+ Its covenant endureth through the dark."*