README.md

---
license: apache-2.0
language:
  - en
tags:
  - kenga
  - programming-language
  - token-prediction
  - small-model
  - linear-classifier
  - neuromodel
  - baseline
  - v0-1
datasets:
  - kenga-corpus
metrics:
  - token-accuracy
model_name: kenga-prophet
pipeline_tag: text-generation
library_name: kenga
---

# Kenga Prophet — M2 baseline (v0.1)

The **first** Kenga-native trained model published externally. This
release is **immutable**: subsequent runs ship under separate model
names (`kenga-prophet-m2-k16`, …). Use this card as a permanent
point of reference for what "6,300 parameters + Kenga corpus" did
on the day of the first release.

## What this model is

* Linear softmax classifier: `P(next_token | last_K_tokens)`
* Vocabulary: **28 tokens** (Kenga lexemes + `ID`/`NUM`)
* Window: **K = 8** preceding tokens
* Parameters: **6,300 trainable weights** (28 × (8 × 28 + 1))
* Trained in **Python with numpy only**, no torch, no GPU
* Inference runs in **Kenga Lite more VM** (no GPU, no Rust)

If `6,300 / 27,000,000,000` sounds absurd, that's exactly the
proportion the user wants to track: small + structurally correct
versus big + general-purpose.

## Numbers (held-out next-token accuracy)

```
kenga_seed_add   19/88  = 21.6 %
kenga_seed_fact  14/62  = 22.6 %
kenga_seed_fib   10/55  = 18.2 %
kenga_seed_max   21/88  = 25.0 %
kenga_seed_mul   16/82  = 20.7 %
kenga_seed_pow   15/68  = 22.1 %
kenga_seed_sqr   13/68  = 19.1 %
kenga_seed_sub   16/82  = 20.7 %
kenga_seed_sum   26/104 = 25.0 %
overall          149/697 = 21.4 %
```

These are **token-accuracy numbers**, not BLEU. The "trick" is that
Kenga's grammar has no ambiguity in the 28-token codec, so even modest
per-token accuracy can produce **syntactically valid** continuations.

## Provenance (frozen at v0.1 release)

```
Kenga commit  :  993187398e8d5cda85e7c8a1fca44e648f87016a
Training   V  :  28
Context    K  :  8
Embedding features/V:  226 (K*V + bias)
Total params      :  6,300
Optimizer         :  Adam  (lr 5e-3, betas 0.9/0.999)
Epochs            :  60
Training corpus   :  168 .kenga source files, 154,000 tokens
Train/test split  :  first 90% / last 10%
Held-out program set : 9 kenga_seed_*.kenga programs

weights blob sha (16 hex):  28f7ef5c39008b52
vocab  blob sha            :  0246917ce1a8f263
train  blob sha            :  bc558fa4207b6db1
test   blob sha            :  d991ac600746b4c8
meta   blob sha            :  d13eb31ddcaba14b

Total on-disk size (all 5 artefacts):  ~ 580 KB
RAM at inference (Lite more VM):       ~ 1 MB
Wall-clock training time:             ~ 1–2 min  (numpy only)
Wall-clock per-token inference:       ~ 30 ms   (Lite more VM, single argmax)
Wall-clock full-prediction inference:  ~ 1 s    (Lite, 100 generated tokens)
CPU-only, no GPU required.
```

The `kenga-prophet` repo on Hugging Face is **immutable** at this
SHA: subsequent improvements go to `kenga-prophet-m2-k16`,
`kenga-prophet-m2-mlp`, etc. The v0.1 card stays as the **first**
point of reference.

**Weights format fix (v2 of this file):** the initial upload serialized
`\n` as literal backslash-n (single-line), which corrupted the weights,
vocab, and meta files for any consumer. This revision re-serializes them
with real newlines. All provenance values above (commit SHA, blob
hashes, params) are unchanged — this is a serialization fix, not a
retrain.

## Program-validity rate (honest, measured)

`tools/kenchat.py --probe` runs the model and feeds the generated
program to `kenga-lite`. Current result for v0.1:

```
compile-ok:    0/9 = 0.0%
run-ok:        0/9 = 0.0%
match value:   0/9 = 0.0%
```

The model cannot yet generate structurally valid programs: greedy
decoding always predicts `fn`, and 21% token accuracy means 79% of
tokens are wrong. This 0/9 is the honest baseline the ladder must
climb — see "What this model CANNOT do" below.

## What this model CAN do

* Given an 8-token prefix from Kenga source, predict the next
  token from the 28-token codec.
* Run in two or three minutes on a 1660-class GPU-less laptop
  (this is the entire training time).
* Be inspected losslessly: weights are integers in the file at
  `minds/mid_prophet_m2_big_w.txt`, vocabulary at
  `minds/mid_prophet_m2_big_vocab.txt`, training config in
  `minds/mid_prophet_m2_big_meta.txt`.

## What this model CANNOT do

* Open-ended chat on natural-language queries. It was trained on
  Kenga source, not on English.
* Pass-rate on long (multi-line) generation at this K=8 window is
  weak because **21% next-token accuracy means 79% wrong tokens**;
  one wrong token later in the program bleeds into syntactic
  breakage.
* Encode Kenga semantics. It is a next-token surface statistic.
  See Mid-Prophet M1 (`docs/PICO_PROPHET.md`) for a non-trained
  signature-based classifier that does better on identity
  classification tasks.

## Why this is genuinely Kenga-native and not "just another Python model"

| axis | this model | a Hugging Face PyTorch reference |
|---|---|---|
| Training | numpy only | PyTorch / JAX / TF |
| Optimizer | hand-rolled Adam (~80 lines) | torch.optim.Adam |
| Data pipeline | `walk('kenga/' + 'examples/')` + tokenize | datasets.load_dataset |
| Inference | `bootstrap/bin/kenga-lite.exe` reading weights from `minds/...` | HF pipeline / transformers |
| Runtime | Kenga Lite more VM (no Rust, no GPU) | CUDA / ROCm / CPU SIMD |
| Tokenisation | custom 28-token Kenga codec | BPE / WordPiece |
| File format | plain text integer weights | safetensors / ONNX |

Every stage is the same Kenga: tokenisation is built around the
Kenga grammar, inference runs on the `kenga-lite` binary that comes
with the language, and there is **no Python dependency in the
critical path** of inference. That is what makes this a
*Kenga-native* model and not "a Python model with Kenga data".

## Fixed sample predictions (token ids 0..27)

For random prefixes drawn from the held-out stream at
position 32 onward, the model picks the following tokens. These
are illustrative raw outputs, **not corrected**.

```
prefix  [13, 7, 14, 15, 7, ...]           predict token 7  (i64)
prefix  [11, 1, 26, 16, 12, ...]           predict token 10 (semicolon)
prefix  [0, 26, 9, 8, 7, 26, 14, ...]     predict token 11 ({)
```

These are toy outputs; the artefact here is **provenance and
ladder position**, not finished quality.

## Reproduce

```
# requires numpy only; on Windows:
git clone https://github.com/GermannM/kenga-lang
cd kenga-lang
python tools/train_m2_big.py
# produces minds/mid_prophet_m2_big_*.txt (~ 580 KB total)
```

```bash
# inference on a token stream:
minds/mid_prophet_m2_big_w.txt minds/mid_prophet_m2_big_vocab.txt  # explicit
bootstrap\bin\kenga-lite.exe run examples\ml\mid_prophet_m2_run.kenga
```

The orchestrator script `scripts/mid-birth-m2.sh` runs inference
against the 9 held-out programs and reports the aggregate accuracy.

## Citation

* `docs/PICO_PROPHET.md`     — the ladder Pico-Prophet → Mid-Prophet M1 → M2
* `docs/NEUROMODEL_27B.md`   — the six-axis stack behind the claim
* `tools/train_m2_big.py`    — the training script that produced this artefact
* `examples/ml/mid_prophet_m2_run.kenga` — the Lite inference harness