Upload folder using huggingface_hub
Browse files- README.md +32 -0
- config.json +9 -0
- scratch_coder.pt +3 -0
README.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags: [from-scratch, opcode, ternary, trit, execution-verified, tiny]
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Binary-Trit Coder (Clay's opcode model)
|
| 7 |
+
|
| 8 |
+
A **from-scratch, 3.17M-param causal transformer** trained from *random init* on Issac's own opcode
|
| 9 |
+
substrate — a bijective ternary ("trit") op grammar: `join (+)`, `take (−)`, `weave (×)` over digit leaves,
|
| 10 |
+
prefix-notation, depth-2. No pretraining, no outside corpus.
|
| 11 |
+
|
| 12 |
+
## Honest benchmark — its OWN domain, not Python
|
| 13 |
+
The metric is **execution-verify rate**: sample a program, *run it for real*, check the model's answer.
|
| 14 |
+
| Test | Score |
|
| 15 |
+
|---|---|
|
| 16 |
+
| in-distribution (depth-2, fresh) | **98.6%** |
|
| 17 |
+
| OOD generalization (depth-3, never trained) | 47.0% |
|
| 18 |
+
|
| 19 |
+
**This is NOT a Python code-gen model.** It has an 18-token opcode vocab; it will score ~0 on
|
| 20 |
+
LiveCodeBench/SciCode (wrong domain). Its "learn from doing" signal is the verify-rate above.
|
| 21 |
+
|
| 22 |
+
## The chrysalis (capability layers, all execution-verified)
|
| 23 |
+
- variables/let-bindings (69.5% end-to-end; 100% with a calculator + show-work)
|
| 24 |
+
- ping-pong φ-lattice solver — recovers a hidden intermediate 100%
|
| 25 |
+
- calc-offload, dark-space inference, combustion candidate-spark
|
| 26 |
+
|
| 27 |
+
## Files
|
| 28 |
+
`scratch_coder.pt` (weights + vocab), `config.json`. Architecture + inference in
|
| 29 |
+
[github.com/issdandavis/loom](https://github.com/issdandavis/loom) `scratch_coder.py`.
|
| 30 |
+
|
| 31 |
+
*Thesis: coding is finite + bijective + execution-verifiable, so a small model from random init can MASTER
|
| 32 |
+
the mapping — proven here at 98.6%.*
|
config.json
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "trit-opcode-gpt",
|
| 3 |
+
"architecture": "TinyGPT (from-scratch causal transformer)",
|
| 4 |
+
"d_model": 256, "heads": 8, "layers": 4, "max_len": 24,
|
| 5 |
+
"vocab": ["<pad>","<bos>","<eos>","=","-","0","1","2","3","4","5","6","7","8","9","join","take","weave"],
|
| 6 |
+
"ops": {"join": "+", "take": "-", "weave": "*"},
|
| 7 |
+
"trained_from": "random init (no pretraining)",
|
| 8 |
+
"metric": "execution-verify rate (run the emitted program, check the answer)"
|
| 9 |
+
}
|
scratch_coder.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2f0cf4a29a6a159241ed812ed5ecaac80ec30978d8ee708d03aa93bffcf15dae
|
| 3 |
+
size 12722131
|