issdandavis commited on
Commit
da8272e
·
verified ·
1 Parent(s): 9112cd2

Upload folder using huggingface_hub

Browse files
Files changed (3) hide show
  1. README.md +32 -0
  2. config.json +9 -0
  3. scratch_coder.pt +3 -0
README.md ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags: [from-scratch, opcode, ternary, trit, execution-verified, tiny]
4
+ ---
5
+
6
+ # Binary-Trit Coder (Clay's opcode model)
7
+
8
+ A **from-scratch, 3.17M-param causal transformer** trained from *random init* on Issac's own opcode
9
+ substrate — a bijective ternary ("trit") op grammar: `join (+)`, `take (−)`, `weave (×)` over digit leaves,
10
+ prefix-notation, depth-2. No pretraining, no outside corpus.
11
+
12
+ ## Honest benchmark — its OWN domain, not Python
13
+ The metric is **execution-verify rate**: sample a program, *run it for real*, check the model's answer.
14
+ | Test | Score |
15
+ |---|---|
16
+ | in-distribution (depth-2, fresh) | **98.6%** |
17
+ | OOD generalization (depth-3, never trained) | 47.0% |
18
+
19
+ **This is NOT a Python code-gen model.** It has an 18-token opcode vocab; it will score ~0 on
20
+ LiveCodeBench/SciCode (wrong domain). Its "learn from doing" signal is the verify-rate above.
21
+
22
+ ## The chrysalis (capability layers, all execution-verified)
23
+ - variables/let-bindings (69.5% end-to-end; 100% with a calculator + show-work)
24
+ - ping-pong φ-lattice solver — recovers a hidden intermediate 100%
25
+ - calc-offload, dark-space inference, combustion candidate-spark
26
+
27
+ ## Files
28
+ `scratch_coder.pt` (weights + vocab), `config.json`. Architecture + inference in
29
+ [github.com/issdandavis/loom](https://github.com/issdandavis/loom) `scratch_coder.py`.
30
+
31
+ *Thesis: coding is finite + bijective + execution-verifiable, so a small model from random init can MASTER
32
+ the mapping — proven here at 98.6%.*
config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "trit-opcode-gpt",
3
+ "architecture": "TinyGPT (from-scratch causal transformer)",
4
+ "d_model": 256, "heads": 8, "layers": 4, "max_len": 24,
5
+ "vocab": ["<pad>","<bos>","<eos>","=","-","0","1","2","3","4","5","6","7","8","9","join","take","weave"],
6
+ "ops": {"join": "+", "take": "-", "weave": "*"},
7
+ "trained_from": "random init (no pretraining)",
8
+ "metric": "execution-verify rate (run the emitted program, check the answer)"
9
+ }
scratch_coder.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f0cf4a29a6a159241ed812ed5ecaac80ec30978d8ee708d03aa93bffcf15dae
3
+ size 12722131