Upload folder using huggingface_hub

README.md

@@ -0,0 +1,96 @@
+---
+license: apache-2.0
+library_name: transformers
+tags:
+  - torchwright
+  - compiled-transformer
+  - calculator_scratchpad
+pipeline_tag: text-generation
+---
+
+# `calculator_scratchpad`, max_digits=6 (torchwright)
+
+A **compiled** transformer: the
+[torchwright](https://github.com/physicsrob/torchwright) compiler emitted
+these weights directly from a computation graph — nothing was trained.  This
+bundle is the `calculator_scratchpad` example built with `max_digits=6`: a computation graph for integer arithmetic (`A op B` with `op` in `+ - *`) that streams its serial carry/borrow work as visible thinking tokens before emitting the answer.
+
+The bundle uses the stock Phi-3 architecture and loads through `transformers`
+without custom model code or `trust_remote_code`.
+
+Run it in **fp32** with **greedy decoding** (`do_sample=False`).  Other
+precisions and decoding modes are outside the supported contract.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo_id = 'physicsrob/torchwright-calculator-scratchpad-max-digits-6'
+model = AutoModelForCausalLM.from_pretrained(repo_id).eval()
+tok = AutoTokenizer.from_pretrained(repo_id)
+
+enc = tok('12*34\n', return_tensors="pt")
+out = model.generate(enc["input_ids"], max_new_tokens=96, do_sample=False,
+                     eos_token_id=tok.eos_token_id, pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Input and output
+
+Prompts are `A op B` terminated by a newline: two non-negative decimal
+operands of up to 6 digits, with `op` one of `+`, `-`, `*`.
+Subtraction may produce a negative result.  Wider operands, or any character
+outside the model's small vocabulary, are outside the contract — the output
+is undefined.
+
+| prompt | output |
+|---|---|
+| `12*34` | `<THINKING>…</THINKING>408` |
+| `7+8` | `<THINKING>…</THINKING>15` |
+| `999999*999999` | `<THINKING>…</THINKING>999998000001` |
+| `999999+1` | `<THINKING>…</THINKING>1000000` |
+| `999999-123456` | `<THINKING>…</THINKING>876543` |
+| `123456-999999` | `<THINKING>…</THINKING>-876543` |
+
+The answer is preceded by a visible scratchpad between `<THINKING>` and
+`</THINKING>`: superscript glyphs (`⁰¹²`) stream the per-column carry /
+borrow / comparison state, plain digits are the unnormalized scratch answer,
+and subscript glyphs (`₀₁₂`) stream the leading-zero cleanup count.  The
+final answer follows the closing tag — e.g. `999999+1` ends
+`…</THINKING>1000000`.
+
+## Intended use and limitations
+
+This model is a demonstration of a computation graph compiled into transformer
+weights.  It is not a general language model or a general-purpose calculator;
+only the input contract above is supported.
+
+## Verification
+
+The examples above are exact reference outputs.  The Modal publishing path
+reloads the emitted checkpoint through stock `transformers`, checks those
+examples plus additional width-limit cases against Python integer arithmetic,
+and refuses to upload on a mismatch.  This is a functional smoke test, not
+exhaustive verification of every allowed expression.
+
+## Size
+
+The checkpoint stores 27.92 GB of dense fp32 weights (6,980,206,592 entries) at
+the example family's shared compile width.  99.99% of those entries are
+exactly zero: the vast majority of the model is unused canvas, so size reflects
+the compile geometry rather than stored knowledge.
+
+The zero entries are not compressed, and dense `transformers` execution still
+pays their memory and compute cost.  CPU execution is supported; allow
+additional RAM beyond the checkpoint size.
+
+## Family
+
+One example of many compiled with torchwright.  Calculator siblings —
+`calculator-simple` (serial arithmetic, depth grows with the digit count),
+`calculator-advanced` (carry-lookahead, near-flat depth), and
+`calculator-scratchpad` (flat depth; the serial work streams out as visible
+thinking tokens) — are published at several digit widths.  Browse the
+[torchwright calculator models](https://huggingface.co/models?search=physicsrob%2Ftorchwright-calculator)
+on Hugging Face.

config.json

@@ -0,0 +1,33 @@
+{
+  "architectures": [
+    "Phi3ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 15,
+  "embd_pdrop": 0.0,
+  "eos_token_id": 16,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 5879,
+  "max_position_embeddings": 512,
+  "model_type": "phi3",
+  "num_attention_heads": 116,
+  "num_hidden_layers": 18,
+  "num_key_value_heads": 116,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": null,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "partial_rotary_factor": 1.0,
+    "rope_theta": 500000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "vocab_size": 45
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "bos_token_id": 15,
+  "eos_token_id": 16,
+  "pad_token_id": 16,
+  "transformers_version": "5.12.1"
+}

model-00001-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2caa957219c4f1b8d1393dacc2bf9acc236c1e92571d4c1c2f45bb6f1d72a67c
+size 1551008208

model-00002-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:048df8f421c543939d4b4f9be88c4da081776a7fcb645c5a2efa312d0e9bd0e4
+size 1551008208

model-00003-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e889bf2765015cd9d638b9890a453ea69cd5d8d16c4a2437351b9158841fdc15
+size 1551008208

model-00004-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dabceb632813d27f0321bbb9aad04ff42f1b65dae56a1e393fbe6cb0ace3751e
+size 1551008208

model-00005-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c3a81a95b7eab6ce2f602f7cb9fb575b33910ba1e24e3a3b0d3d87d44b362b4
+size 1551008208

model-00006-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:088ddc7e9e167cd21c266c3fbd6131b3ffd32a5f5d489431a86a6ac2435edb2e
+size 1551008208

model-00007-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:884e42543357786d3beecadf23fcd25007d7212dda69ad08257257189efd4ab3
+size 1551008208

model-00008-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5cdd83b6925c108446e1e8da75397b1002e39d85da945390aa56bdd41e8bd40
+size 1551008208

model-00009-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10ea2265a24901d9b555b1042a65e36a20e33572cce022882590c4c3451b803d
+size 1551008208

model-00010-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f4aa09ddaf32b4afb700bf508b637baad3c7dce402d1b6a681ec201704e26b1
+size 1551008208

model-00011-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57f427eadf9a6f30133e311c76634712b9acbb10cc6f4312486d3c4a8700f3a5
+size 1551008208

model-00012-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efab1419ac321e437affaf72c27779e5db6487ad08113a4e7c36ae5a00cf2e5f
+size 1551008208

model-00013-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22ee2e2671535b990af6c05b571514a126487d56fb09c561551759088a23429d
+size 1551008208

model-00014-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:358560dbd3c89c72d3355483cb8292c932e13c3b41acc5c303d923f11cc85e4f
+size 1551008208

model-00015-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44e0b4c35571dc1bd4525f0a54b430b67df12809cc12d30de4997e90eb66f6a1
+size 1551008208

model-00016-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c74d00d69153e7148a0a45f3cd031d761e88aca14d33cca18a89bec8d5d1e19b
+size 1551008208

model-00017-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:576e2a087c57085754da845a006698effbbb60b77175f81d81f48b9b1eb61dfc
+size 1551008208

model-00018-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c381ecff486cbd4807a1b4b3b3672deaf57f122ae278b22d991f2680a4364363
+size 1551008208

model-00019-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac65bf30b9cfbfea0847c03cc3674e44afc6034b56e164a698b10630c6d7725c
+size 2691080

model.safetensors.index.json

@@ -0,0 +1 @@
+{"metadata": {"total_size": 27920826368}, "weight_map": {"model.layers.0.self_attn.qkv_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.mlp.gate_up_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.mlp.down_proj.weight": "model-00001-of-00019.safetensors", "model.layers.1.self_attn.qkv_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.self_attn.o_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.mlp.gate_up_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.mlp.down_proj.weight": "model-00002-of-00019.safetensors", "model.layers.2.self_attn.qkv_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.self_attn.o_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.mlp.gate_up_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.mlp.down_proj.weight": "model-00003-of-00019.safetensors", "model.layers.3.self_attn.qkv_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.self_attn.o_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.mlp.gate_up_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.mlp.down_proj.weight": "model-00004-of-00019.safetensors", "model.layers.4.self_attn.qkv_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.self_attn.o_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.mlp.gate_up_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.mlp.down_proj.weight": "model-00005-of-00019.safetensors", "model.layers.5.self_attn.qkv_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.self_attn.o_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.mlp.gate_up_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.mlp.down_proj.weight": "model-00006-of-00019.safetensors", "model.layers.6.self_attn.qkv_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.self_attn.o_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.mlp.gate_up_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.mlp.down_proj.weight": "model-00007-of-00019.safetensors", "model.layers.7.self_attn.qkv_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.self_attn.o_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.mlp.gate_up_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.mlp.down_proj.weight": "model-00008-of-00019.safetensors", "model.layers.8.self_attn.qkv_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.self_attn.o_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.mlp.gate_up_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.mlp.down_proj.weight": "model-00009-of-00019.safetensors", "model.layers.9.self_attn.qkv_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.self_attn.o_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.mlp.gate_up_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.mlp.down_proj.weight": "model-00010-of-00019.safetensors", "model.layers.10.self_attn.qkv_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.self_attn.o_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.mlp.gate_up_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.mlp.down_proj.weight": "model-00011-of-00019.safetensors", "model.layers.11.self_attn.qkv_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.self_attn.o_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.mlp.gate_up_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.mlp.down_proj.weight": "model-00012-of-00019.safetensors", "model.layers.12.self_attn.qkv_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.self_attn.o_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.mlp.gate_up_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.mlp.down_proj.weight": "model-00013-of-00019.safetensors", "model.layers.13.self_attn.qkv_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.self_attn.o_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.mlp.gate_up_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.mlp.down_proj.weight": "model-00014-of-00019.safetensors", "model.layers.14.self_attn.qkv_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.self_attn.o_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.mlp.gate_up_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.mlp.down_proj.weight": "model-00015-of-00019.safetensors", "model.layers.15.self_attn.qkv_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.self_attn.o_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.mlp.gate_up_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.mlp.down_proj.weight": "model-00016-of-00019.safetensors", "model.layers.16.self_attn.qkv_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.self_attn.o_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.mlp.gate_up_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.mlp.down_proj.weight": "model-00017-of-00019.safetensors", "model.layers.17.self_attn.qkv_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.self_attn.o_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.mlp.gate_up_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.mlp.down_proj.weight": "model-00018-of-00019.safetensors", "model.embed_tokens.weight": "model-00019-of-00019.safetensors", "model.norm.weight": "model-00019-of-00019.safetensors", "model.layers.0.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.0.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.1.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.1.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.2.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.2.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.3.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.3.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.4.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.4.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.5.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.5.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.6.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.6.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.7.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.7.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.8.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.8.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.9.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.9.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.10.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.10.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.11.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.11.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.12.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.12.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.13.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.13.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.14.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.14.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.15.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.15.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.16.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.16.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.17.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.17.post_attention_layernorm.weight": "model-00019-of-00019.safetensors"}}

tokenizer.json

@@ -0,0 +1,142 @@
+{
+  "version": "1.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 15,
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 16,
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": null,
+  "pre_tokenizer": {
+    "type": "Split",
+    "pattern": {
+      "Regex": "[\\s\\S]"
+    },
+    "behavior": "Isolated",
+    "invert": false
+  },
+  "post_processor": {
+    "type": "TemplateProcessing",
+    "single": [
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      }
+    ],
+    "pair": [
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      },
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "B",
+          "type_id": 0
+        }
+      }
+    ],
+    "special_tokens": {
+      "<bos>": {
+        "id": "<bos>",
+        "ids": [
+          15
+        ],
+        "tokens": [
+          "<bos>"
+        ]
+      }
+    }
+  },
+  "decoder": {
+    "type": "Fuse"
+  },
+  "model": {
+    "type": "WordLevel",
+    "vocab": {
+      "0": 0,
+      "1": 1,
+      "2": 2,
+      "3": 3,
+      "4": 4,
+      "5": 5,
+      "6": 6,
+      "7": 7,
+      "8": 8,
+      "9": 9,
+      "+": 10,
+      "-": 11,
+      "*": 12,
+      "\n": 13,
+      " ": 14,
+      "<bos>": 15,
+      "<eos>": 16,
+      "<THINKING>": 17,
+      "⁰ ": 18,
+      "¹ ": 19,
+      "² ": 20,
+      "³ ": 21,
+      "⁴ ": 22,
+      "⁵ ": 23,
+      "⁶ ": 24,
+      "⁷ ": 25,
+      "⁸ ": 26,
+      "⁹ ": 27,
+      "¹⁰ ": 28,
+      "¹¹ ": 29,
+      "¹² ": 30,
+      "₀ ": 31,
+      "₁ ": 32,
+      "₂ ": 33,
+      "₃ ": 34,
+      "₄ ": 35,
+      "₅ ": 36,
+      "₆ ": 37,
+      "₇ ": 38,
+      "₈ ": 39,
+      "₉ ": 40,
+      "₁₀ ": 41,
+      "₁₁ ": 42,
+      "₁₂ ": 43,
+      "</THINKING>": 44
+    },
+    "unk_token": "<unk>"
+  }
+}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null
+}