Upload folder using huggingface_hub

README.md

@@ -0,0 +1,96 @@
+---
+license: apache-2.0
+library_name: transformers
+tags:
+  - torchwright
+  - compiled-transformer
+  - calculator_scratchpad
+pipeline_tag: text-generation
+---
+
+# `calculator_scratchpad`, max_digits=3 (torchwright)
+
+A **compiled** transformer: the
+[torchwright](https://github.com/physicsrob/torchwright) compiler emitted
+these weights directly from a computation graph — nothing was trained.  This
+bundle is the `calculator_scratchpad` example built with `max_digits=3`: a computation graph for integer arithmetic (`A op B` with `op` in `+ - *`) that streams its serial carry/borrow work as visible thinking tokens before emitting the answer.
+
+The bundle uses the stock Phi-3 architecture and loads through `transformers`
+without custom model code or `trust_remote_code`.
+
+Run it in **fp32** with **greedy decoding** (`do_sample=False`).  Other
+precisions and decoding modes are outside the supported contract.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo_id = 'physicsrob/torchwright-calculator-scratchpad-max-digits-3'
+model = AutoModelForCausalLM.from_pretrained(repo_id).eval()
+tok = AutoTokenizer.from_pretrained(repo_id)
+
+enc = tok('12*34\n', return_tensors="pt")
+out = model.generate(enc["input_ids"], max_new_tokens=96, do_sample=False,
+                     eos_token_id=tok.eos_token_id, pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Input and output
+
+Prompts are `A op B` terminated by a newline: two non-negative decimal
+operands of up to 3 digits, with `op` one of `+`, `-`, `*`.
+Subtraction may produce a negative result.  Wider operands, or any character
+outside the model's small vocabulary, are outside the contract — the output
+is undefined.
+
+| prompt | output |
+|---|---|
+| `12*34` | `<THINKING>…</THINKING>408` |
+| `7+8` | `<THINKING>…</THINKING>15` |
+| `999*999` | `<THINKING>…</THINKING>998001` |
+| `999+1` | `<THINKING>…</THINKING>1000` |
+| `999-123` | `<THINKING>…</THINKING>876` |
+| `123-999` | `<THINKING>…</THINKING>-876` |
+
+The answer is preceded by a visible scratchpad between `<THINKING>` and
+`</THINKING>`: superscript glyphs (`⁰¹²`) stream the per-column carry /
+borrow / comparison state, plain digits are the unnormalized scratch answer,
+and subscript glyphs (`₀₁₂`) stream the leading-zero cleanup count.  The
+final answer follows the closing tag — e.g. `999+1` ends
+`…</THINKING>1000`.
+
+## Intended use and limitations
+
+This model is a demonstration of a computation graph compiled into transformer
+weights.  It is not a general language model or a general-purpose calculator;
+only the input contract above is supported.
+
+## Verification
+
+The examples above are exact reference outputs.  The Modal publishing path
+reloads the emitted checkpoint through stock `transformers`, checks those
+examples plus additional width-limit cases against Python integer arithmetic,
+and refuses to upload on a mismatch.  This is a functional smoke test, not
+exhaustive verification of every allowed expression.
+
+## Size
+
+The checkpoint stores 11.61 GB of dense fp32 weights (2,901,770,240 entries) at
+the example family's shared compile width.  99.99% of those entries are
+exactly zero: the vast majority of the model is unused canvas, so size reflects
+the compile geometry rather than stored knowledge.
+
+The zero entries are not compressed, and dense `transformers` execution still
+pays their memory and compute cost.  CPU execution is supported; allow
+additional RAM beyond the checkpoint size.
+
+## Family
+
+One example of many compiled with torchwright.  Calculator siblings —
+`calculator-simple` (serial arithmetic, depth grows with the digit count),
+`calculator-advanced` (carry-lookahead, near-flat depth), and
+`calculator-scratchpad` (flat depth; the serial work streams out as visible
+thinking tokens) — are published at several digit widths.  Browse the
+[torchwright calculator models](https://huggingface.co/models?search=physicsrob%2Ftorchwright-calculator)
+on Hugging Face.

config.json

@@ -0,0 +1,33 @@
+{
+  "architectures": [
+    "Phi3ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 15,
+  "embd_pdrop": 0.0,
+  "eos_token_id": 16,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 2121,
+  "max_position_embeddings": 512,
+  "model_type": "phi3",
+  "num_attention_heads": 52,
+  "num_hidden_layers": 18,
+  "num_key_value_heads": 52,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": null,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "partial_rotary_factor": 1.0,
+    "rope_theta": 500000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "vocab_size": 33
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "bos_token_id": 15,
+  "eos_token_id": 16,
+  "pad_token_id": 16,
+  "transformers_version": "5.12.1"
+}

model-00001-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85ee87a5e222978318181865aa25e83d3b540628c6e29aed9257bcf978b6e7b6
+size 644710856

model-00002-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1f665c047cbfd4dbc43240be96a95dbad4a497d033e500a920804cef425691f
+size 644710856

model-00003-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bad201a26fe055bc2309b32d12fcff92cc16a1055f80fb0f87b7cb395b4b9c47
+size 644710856

model-00004-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a14a4c3d91dc1d621f6b547f35898832ae98c0683b5f8736ca97384c038b2569
+size 644710856

model-00005-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b7f9c08ba9a5e909f8c49c7eaab9fd67bf5d810e0c6fb7b4972fc2387dfeed4
+size 644710856

model-00006-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98d7818b8dc97e8b4fdad28f09cd57393c3a4cc2631a2fe83264485d9ad57695
+size 644710856

model-00007-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67c3006ec2096bfaf6d616046ef9a22a42b5eed9534a21021fa881edbe587268
+size 644710856

model-00008-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcc13c38b4827b9c23cf9086521b07569a514d55163065a1b01591f441a5af4b
+size 644710856

model-00009-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:174d17f5397befa40bd7ce421e71fd932b8cc61dd9a105891960cb3ab83d4ced
+size 644710856

model-00010-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:609c4450b96e0e32e5265a0b10a1efb2aae4ba3931757099af6193528db6313a
+size 644710856

model-00011-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eb4a903bd41731d62a6cd4b9927776f1aaa64c1f5ecd607074babb3c858ee8a
+size 644710864

model-00012-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4914e1570654418a4f60081ee622ea4f767433f358c83494ee44c784c9a39728
+size 644710864

model-00013-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87e208a5388a0d35685a0a44585058f63cc43aad2e0c74a334ba1d137f58f2be
+size 644710864

model-00014-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49d8c9a196347daa47e4e5ce3f2694b8a3ee45313baab0b0c336ca78d0be70f0
+size 644710864

model-00015-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e2416c7c802308253d0bbc7c0ed504357d21161c4b0b0958b7d2aac8ce8aa85
+size 644710864

model-00016-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fa843a75b38c880e47be88814e46b3da4e1fa2e054ba39b0a71db17fc6fcf25
+size 644710864

model-00017-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bda4a6113b521c062553962373944c078979aa59bddbb3117901cbd070c0ea9
+size 644710864

model-00018-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:849ce82231e9c3463355e59f986464b8eab084e9d988839e7b877cf482545b52
+size 644710864

model-00019-of-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b80ff279f11140637661fd279d70b04644b5b2b5d62d5afd96f27caa18d5aa9
+size 2297864

model.safetensors.index.json

@@ -0,0 +1 @@
+{"metadata": {"total_size": 11607080960}, "weight_map": {"model.layers.0.self_attn.qkv_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.mlp.gate_up_proj.weight": "model-00001-of-00019.safetensors", "model.layers.0.mlp.down_proj.weight": "model-00001-of-00019.safetensors", "model.layers.1.self_attn.qkv_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.self_attn.o_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.mlp.gate_up_proj.weight": "model-00002-of-00019.safetensors", "model.layers.1.mlp.down_proj.weight": "model-00002-of-00019.safetensors", "model.layers.2.self_attn.qkv_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.self_attn.o_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.mlp.gate_up_proj.weight": "model-00003-of-00019.safetensors", "model.layers.2.mlp.down_proj.weight": "model-00003-of-00019.safetensors", "model.layers.3.self_attn.qkv_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.self_attn.o_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.mlp.gate_up_proj.weight": "model-00004-of-00019.safetensors", "model.layers.3.mlp.down_proj.weight": "model-00004-of-00019.safetensors", "model.layers.4.self_attn.qkv_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.self_attn.o_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.mlp.gate_up_proj.weight": "model-00005-of-00019.safetensors", "model.layers.4.mlp.down_proj.weight": "model-00005-of-00019.safetensors", "model.layers.5.self_attn.qkv_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.self_attn.o_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.mlp.gate_up_proj.weight": "model-00006-of-00019.safetensors", "model.layers.5.mlp.down_proj.weight": "model-00006-of-00019.safetensors", "model.layers.6.self_attn.qkv_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.self_attn.o_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.mlp.gate_up_proj.weight": "model-00007-of-00019.safetensors", "model.layers.6.mlp.down_proj.weight": "model-00007-of-00019.safetensors", "model.layers.7.self_attn.qkv_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.self_attn.o_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.mlp.gate_up_proj.weight": "model-00008-of-00019.safetensors", "model.layers.7.mlp.down_proj.weight": "model-00008-of-00019.safetensors", "model.layers.8.self_attn.qkv_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.self_attn.o_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.mlp.gate_up_proj.weight": "model-00009-of-00019.safetensors", "model.layers.8.mlp.down_proj.weight": "model-00009-of-00019.safetensors", "model.layers.9.self_attn.qkv_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.self_attn.o_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.mlp.gate_up_proj.weight": "model-00010-of-00019.safetensors", "model.layers.9.mlp.down_proj.weight": "model-00010-of-00019.safetensors", "model.layers.10.self_attn.qkv_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.self_attn.o_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.mlp.gate_up_proj.weight": "model-00011-of-00019.safetensors", "model.layers.10.mlp.down_proj.weight": "model-00011-of-00019.safetensors", "model.layers.11.self_attn.qkv_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.self_attn.o_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.mlp.gate_up_proj.weight": "model-00012-of-00019.safetensors", "model.layers.11.mlp.down_proj.weight": "model-00012-of-00019.safetensors", "model.layers.12.self_attn.qkv_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.self_attn.o_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.mlp.gate_up_proj.weight": "model-00013-of-00019.safetensors", "model.layers.12.mlp.down_proj.weight": "model-00013-of-00019.safetensors", "model.layers.13.self_attn.qkv_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.self_attn.o_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.mlp.gate_up_proj.weight": "model-00014-of-00019.safetensors", "model.layers.13.mlp.down_proj.weight": "model-00014-of-00019.safetensors", "model.layers.14.self_attn.qkv_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.self_attn.o_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.mlp.gate_up_proj.weight": "model-00015-of-00019.safetensors", "model.layers.14.mlp.down_proj.weight": "model-00015-of-00019.safetensors", "model.layers.15.self_attn.qkv_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.self_attn.o_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.mlp.gate_up_proj.weight": "model-00016-of-00019.safetensors", "model.layers.15.mlp.down_proj.weight": "model-00016-of-00019.safetensors", "model.layers.16.self_attn.qkv_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.self_attn.o_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.mlp.gate_up_proj.weight": "model-00017-of-00019.safetensors", "model.layers.16.mlp.down_proj.weight": "model-00017-of-00019.safetensors", "model.layers.17.self_attn.qkv_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.self_attn.o_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.mlp.gate_up_proj.weight": "model-00018-of-00019.safetensors", "model.layers.17.mlp.down_proj.weight": "model-00018-of-00019.safetensors", "model.embed_tokens.weight": "model-00019-of-00019.safetensors", "model.norm.weight": "model-00019-of-00019.safetensors", "model.layers.0.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.0.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.1.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.1.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.2.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.2.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.3.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.3.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.4.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.4.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.5.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.5.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.6.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.6.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.7.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.7.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.8.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.8.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.9.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.9.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.10.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.10.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.11.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.11.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.12.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.12.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.13.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.13.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.14.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.14.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.15.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.15.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.16.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.16.post_attention_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.17.input_layernorm.weight": "model-00019-of-00019.safetensors", "model.layers.17.post_attention_layernorm.weight": "model-00019-of-00019.safetensors"}}

tokenizer.json

@@ -0,0 +1,130 @@
+{
+  "version": "1.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 15,
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 16,
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": null,
+  "pre_tokenizer": {
+    "type": "Split",
+    "pattern": {
+      "Regex": "[\\s\\S]"
+    },
+    "behavior": "Isolated",
+    "invert": false
+  },
+  "post_processor": {
+    "type": "TemplateProcessing",
+    "single": [
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      }
+    ],
+    "pair": [
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "A",
+          "type_id": 0
+        }
+      },
+      {
+        "SpecialToken": {
+          "id": "<bos>",
+          "type_id": 0
+        }
+      },
+      {
+        "Sequence": {
+          "id": "B",
+          "type_id": 0
+        }
+      }
+    ],
+    "special_tokens": {
+      "<bos>": {
+        "id": "<bos>",
+        "ids": [
+          15
+        ],
+        "tokens": [
+          "<bos>"
+        ]
+      }
+    }
+  },
+  "decoder": {
+    "type": "Fuse"
+  },
+  "model": {
+    "type": "WordLevel",
+    "vocab": {
+      "0": 0,
+      "1": 1,
+      "2": 2,
+      "3": 3,
+      "4": 4,
+      "5": 5,
+      "6": 6,
+      "7": 7,
+      "8": 8,
+      "9": 9,
+      "+": 10,
+      "-": 11,
+      "*": 12,
+      "\n": 13,
+      " ": 14,
+      "<bos>": 15,
+      "<eos>": 16,
+      "<THINKING>": 17,
+      "⁰ ": 18,
+      "¹ ": 19,
+      "² ": 20,
+      "³ ": 21,
+      "⁴ ": 22,
+      "⁵ ": 23,
+      "⁶ ": 24,
+      "₀ ": 25,
+      "₁ ": 26,
+      "₂ ": 27,
+      "₃ ": 28,
+      "₄ ": 29,
+      "₅ ": 30,
+      "₆ ": 31,
+      "</THINKING>": 32
+    },
+    "unk_token": "<unk>"
+  }
+}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null
+}