Upload folder using huggingface_hub

README.md

@@ -0,0 +1,120 @@
+---
+license: mit
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- function-calling
+- tool-calling
+- on-device
+- small-model
+- grammar-constrained-decoding
+library_name: pytorch
+model-index:
+- name: thimble-v6
+  results:
+  - task:
+      type: text-generation
+      name: Function calling (ordered strict exact match)
+    dataset:
+      name: Seal-Tools in-domain
+      type: seal-tools
+    metrics:
+    - type: exact_match
+      value: 33.1
+      name: Seal-Tools in-domain
+  - task:
+      type: text-generation
+      name: Function calling (ordered strict exact match)
+    dataset:
+      name: Mobile Actions
+      type: mobile-actions
+    metrics:
+    - type: exact_match
+      value: 86.3
+      name: Mobile Actions
+  - task:
+      type: text-generation
+      name: Function calling (ordered strict exact match)
+    dataset:
+      name: DroidCall
+      type: droidcall
+    metrics:
+    - type: exact_match
+      value: 52.5
+      name: DroidCall
+---
+
+# Thimble v6
+
+**Thimble** is a **48.12M-parameter** factorized tool-calling model that beats
+[Needle 2](https://cactuscompute.com/needle) (Cactus Compute, 45M, 153B training
+tokens) on **three of its five published benchmark tables** — including
+Seal-Tools, the suite its evaluation is built around — while training on
+**~1B tokens (150x less)** for a total cost of about $260.
+
+| Suite | Thimble v6 | Needle 2 (45M) |
+|---|---|---|
+| **Seal-Tools in-domain** | **33.1** | 32.6 |
+| **Mobile Actions** | **86.3** | 63.7 |
+| **DroidCall** | **52.5** | 17.0 |
+| Seal-Tools out-of-domain | 28.1 | **28.7** |
+| BFCL v4 single-turn | 23.5 | **42.6** |
+
+Metric: ordered strict exact match — a row passes only if function names, call
+order, and every argument value match exactly. Needle 2's numbers are from
+their published tables. The Seal-in margin (+0.5 on 700 rows) is within
+sampling noise and is stated as measured, not as a decisive gap. Full
+disclosures (parameter-class matching, a champion-selection protocol failure,
+the DroidCall split caveat) live in the
+[GitHub repo](https://github.com/nikshepsvn/tiny-toolcall)'s RESULTS.md.
+
+## How it works
+
+- **Deep-thin gated trunk**: d=448, 20 layers, GQA 8/4, SwiGLU x2.0, QK-norm,
+  sandwich RMSNorm, tied embeddings; 16,384-token BPE with digits as
+  singletons.
+- **Grammar-constrained decoding**: JSON structure and argument keys are
+  force-fed from the tool schemas; the model is consulted at exactly five
+  choice points (refuse, tool name, optional-include, value, stop). Malformed
+  JSON and hallucinated parameter names are unreachable by construction —
+  100% well-formed output.
+- **Plan-conditioned retrieval**: the legal tool-name set is re-retrieved
+  between emitted calls, conditioned on the calls already made.
+- **Error-driven training data**: the v6 corpus was synthesized against the
+  measured failure buckets of the previous version (spurious optional
+  arguments, wrong-slot entity binding, date canonicalization), then
+  **annealed into the learning-rate decay phase** of a continued run — the
+  controlled twin experiment showing anneal-beats-scratch for corrective data
+  is in the repo.
+
+## Files
+
+- `thimble-v6.pt` — the released checkpoint (`v6c_ema`): PyTorch
+  `state_dict` plus config dict (`torch.load(..., weights_only=False)`,
+  keys: `model`, `cfg`).
+- `tokenizer.json` — 16,384-entry BPE vocabulary + merges.
+
+## Usage
+
+Inference requires the grammar decoder from the repo (the checkpoint is a
+plain transformer; the decoding harness is where the guarantees live):
+
+```bash
+git clone https://github.com/nikshepsvn/tiny-toolcall
+cd tiny-toolcall && uv venv && uv pip install -e .
+# place thimble-v6.pt in checkpoints/, tokenizer.json in data/
+python scripts/final_eval.py --ckpt thimble-v6 --suite seal-tools-in
+```
+
+## Training provenance
+
+Trained on public function-calling corpora (xlam, ToolACE, Dolci, Glaive,
+official benchmark train splits) plus validated synthetic data
+(deepseek-v4-flash teacher, stepwise-validated, evidence-filtered). Every
+training row passed an 8-gram contamination firewall against every evaluation
+query of every reported suite, BFCL included. Nothing was ever tuned on an
+eval set; the full experimental record — including every negative result —
+is in the repo.
+
+MIT. Built by one person and an AI assistant in about a week of evenings.

thimble-v6.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10b9e5530cc5ac434a838299f67ed4c165df102166c9d6977501a37eeaf80f01
+size 192586040