-
Regenerate prebuilt CQ4 bundle from the aligned repo state
-
Align repo with other models: weights+config+tokenizer only (needle code lives in cactus)
-
Merge HF-format model into repo (config.json keeps JAX identity under jax_config)
-
Add prebuilt CQ4 runnable bundle
-
Upload config.json with huggingface_hub
Henry Ndubuaku
-
Upload tokenizer/needle.vocab with huggingface_hub
Henry Ndubuaku
-
Upload tokenizer/needle.model with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Upload README.md with huggingface_hub
Henry Ndubuaku
-
Create README.md
Henry Ndubuaku
---
license: mit
library_name: jax
tags:
- function-calling
- tool-use
- encoder-decoder
- edge
- on-device
- jax
- flax
datasets:
- Cactus-Compute/tool-calls
---
# Needle
A 26M parameter encoder-decoder transformer for on-device function calling, built on a "Simple Attention Network" architecture (no feedforward layers).
Distilled from Gemini 3.1 Flash Lite. Runs at 6000 tok/s prefill and 1200 tok/s decode on [Cactus](https://github.com/cactus-compute/cactus).
## Model Details
| | |
|---|---|
| Parameters | 26M |
| Architecture | Encoder-decoder, pure attention (no FFN) |
| Encoder | 12 layers, GQA (8H/4KV), RoPE, gated residuals |
| Decoder | 8 layers, self-attn + cross-attn, gated residuals |
| d_model | 512 |
| Vocab | 8192 (SentencePiece BPE) |
| Norm | ZCRMSNorm (zero-centered, init=0) |
| Precision | bfloat16 (INT4 QAT during training) |
| Pretraining | 200B tokens on 16x TPU v6e (27hrs) |
| Post-training | 2B tokens of function call data (45mins) |
## Arch…
-
Upload needle.pkl
Henry Ndubuaku
-
Delete needle.pkl
Henry Ndubuaku
-
Upload needle.pkl
Henry Ndubuaku
-
Clean repo for public release
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_768_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_8_768_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_768_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_768_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_4_768_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_1024_4096_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_1024_4096_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_1024_4096_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_768_3072_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_6_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_8_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_8_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_512_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_1024_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_2048_best.pkl with huggingface_hub
Henry Ndubuaku
-
Upload needle_12_512_1024_best.pkl with huggingface_hub
Henry Ndubuaku