Upload README.md with huggingface_hub

README.md

@@ -1,7 +1,8 @@
 ---
 license: apache-2.0
 base_model: HuggingFaceTB/SmolLM2-360M-Instruct
-tags: [dispatch-ai, mobile, quantized, gguf, phone-farm-tested]
+tags:
+- speculative-decoding-draft [dispatch-ai, mobile, quantized, gguf, phone-farm-tested]
 pipeline_tag: text-generation
 language: [en]
 ---
@@ -17,3 +18,38 @@ Size: 258 MB. Q4_K_M GGUF for llama.cpp.
 ./llama-cli -m model.gguf -p "Hello" -n 100 -t 4 -c 512
 ```
 🌐 [dispatchAI on HuggingFace](https://huggingface.co/dispatchAI)
+
+
+## Speculative Decoding Draft Model
+
+This model is optimized for use as a **draft model** in speculative decoding setups.
+
+### What is speculative decoding?
+Speculative decoding pairs a small, fast "draft" model with a larger "target" model.
+The draft model proposes tokens that the target model verifies in parallel, achieving
+2-3x speedup with zero quality loss.
+
+### Why this model?
+- **Small and fast**: Sub-1B parameters = minimal draft overhead
+- **Mobile-optimized**: Already quantized and pruned for edge deployment
+- **Same family**: Pairs naturally with larger models of the same architecture
+
+### Usage with vLLM
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(
+    model="target-model-7b",
+    speculative_model="dispatchAI/SmolLM2-360M-Instruct-mobile",
+    num_speculative_tokens=5,
+)
+```
+
+### Usage with transformers
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+target = AutoModelForCausalLM.from_pretrained("target-model-7b")
+draft = AutoModelForCausalLM.from_pretrained("dispatchAI/SmolLM2-360M-Instruct-mobile")
+# See transformers docs for assisted_generation
+```