Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,46 @@
+---
+license: apache-2.0
+tags:
+  - speculative-decoding
+  - knowledge-distillation
+  - mistral
+base_model: Lite-Mistral-150M-v2-Instruct
+---
+
+# drafter-understanding
+
+Domain-specific draft model for speculative decoding, trained on **Understanding** tasks.
+
+## Model Details
+
+| Parameter | Value |
+|-----------|-------|
+| Base model | Lite-Mistral-150M-v2-Instruct (156M params) |
+| Architecture | MistralForCausalLM |
+| Target model | TurboSparse-Mistral-Instruct (7B) |
+| Domain | Understanding (21 Flan clusters) |
+| Training samples | 395K |
+| Epochs | 4.5 (early stop from 25) |
+| Training time | 4.3 hours (1x RTX 3090) |
+| Loss | Mixed: 0.5 x CE + 0.5 x KL (T=1.0) |
+| Final eval_loss | 1.193 |
+| Final top1_accuracy | 65.05% |
+| Overlap Area (AR proxy) | **0.7558** on own domain |
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("mikhialo/drafter-understanding")
+tokenizer = AutoTokenizer.from_pretrained("mikhialo/drafter-understanding")
+```
+
+## Training Data
+
+[mikhialo/domain-aware-sd-synthetic](https://huggingface.co/datasets/mikhialo/domain-aware-sd-synthetic)
+
+## Citation
+
+Part of the Domain-Aware Speculative Decoding research project:
+[GitHub](https://github.com/MikhailRudenk0/Domain-Aware-SD)