Instructions to use Shaq2/distill-student-intent with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Shaq2/distill-student-intent with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-0.8B-Base") model = PeftModel.from_pretrained(base_model, "Shaq2/distill-student-intent") - Notebooks
- Google Colab
- Kaggle
distill-student-intent โ LoRA adapter
0.8B student adapter for the Distill gateway โ banking-intent classification (Banking77 TEST golden, 77 labels). Serves beside a shared base GGUF via llama.cpp multi-LoRA.
Contents
adapter_model.safetensors+adapter_config.jsonโ PEFT adapteradapter.ggufโ same adapter converted for llama.cpp (convert_lora_to_gguf.py)tokenizer.json,tokenizer_config.json,chat_template.jinja
LoRA: r=16, alpha=32, targets q/k/v/o/gate/up/down. Base: Qwen/Qwen3.5-0.8B-Base
(Apache-2.0). Trained on Kaggle (see kaggle/train.py).
Eval (200-row golden, student vs incumbent gpt-5-nano)
| metric | student | incumbent |
|---|---|---|
| field-F1 n/a | acc 83.5% / macro-F1 0.802 | acc 73.5% / 0.686 |
| $/1k req | ~$0.014โ0.024 | $0.11โ0.28 |
Golden = official Banking77 TEST split (zero train overlap, enforced).
Serving
Per-request adapter select (llama.cpp): {"lora":[{"id":<sorted-adapter-idx>,"scale":1.0}],"chat_template_kwargs":{"enable_thinking":false}} โ see modal/distill_serve.py.
Training data
Shaq2/distill-intent-routing โ training-data license: cc-by-4.0
(provenance + per-source licenses in that dataset card and
data/SOURCES.md). Teacher labels generated by cerebras:gpt-oss-120b.
- Downloads last month
- 55
We're not able to determine the quantization variants.
Model tree for Shaq2/distill-student-intent
Base model
Qwen/Qwen3.5-0.8B-Base