Instructions to use Shaq2/distill-student-extract with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Shaq2/distill-student-extract with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-0.8B-Base") model = PeftModel.from_pretrained(base_model, "Shaq2/distill-student-extract") - Notebooks
- Google Colab
- Kaggle
distill-student-structured โ LoRA adapter
0.8B student adapter for the Distill gateway โ JSON field extraction from support emails (product/issue/priority). Serves beside a shared base GGUF via llama.cpp multi-LoRA.
Contents
adapter_model.safetensors+adapter_config.jsonโ PEFT adapteradapter.ggufโ same adapter converted for llama.cpp (convert_lora_to_gguf.py)tokenizer.json,tokenizer_config.json,chat_template.jinja
LoRA: r=16, alpha=32, targets q/k/v/o/gate/up/down. Base: Qwen/Qwen3.5-0.8B-Base
(Apache-2.0). Trained on Kaggle (see kaggle/train.py).
Eval (200-row golden, student vs incumbent gpt-5-nano)
| metric | student | incumbent |
|---|---|---|
| field-F1 / schema% | 0.691 / 100% | 0.588 / 100% |
| $/1k req | ~$0.014โ0.024 | $0.11โ0.28 |
Golden labels are teacher-written โ field-F1 pending human review (see Distill repo README).
Serving
Per-request adapter select (llama.cpp): {"lora":[{"id":<sorted-adapter-idx>,"scale":1.0}],"chat_template_kwargs":{"enable_thinking":false}} โ see modal/distill_serve.py.
Training data
Shaq2/distill-structured-extraction โ training-data license: apache-2.0
(provenance + per-source licenses in that dataset card and
data/SOURCES.md). Teacher labels generated by cerebras:gpt-oss-120b.
- Downloads last month
- 55
We're not able to determine the quantization variants.
Model tree for Shaq2/distill-student-extract
Base model
Qwen/Qwen3.5-0.8B-Base