distill-student-structured โ€” LoRA adapter

0.8B student adapter for the Distill gateway โ€” JSON field extraction from support emails (product/issue/priority). Serves beside a shared base GGUF via llama.cpp multi-LoRA.

Contents

  • adapter_model.safetensors + adapter_config.json โ€” PEFT adapter
  • adapter.gguf โ€” same adapter converted for llama.cpp (convert_lora_to_gguf.py)
  • tokenizer.json, tokenizer_config.json, chat_template.jinja

LoRA: r=16, alpha=32, targets q/k/v/o/gate/up/down. Base: Qwen/Qwen3.5-0.8B-Base (Apache-2.0). Trained on Kaggle (see kaggle/train.py).

Eval (200-row golden, student vs incumbent gpt-5-nano)

metric student incumbent
field-F1 / schema% 0.691 / 100% 0.588 / 100%
$/1k req ~$0.014โ€“0.024 $0.11โ€“0.28

Golden labels are teacher-written โ€” field-F1 pending human review (see Distill repo README).

Serving

Per-request adapter select (llama.cpp): {"lora":[{"id":<sorted-adapter-idx>,"scale":1.0}],"chat_template_kwargs":{"enable_thinking":false}} โ€” see modal/distill_serve.py.

Training data

Shaq2/distill-structured-extraction โ€” training-data license: apache-2.0 (provenance + per-source licenses in that dataset card and data/SOURCES.md). Teacher labels generated by cerebras:gpt-oss-120b.

Downloads last month
55
GGUF
Model size
6.39M params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Shaq2/distill-student-extract

Adapter
(67)
this model