distill-student-title โ€” LoRA adapter

0.8B student adapter for the Distill gateway โ€” short ticket-title generation (teacher-synthesized tickets). Serves beside a shared base GGUF via llama.cpp multi-LoRA.

Contents

  • adapter_model.safetensors + adapter_config.json โ€” PEFT adapter
  • adapter.gguf โ€” same adapter converted for llama.cpp (convert_lora_to_gguf.py)
  • tokenizer.json, tokenizer_config.json, chat_template.jinja

LoRA: r=16, alpha=32, targets q/k/v/o/gate/up/down. Base: Qwen/Qwen3.5-0.8B-Base (Apache-2.0). Trained on Kaggle (see kaggle/train.py).

Eval (200-row golden, student vs incumbent gpt-5-nano)

metric student incumbent
token-F1 0.675 / 13.5% exact 0.638 / 11.5%
$/1k req ~$0.014โ€“0.024 $0.11โ€“0.28

Golden labels are teacher-written โ€” same teacher as train; NOT human-verified (student NOT promoted after blind review).

Serving

Per-request adapter select (llama.cpp): {"lora":[{"id":<sorted-adapter-idx>,"scale":1.0}],"chat_template_kwargs":{"enable_thinking":false}} โ€” see modal/distill_serve.py.

Training data

Shaq2/distill-title-generation โ€” training-data license: apache-2.0 (provenance + per-source licenses in that dataset card and data/SOURCES.md). Teacher labels generated by cerebras:gpt-oss-120b.

Downloads last month
53
GGUF
Model size
6.39M params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Shaq2/distill-student-title

Adapter
(67)
this model