build-small-hackathon/deku-gguf

Deku — GGUF (llama.cpp)

GGUF builds of build-small-hackathon/deku, the One for All student: a Qwen2.5-0.5B distilled from 6 teachers via gated CKA geometry distillation. The LoRA adapter is merged into the base, then converted with llama.cpp's convert_hf_to_gguf.py.

Files

File	Size	Use
deku-q8_0.gguf	~531 MB	what the Space serves — near-lossless, CPU-friendly
deku-f16.gguf	~994 MB	archival full-precision build
gating.npz	~22 KB	the teacher-gating head as numpy (weight 6×896, bias 6)

Run

llama-cli -m deku-q8_0.gguf -p "Explain gradient descent in one sentence."

from llama_cpp import Llama
llm = Llama(model_path="deku-q8_0.gguf", n_ctx=2048)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Why is the sky blue?"}]
)["choices"][0]["message"]["content"])

Teacher gating without torch

gating.npz lets you reproduce the live "teacher influence" meters from the Space using only numpy on a mean-pooled embedding from llama.cpp:

import numpy as np
g = np.load("gating.npz")          # g["weight"] (6, 896), g["bias"] (6,)
def gate(emb):                     # emb: 896-dim pooled embedding
    z = g["weight"] @ emb + g["bias"]
    e = np.exp(z - z.max())
    return e / e.sum()             # softmax over the 6 teachers

Teacher order: qwen, smollm, phi, gemma, minicpm, nemotron.