Model Card for general_knowledge_model

Post-trained version of Qwen/Qwen3-1.7B for the General Knowledge benchmark of EPFL CS-552 — Modern NLP (Spring 2026), team CentraleSupéchec.

The task is closed-book multiple-choice QA (2–20 options). The model reasons inside a <think> ... </think> block and ends its reply with the answer wrapped in \boxed{LETTER}, which is parsed for pass@1 scoring.

Training

The model is trained with Rejection Fine-Tuning (RFT) — STaR-style self-distillation — with an answer-only loss:

1. Sample n=8 completions (T=0.7) from the base model over a ~4.7k-question pool of GPQA and MMLU-Pro (excluding Math/CS).
2. Keep the 722 questions the base fails at pass@1 but solves under repeated sampling, producing self-generated correct reasoning traces.
3. Fine-tune a LoRA adapter (r=16, α=32) with the cross-entropy loss masked to the \boxed{} answer span only — the <think> reasoning conditions the forward pass but receives no gradient. This preserves the model's pretrained reasoning while sharpening answer commitment and output formatting.

The chat template (baked into the tokenizer) enforces a strict \boxed{LETTER} output and a 16,384-token reasoning budget.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "cs-552-2026-centralesupechec/general_knowledge_model"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="cuda")

question = (
    "Which of the following is the capital of Australia?\n\n"
    "Choices:\nA. Sydney\nB. Melbourne\nC. Canberra\nD. Perth"
)
inputs = tok.apply_chat_template(
    [{"role": "user", "content": question}],
    add_generation_prompt=True, return_tensors="pt",
).to(model.device)

out = model.generate(inputs, max_new_tokens=16384, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
# ... reasoning ... \boxed{C}

For vLLM, mirror the CI: apply the model's chat template, seed=42, max_new_tokens=16384, temperature=0.6, top_p=0.95, top_k=20.

Generation config

max_new_tokens: 16384 · temperature: 0.6 · top_p: 0.95 · top_k: 20 · do_sample: true. The 16k budget is essential: it removes the format failures that occur when reasoning is truncated before the boxed answer.

Evaluation

pass@1 on held-out sets disjoint from training (n=4, 16k tokens):

Set	pass@1
650-question MMLU sweep (26 subjects)	~0.74
Internal 100-question expert set	~0.59

See the project report and code for the full comparison against the base model, full-trace SFT, and GRPO.

Framework versions

• Transformers 5.7.0
• PyTorch 2.10.0+cu128
• TRL 0.12, PEFT 0.13

Citation

@inproceedings{zelikman2022star,
  title     = {{STaR}: Bootstrapping Reasoning With Reasoning},
  author    = {Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah D.},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2022}
}