README.md

---
library_name: transformers
license: apache-2.0
base_model: Qwen/Qwen3-1.7B
pipeline_tag: text-generation
model_name: general_knowledge_model
tags:
  - general-knowledge
  - multiple-choice
  - reasoning
  - rejection-sampling
  - rft
  - lora
  - cs-552
---

# Model Card for `general_knowledge_model`

Post-trained version of [`Qwen/Qwen3-1.7B`](https://huggingface.co/Qwen/Qwen3-1.7B)
for the **General Knowledge** benchmark of EPFL **CS-552 — Modern NLP** (Spring 2026),
team CentraleSupéchec.

The task is **closed-book multiple-choice QA** (2–20 options). The model reasons
inside a `<think> ... </think>` block and ends its reply with the answer wrapped in
`\boxed{LETTER}`, which is parsed for `pass@1` scoring.

## Training

The model is trained with **Rejection Fine-Tuning (RFT)** — STaR-style
self-distillation — with an **answer-only loss**:

1. Sample `n=8` completions (`T=0.7`) from the base model over a ~4.7k-question
   pool of **GPQA** and **MMLU-Pro** (excluding Math/CS).
2. Keep the 722 questions the base fails at `pass@1` but solves under repeated
   sampling, producing self-generated correct reasoning traces.
3. Fine-tune a **LoRA** adapter (`r=16`, `α=32`) with the cross-entropy loss
   **masked to the `\boxed{}` answer span only** — the `<think>` reasoning
   conditions the forward pass but receives no gradient. This preserves the
   model's pretrained reasoning while sharpening answer commitment and output
   formatting.

The chat template (baked into the tokenizer) enforces a strict `\boxed{LETTER}`
output and a 16,384-token reasoning budget.

## Quick start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "cs-552-2026-centralesupechec/general_knowledge_model"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="cuda")

question = (
    "Which of the following is the capital of Australia?\n\n"
    "Choices:\nA. Sydney\nB. Melbourne\nC. Canberra\nD. Perth"
)
inputs = tok.apply_chat_template(
    [{"role": "user", "content": question}],
    add_generation_prompt=True, return_tensors="pt",
).to(model.device)

out = model.generate(inputs, max_new_tokens=16384, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
# ... reasoning ... \boxed{C}
```

For vLLM, mirror the CI: apply the model's chat template, `seed=42`,
`max_new_tokens=16384`, `temperature=0.6`, `top_p=0.95`, `top_k=20`.

## Generation config

`max_new_tokens: 16384` · `temperature: 0.6` · `top_p: 0.95` · `top_k: 20` ·
`do_sample: true`. The 16k budget is essential: it removes the format failures
that occur when reasoning is truncated before the boxed answer.

## Evaluation

`pass@1` on held-out sets disjoint from training (n=4, 16k tokens):

| Set | pass@1 |
|---|---|
| 650-question MMLU sweep (26 subjects) | ~0.74 |
| Internal 100-question expert set | ~0.59 |

See the project report and code for the full comparison against the base model,
full-trace SFT, and GRPO.

## Framework versions

- Transformers 5.7.0
- PyTorch 2.10.0+cu128
- TRL 0.12, PEFT 0.13

## Citation

```bibtex
@inproceedings{zelikman2022star,
  title     = {{STaR}: Bootstrapping Reasoning With Reasoning},
  author    = {Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah D.},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2022}
}
```