| --- |
| license: apache-2.0 |
| base_model: allenai/Olmo-3-1025-7B |
| pipeline_tag: text-generation |
| tags: [olmo, reasoning, sft, hyperbolic, think] |
| --- |
| |
| # zimplex/olmo7-think-sft-hyp |
|
|
| Thinking-SFT of `allenai/Olmo-3-1025-7B` on `allenai/Dolci-Think-SFT-7B` with a **hyperbolic (Lorentzian) output |
| head** (output-only, curvature `c = 0.05`), trained with the `<think>` ChatML template. |
|
|
| ## IMPORTANT — the hyperbolic head is not the standard `lm_head` |
| A plain `AutoModelForCausalLM.from_pretrained(...)` loads the **euclidean** head and gives the WRONG |
| model. To use the hyperbolic head you must load `hyp_head_state.pt` and swap the output head using |
| the training repo's code (`scripts/eval_think.py` → `load_hyp_head` / `HyperbolicLMHead`; the vLLM |
| path mirrors it). `hyp_head_state.pt` is authoritative for curvature / scale / projection / adapter. |
|
|
| ## Prompting & grading |
| Use the repo's `<think>` template (`openrlhf/utils/think_chat.py`); greedy decoding. Take the answer |
| **after** `</think>` (last `\boxed{}` / `####` / number) and grade math with `math_verify`. |
|
|
| ## Files |
| Standard HF Olmo3 weights + tokenizer + `chat_template.jinja`, plus `hyp_head_state.pt` |
| (`vocab_emb`, `down_proj`, `log_input_scale`, `log_temp`, `curvature`). |
