README.md

---
license: apache-2.0
base_model: allenai/Olmo-3-1025-7B
pipeline_tag: text-generation
tags: [olmo, reasoning, sft, hyperbolic, think]
---

# zimplex/olmo7-think-sft-hyp

Thinking-SFT of `allenai/Olmo-3-1025-7B` on `allenai/Dolci-Think-SFT-7B` with a **hyperbolic (Lorentzian) output
head** (output-only, curvature `c = 0.05`), trained with the `<think>` ChatML template.

## IMPORTANT — the hyperbolic head is not the standard `lm_head`
A plain `AutoModelForCausalLM.from_pretrained(...)` loads the **euclidean** head and gives the WRONG
model. To use the hyperbolic head you must load `hyp_head_state.pt` and swap the output head using
the training repo's code (`scripts/eval_think.py` → `load_hyp_head` / `HyperbolicLMHead`; the vLLM
path mirrors it). `hyp_head_state.pt` is authoritative for curvature / scale / projection / adapter.

## Prompting & grading
Use the repo's `<think>` template (`openrlhf/utils/think_chat.py`); greedy decoding. Take the answer
**after** `</think>` (last `\boxed{}` / `####` / number) and grade math with `math_verify`.

## Files
Standard HF Olmo3 weights + tokenizer + `chat_template.jinja`, plus `hyp_head_state.pt`
(`vocab_emb`, `down_proj`, `log_input_scale`, `log_temp`, `curvature`).