| --- |
| license: apache-2.0 |
| base_model: ornith_9b |
| datasets: |
| - SupraLabs/reasoning-summaries-61k |
| tags: |
| - reasoning |
| - fine-tuned |
| - qwen3_5 |
| --- |
| |
| # ornith_9b_enhancedreasoning |
|
|
| A fine-tune of **Ornith 9B** aimed at strengthening its reasoning ability beyond the base release. |
|
|
| ## Training data |
|
|
| Fine-tuned on [`SupraLabs/reasoning-summaries-61k`](https://huggingface.co/datasets/SupraLabs/reasoning-summaries-61k), a 61k-sample dataset of reasoning traces paired with structured summaries covering math, code, tool-use, and multi-step problem solving. |
|
|
| ## Training setup |
|
|
| | Hyperparameter | Value | |
| |---|---| |
| | Learning rate | 5e-5 | |
| | LR scheduler | cosine | |
| | Epochs | 3.0 | |
| | Batch size | 2 | |
| | Gradient accumulation | 8 | |
| | Max gradient norm | 1.0 | |
| | Cutoff length | 10200 | |
| | Compute type | bf16 | |
| | Val size | 0 | |
|
|
| Loss dropped sharply in the first ~200 steps and settled into a steady decline from ~0.65 to ~0.50 over roughly 11k steps, with no signs of divergence or overfitting. |
|
|
| ## Format |
|
|
| Released as safetensors checkpoints (BF16), ready to drop into standard `transformers`-based loading pipelines. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "jamesesqueleto/ornith_9b_enhancedreasoning" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto") |
| ``` |
|
|
| ## Notes |
|
|
| This is a fine-tune, not an from-scratch model — general capabilities and limitations of the base Ornith 9B model still apply. Feedback and issues welcome via the Community tab. |
|
|
