metadata
license: apache-2.0
base_model: ornith_9b
datasets:
- SupraLabs/reasoning-summaries-61k
tags:
- reasoning
- fine-tuned
- qwen3_5
ornith_9b_enhancedreasoning
A fine-tune of Ornith 9B aimed at strengthening its reasoning ability beyond the base release.
Training data
Fine-tuned on SupraLabs/reasoning-summaries-61k, a 61k-sample dataset of reasoning traces paired with structured summaries covering math, code, tool-use, and multi-step problem solving.
Training setup
| Hyperparameter | Value |
|---|---|
| Learning rate | 5e-5 |
| LR scheduler | cosine |
| Epochs | 3.0 |
| Batch size | 2 |
| Gradient accumulation | 8 |
| Max gradient norm | 1.0 |
| Cutoff length | 10200 |
| Compute type | bf16 |
| Val size | 0 |
Loss dropped sharply in the first ~200 steps and settled into a steady decline from ~0.65 to ~0.50 over roughly 11k steps, with no signs of divergence or overfitting.
Format
Released as safetensors checkpoints (BF16), ready to drop into standard transformers-based loading pipelines.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "jamesesqueleto/ornith_9b_enhancedreasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
Notes
This is a fine-tune, not an from-scratch model — general capabilities and limitations of the base Ornith 9B model still apply. Feedback and issues welcome via the Community tab.