ornith_9b_enhancedreasoning

A fine-tune of Ornith 9B aimed at strengthening its reasoning ability beyond the base release.

Training data

Fine-tuned on SupraLabs/reasoning-summaries-61k, a 61k-sample dataset of reasoning traces paired with structured summaries covering math, code, tool-use, and multi-step problem solving.

Training setup

Hyperparameter	Value
Learning rate	5e-5
LR scheduler	cosine
Epochs	3.0
Batch size	2
Gradient accumulation	8
Max gradient norm	1.0
Cutoff length	10200
Compute type	bf16
Val size	0

Loss dropped sharply in the first ~200 steps and settled into a steady decline from ~0.65 to ~0.50 over roughly 11k steps, with no signs of divergence or overfitting.

Format

Released as safetensors checkpoints (BF16), ready to drop into standard transformers-based loading pipelines.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jamesesqueleto/ornith_9b_enhancedreasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

Notes

This is a fine-tune, not an from-scratch model — general capabilities and limitations of the base Ornith 9B model still apply. Feedback and issues welcome via the Community tab.