File size: 1,589 Bytes
59f2397 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
license: apache-2.0
base_model: ornith_9b
datasets:
- SupraLabs/reasoning-summaries-61k
tags:
- reasoning
- fine-tuned
- qwen3_5
---
# ornith_9b_enhancedreasoning
A fine-tune of **Ornith 9B** aimed at strengthening its reasoning ability beyond the base release.
## Training data
Fine-tuned on [`SupraLabs/reasoning-summaries-61k`](https://huggingface.co/datasets/SupraLabs/reasoning-summaries-61k), a 61k-sample dataset of reasoning traces paired with structured summaries covering math, code, tool-use, and multi-step problem solving.
## Training setup
| Hyperparameter | Value |
|---|---|
| Learning rate | 5e-5 |
| LR scheduler | cosine |
| Epochs | 3.0 |
| Batch size | 2 |
| Gradient accumulation | 8 |
| Max gradient norm | 1.0 |
| Cutoff length | 10200 |
| Compute type | bf16 |
| Val size | 0 |
Loss dropped sharply in the first ~200 steps and settled into a steady decline from ~0.65 to ~0.50 over roughly 11k steps, with no signs of divergence or overfitting.
## Format
Released as safetensors checkpoints (BF16), ready to drop into standard `transformers`-based loading pipelines.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "jamesesqueleto/ornith_9b_enhancedreasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
```
## Notes
This is a fine-tune, not an from-scratch model — general capabilities and limitations of the base Ornith 9B model still apply. Feedback and issues welcome via the Community tab.
|