Publish UMSR reasoning model

README.md

@@ -2,23 +2,182 @@
 language:
 - en
 library_name: transformers
+pipeline_tag: text-generation
 datasets:
 - NorthernTribe-Research/UMSR-v1
 tags:
 - reasoning
-- autonomous-training
+- instruction-following
+- structured-output
+- math
+- science
+- logic
+- strategy
 ---
 
-# UMSR Reasoner 7B
+# UMSR-Reasoner-7B
 
-Standalone reasoning model trained from UMSR-v1 using the autonomous trainer Space.
+## Overview
 
-- Dataset: https://huggingface.co/datasets/NorthernTribe-Research/UMSR-v1
-- Base model: `sshleifer/tiny-gpt2`
-- Model repo: `https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B`
+UMSR-Reasoner-7B is a standalone 7B reasoning model for structured multi-step problem solving.
+
+It is optimized for tasks that require:
+
+- explicit reasoning traces
+- deterministic final-answer formatting
+- consistent performance across math, science, logic, and strategy domains
+
+## Dataset
+
+- Primary dataset: https://huggingface.co/datasets/NorthernTribe-Research/UMSR-v1
+
+## Training Strategy
+
+- student architecture: `NorthernTribe-Research/UMSR-Reasoner-7B`
+- teacher architecture: `NorthernTribe-Research/UMSR-Reasoner-7B` self-distillation (default)
+- objective: blended CE + KL distillation with temperature and weight scheduling
+- continuity: checkpointed autonomous training cycles with resume support
 
 ## Output Contract
 
-Use:
+For best reliability, prompt the model to end with:
+
+```text
+<final_answer>...</final_answer>
+```
+
+Optional reasoning can be requested in:
+
+```text
+<reasoning>...</reasoning>
+```
+
+## Model Tree
+
+| Variant | Repository | Purpose |
+|---|---|---|
+| Base FP model | `NorthernTribe-Research/UMSR-Reasoner-7B` | Primary inference and fine-tuning target |
+| INT8 runtime profile | `NorthernTribe-Research/UMSR-Reasoner-7B-INT8` | Lower-memory deployment |
+| NF4 runtime profile | `NorthernTribe-Research/UMSR-Reasoner-7B-NF4` | Max compression for constrained GPUs |
+| Smoke INT8 profile | `NorthernTribe-Research/UMSR-Reasoner-7B-Smoke-INT8` | Fast CI/smoke validation profile linked to the main model tree |
+
+## Quickstart
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+messages = [
+    {"role": "system", "content": "Solve step by step and finish with <final_answer>...</final_answer>."},
+    {"role": "user", "content": "If 3x + 5 = 20, what is x?"},
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(
+    inputs,
+    max_new_tokens=256,
+    temperature=0.2,
+    top_p=0.9,
+)
+
+print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+## Quantized Runtime
+
+### INT8
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
+bnb_config = BitsAndBytesConfig(load_in_8bit=True)
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    quantization_config=bnb_config,
+)
+```
+
+### NF4
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_compute_dtype=torch.bfloat16,
+)
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    quantization_config=bnb_config,
+)
+```
+
+## Llamafile Packaging
+
+For single-binary deployment, use:
+
+```bash
+python scripts/create_llamafile.py \
+  --gguf /path/to/UMSR-Reasoner-7B.Q4_K_M.gguf \
+  --runtime-bin tools/llamafile \
+  --output dist/UMSR-Reasoner-7B.Q4_K_M.llamafile \
+  --force
+```
+
+## Code-Aware Robust Evaluation
+
+`scripts/eval_reasoner.py` supports code-focused robustness checks:
+
+- code-task detection
+- Python code-block syntax validation
+- optional row-level unit-test execution
+- optional TensorFlow-backed multi-candidate scorer
+
+## Trainer Integration
+
+An autonomous trainer Space is available for continuous training cycles against UMSR-v1. It supports:
+
+- teacher-student distillation mode with configurable in-house teacher model
+- live run telemetry (`live_progress.json`, `live_events.jsonl`) for real-time monitoring
+- scheduled or continuous operation
+- checkpoint auto-resume (`UMSR_RESUME_FROM_CHECKPOINT=auto`)
+- warmup-step and warmup-ratio control
+- push-to-hub automation
+- run monitoring through live dashboard and logs
+
+## Best Practices
+
+- keep prompts explicit about output tags
+- validate final answers for high-stakes workflows
+- prefer domain-specific evaluation before production deployment
+
+## Limitations
 
-`<final_answer>...</final_answer>`
+- reasoning text may contain errors even when final format is correct
+- quality depends on prompt clarity and task scope
+- not suitable as a sole decision-maker for legal, medical, or safety-critical use

config.json

@@ -18,7 +18,7 @@
   "n_inner": null,
   "n_layer": 2,
   "n_positions": 1024,
-  "pad_token_id": 50256,
+  "pad_token_id": null,
   "reorder_and_upcast_attn": false,
   "resid_pdrop": 0.1,
   "scale_attn_by_inverse_layer_idx": false,
@@ -34,7 +34,7 @@
       "max_length": 50
     }
   },
-  "tie_word_embeddings": false,
+  "tie_word_embeddings": true,
   "transformers_version": "5.2.0",
   "use_cache": false,
   "vocab_size": 50257

metrics/eval_metrics.json

@@ -1,8 +1,8 @@
 {
   "epoch": 1.0,
-  "eval_loss": 10.716222763061523,
-  "eval_runtime": 1.9392,
-  "eval_samples": 64,
-  "eval_samples_per_second": 33.003,
-  "eval_steps_per_second": 33.003
-}
+  "eval_loss": 5.414119720458984,
+  "eval_runtime": 0.7383,
+  "eval_samples": 8,
+  "eval_samples_per_second": 10.836,
+  "eval_steps_per_second": 10.836
+}

metrics/train_metrics.json

@@ -1,9 +1,16 @@
 {
+  "ce_weight_end": 0.5,
+  "ce_weight_start": 0.5,
   "epoch": 1.0,
-  "total_flos": 39501942396.0,
-  "train_loss": 10.738998085260391,
-  "train_runtime": 49.2357,
-  "train_samples": 256,
-  "train_samples_per_second": 5.199,
-  "train_steps_per_second": 5.199
-}
+  "kd_weight_end": 0.5,
+  "kd_weight_start": 0.5,
+  "teacher_count": 1,
+  "temperature_end": 1.5,
+  "temperature_start": 2.0,
+  "total_flos": 3359425752.0,
+  "train_loss": 5.411877933301423,
+  "train_runtime": 7.7818,
+  "train_samples": 19,
+  "train_samples_per_second": 2.442,
+  "train_steps_per_second": 2.442
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8af636bf1ea0ba12e8e0ed9858fe7a4fb9ce267806245af516a5f024c1c370c1
+oid sha256:b75f6f94d4b470fd42d5cb3c7135ca7c1d669d0514eddce08abe5825cf9d5c48
 size 413296

run_summary.json

@@ -1,17 +1,15 @@
 {
-  "base_model": "sshleifer/tiny-gpt2",
-  "bf16": false,
-  "cuda_available": false,
+  "ce_weight_end": 0.5,
+  "ce_weight_start": 0.5,
   "dataset_id": "NorthernTribe-Research/UMSR-v1",
-  "device": "cpu",
-  "eval_rows": 64,
-  "finished_at": "2026-02-23T22:08:30.897247+00:00",
-  "fp16": false,
-  "mps_available": false,
-  "output_dir": "/app/runs/20260223_220733",
-  "target_repo_id": "NorthernTribe-Research/UMSR-Reasoner-7B",
-  "tie_word_embeddings": false,
-  "total_train_steps_estimate": 256,
-  "train_rows": 256,
-  "warmup_steps": 0
+  "eval_rows": 8,
+  "kd_weight_end": 0.5,
+  "kd_weight_start": 0.5,
+  "output_dir": "outputs/umsr_reasoner_7b_standalone",
+  "student_model": "NorthernTribe-Research/UMSR-Reasoner-7B",
+  "teacher_count": 1,
+  "teacher_model": "NorthernTribe-Research/UMSR-Reasoner-7B",
+  "temperature_end": 1.5,
+  "temperature_start": 2.0,
+  "train_rows": 19
 }

tokenizer.json

@@ -1,11 +1,6 @@
 {
   "version": "1.0",
-  "truncation": {
-    "direction": "Right",
-    "max_length": 512,
-    "strategy": "LongestFirst",
-    "stride": 0
-  },
+  "truncation": null,
   "padding": null,
   "added_tokens": [
     {

trainer_state.json

@@ -3,282 +3,192 @@
   "best_metric": null,
   "best_model_checkpoint": null,
   "epoch": 1.0,
-  "eval_steps": 25,
-  "global_step": 256,
+  "eval_steps": 4,
+  "global_step": 19,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.0390625,
-      "grad_norm": 0.6599220633506775,
-      "learning_rate": 9.6484375e-05,
-      "loss": 10.769454956054688,
-      "step": 10
-    },
-    {
-      "epoch": 0.078125,
-      "grad_norm": 0.7802038192749023,
-      "learning_rate": 9.257812500000001e-05,
-      "loss": 10.763140106201172,
-      "step": 20
-    },
-    {
-      "epoch": 0.09765625,
-      "eval_loss": 10.76131820678711,
-      "eval_runtime": 2.0029,
-      "eval_samples_per_second": 31.953,
-      "eval_steps_per_second": 31.953,
-      "step": 25
-    },
-    {
-      "epoch": 0.1171875,
-      "grad_norm": 0.17565655708312988,
-      "learning_rate": 8.8671875e-05,
-      "loss": 10.768231201171876,
-      "step": 30
+      "epoch": 0.05263157894736842,
+      "grad_norm": 0.374282568693161,
+      "learning_rate": 0.0001,
+      "loss": 5.411169052124023,
+      "step": 1
     },
     {
-      "epoch": 0.15625,
-      "grad_norm": 0.4683985710144043,
-      "learning_rate": 8.4765625e-05,
-      "loss": 10.772106170654297,
-      "step": 40
+      "epoch": 0.10526315789473684,
+      "grad_norm": 0.4834381341934204,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 5.413699626922607,
+      "step": 2
     },
     {
-      "epoch": 0.1953125,
-      "grad_norm": 0.3628169298171997,
-      "learning_rate": 8.0859375e-05,
-      "loss": 10.764241790771484,
-      "step": 50
+      "epoch": 0.15789473684210525,
+      "grad_norm": 1.9866410493850708,
+      "learning_rate": 9.729086208503174e-05,
+      "loss": 5.407270431518555,
+      "step": 3
     },
     {
-      "epoch": 0.1953125,
-      "eval_loss": 10.752297401428223,
-      "eval_runtime": 2.1266,
-      "eval_samples_per_second": 30.095,
-      "eval_steps_per_second": 30.095,
-      "step": 50
+      "epoch": 0.21052631578947367,
+      "grad_norm": 0.13184547424316406,
+      "learning_rate": 9.397368756032445e-05,
+      "loss": 5.411037921905518,
+      "step": 4
     },
     {
-      "epoch": 0.234375,
-      "grad_norm": 0.6150558590888977,
-      "learning_rate": 7.695312500000001e-05,
-      "loss": 10.760368347167969,
-      "step": 60
+      "epoch": 0.21052631578947367,
+      "eval_loss": 5.415262222290039,
+      "eval_runtime": 0.6782,
+      "eval_samples_per_second": 11.796,
+      "eval_steps_per_second": 11.796,
+      "step": 4
     },
     {
-      "epoch": 0.2734375,
-      "grad_norm": 0.605256974697113,
-      "learning_rate": 7.3046875e-05,
-      "loss": 10.758135986328124,
-      "step": 70
+      "epoch": 0.2631578947368421,
+      "grad_norm": 0.08807552605867386,
+      "learning_rate": 8.945702546981969e-05,
+      "loss": 5.413403034210205,
+      "step": 5
     },
     {
-      "epoch": 0.29296875,
-      "eval_loss": 10.743587493896484,
-      "eval_runtime": 2.0832,
-      "eval_samples_per_second": 30.721,
-      "eval_steps_per_second": 30.721,
-      "step": 75
+      "epoch": 0.3157894736842105,
+      "grad_norm": 0.3944004476070404,
+      "learning_rate": 8.386407858128706e-05,
+      "loss": 5.411047458648682,
+      "step": 6
     },
     {
-      "epoch": 0.3125,
-      "grad_norm": 0.3574382960796356,
-      "learning_rate": 6.9140625e-05,
-      "loss": 10.750237274169923,
-      "step": 80
+      "epoch": 0.3684210526315789,
+      "grad_norm": 0.19243471324443817,
+      "learning_rate": 7.734740790612136e-05,
+      "loss": 5.410379886627197,
+      "step": 7
     },
     {
-      "epoch": 0.3515625,
-      "grad_norm": 0.6408470869064331,
-      "learning_rate": 6.5234375e-05,
-      "loss": 10.751436614990235,
-      "step": 90
+      "epoch": 0.42105263157894735,
+      "grad_norm": 1.1386080980300903,
+      "learning_rate": 7.008477123264848e-05,
+      "loss": 5.4107584953308105,
+      "step": 8
     },
     {
-      "epoch": 0.390625,
-      "grad_norm": 0.2543087303638458,
-      "learning_rate": 6.132812500000001e-05,
-      "loss": 10.753249359130859,
-      "step": 100
+      "epoch": 0.42105263157894735,
+      "eval_loss": 5.414775848388672,
+      "eval_runtime": 0.6096,
+      "eval_samples_per_second": 13.123,
+      "eval_steps_per_second": 13.123,
+      "step": 8
     },
     {
-      "epoch": 0.390625,
-      "eval_loss": 10.736135482788086,
-      "eval_runtime": 1.9456,
-      "eval_samples_per_second": 32.894,
-      "eval_steps_per_second": 32.894,
-      "step": 100
+      "epoch": 0.47368421052631576,
+      "grad_norm": 1.593002200126648,
+      "learning_rate": 6.227427435703997e-05,
+      "loss": 5.406350135803223,
+      "step": 9
     },
     {
-      "epoch": 0.4296875,
-      "grad_norm": 0.29619327187538147,
-      "learning_rate": 5.7421875000000005e-05,
-      "loss": 10.720957946777343,
-      "step": 110
-    },
-    {
-      "epoch": 0.46875,
-      "grad_norm": 0.19426824152469635,
-      "learning_rate": 5.3515625e-05,
-      "loss": 10.75610580444336,
-      "step": 120
-    },
-    {
-      "epoch": 0.48828125,
-      "eval_loss": 10.730175018310547,
-      "eval_runtime": 2.0492,
-      "eval_samples_per_second": 31.232,
-      "eval_steps_per_second": 31.232,
-      "step": 125
-    },
-    {
-      "epoch": 0.5078125,
-      "grad_norm": 0.400691419839859,
-      "learning_rate": 4.9609375000000005e-05,
-      "loss": 10.729013061523437,
-      "step": 130
-    },
-    {
-      "epoch": 0.546875,
-      "grad_norm": 0.2926430106163025,
-      "learning_rate": 4.5703125e-05,
-      "loss": 10.718487548828126,
-      "step": 140
-    },
-    {
-      "epoch": 0.5859375,
-      "grad_norm": 0.24082393944263458,
-      "learning_rate": 4.1796875000000005e-05,
-      "loss": 10.736891174316407,
-      "step": 150
-    },
-    {
-      "epoch": 0.5859375,
-      "eval_loss": 10.725324630737305,
-      "eval_runtime": 1.9834,
-      "eval_samples_per_second": 32.268,
-      "eval_steps_per_second": 32.268,
-      "step": 150
-    },
-    {
-      "epoch": 0.625,
-      "grad_norm": 0.371867835521698,
-      "learning_rate": 3.7890625e-05,
-      "loss": 10.728065490722656,
-      "step": 160
-    },
-    {
-      "epoch": 0.6640625,
-      "grad_norm": 0.3023074269294739,
-      "learning_rate": 3.3984375000000004e-05,
-      "loss": 10.73614501953125,
-      "step": 170
-    },
-    {
-      "epoch": 0.68359375,
-      "eval_loss": 10.721412658691406,
-      "eval_runtime": 2.1993,
-      "eval_samples_per_second": 29.1,
-      "eval_steps_per_second": 29.1,
-      "step": 175
+      "epoch": 0.5263157894736842,
+      "grad_norm": 0.2701893746852875,
+      "learning_rate": 5.4128967273616625e-05,
+      "loss": 5.415624141693115,
+      "step": 10
     },
     {
-      "epoch": 0.703125,
-      "grad_norm": 0.38127991557121277,
-      "learning_rate": 3.0078125e-05,
-      "loss": 10.722113037109375,
-      "step": 180
+      "epoch": 0.5789473684210527,
+      "grad_norm": 2.5317883491516113,
+      "learning_rate": 4.5871032726383386e-05,
+      "loss": 5.4125213623046875,
+      "step": 11
     },
     {
-      "epoch": 0.7421875,
-      "grad_norm": 0.3930608928203583,
-      "learning_rate": 2.6171875e-05,
-      "loss": 10.726372528076173,
-      "step": 190
+      "epoch": 0.631578947368421,
+      "grad_norm": 0.18132847547531128,
+      "learning_rate": 3.772572564296005e-05,
+      "loss": 5.411061763763428,
+      "step": 12
     },
     {
-      "epoch": 0.78125,
-      "grad_norm": 0.3989920914173126,
-      "learning_rate": 2.2265625e-05,
-      "loss": 10.720677185058594,
-      "step": 200
+      "epoch": 0.631578947368421,
+      "eval_loss": 5.414316177368164,
+      "eval_runtime": 0.6242,
+      "eval_samples_per_second": 12.817,
+      "eval_steps_per_second": 12.817,
+      "step": 12
     },
     {
-      "epoch": 0.78125,
-      "eval_loss": 10.718664169311523,
-      "eval_runtime": 2.0837,
-      "eval_samples_per_second": 30.714,
-      "eval_steps_per_second": 30.714,
-      "step": 200
+      "epoch": 0.6842105263157895,
+      "grad_norm": 0.25826773047447205,
+      "learning_rate": 2.991522876735154e-05,
+      "loss": 5.412132740020752,
+      "step": 13
     },
     {
-      "epoch": 0.8203125,
-      "grad_norm": 0.20773838460445404,
-      "learning_rate": 1.8359375e-05,
-      "loss": 10.722401428222657,
-      "step": 210
+      "epoch": 0.7368421052631579,
+      "grad_norm": 0.32883742451667786,
+      "learning_rate": 2.2652592093878666e-05,
+      "loss": 5.409476280212402,
+      "step": 14
     },
     {
-      "epoch": 0.859375,
-      "grad_norm": 0.31527552008628845,
-      "learning_rate": 1.4453125e-05,
-      "loss": 10.71902084350586,
-      "step": 220
+      "epoch": 0.7894736842105263,
+      "grad_norm": 0.15471753478050232,
+      "learning_rate": 1.6135921418712956e-05,
+      "loss": 5.413200855255127,
+      "step": 15
     },
     {
-      "epoch": 0.87890625,
-      "eval_loss": 10.71699047088623,
-      "eval_runtime": 2.0085,
-      "eval_samples_per_second": 31.864,
-      "eval_steps_per_second": 31.864,
-      "step": 225
+      "epoch": 0.8421052631578947,
+      "grad_norm": 0.2990401089191437,
+      "learning_rate": 1.0542974530180327e-05,
+      "loss": 5.4155683517456055,
+      "step": 16
     },
     {
-      "epoch": 0.8984375,
-      "grad_norm": 0.2394668161869049,
-      "learning_rate": 1.0546875e-05,
-      "loss": 10.706802368164062,
-      "step": 230
+      "epoch": 0.8421052631578947,
+      "eval_loss": 5.414139747619629,
+      "eval_runtime": 0.7326,
+      "eval_samples_per_second": 10.92,
+      "eval_steps_per_second": 10.92,
+      "step": 16
     },
     {
-      "epoch": 0.9375,
-      "grad_norm": 0.23905383050441742,
-      "learning_rate": 6.6406250000000005e-06,
-      "loss": 10.697933959960938,
-      "step": 240
+      "epoch": 0.8947368421052632,
+      "grad_norm": 2.2788937091827393,
+      "learning_rate": 6.026312439675552e-06,
+      "loss": 5.411947727203369,
+      "step": 17
     },
     {
-      "epoch": 0.9765625,
-      "grad_norm": 0.3481399416923523,
-      "learning_rate": 2.734375e-06,
-      "loss": 10.72392578125,
-      "step": 250
+      "epoch": 0.9473684210526315,
+      "grad_norm": 0.20979730784893036,
+      "learning_rate": 2.7091379149682685e-06,
+      "loss": 5.413649082183838,
+      "step": 18
     },
     {
-      "epoch": 0.9765625,
-      "eval_loss": 10.716254234313965,
-      "eval_runtime": 2.0417,
-      "eval_samples_per_second": 31.347,
-      "eval_steps_per_second": 31.347,
-      "step": 250
+      "epoch": 1.0,
+      "grad_norm": 0.20296302437782288,
+      "learning_rate": 6.819348298638839e-07,
+      "loss": 5.415382385253906,
+      "step": 19
     },
     {
       "epoch": 1.0,
-      "step": 256,
-      "total_flos": 39501942396.0,
-      "train_loss": 10.738998085260391,
-      "train_runtime": 49.2357,
-      "train_samples_per_second": 5.199,
-      "train_steps_per_second": 5.199
+      "step": 19,
+      "total_flos": 3359425752.0,
+      "train_loss": 5.411877933301423,
+      "train_runtime": 7.7818,
+      "train_samples_per_second": 2.442,
+      "train_steps_per_second": 2.442
     }
   ],
-  "logging_steps": 10,
-  "max_steps": 256,
+  "logging_steps": 1,
+  "max_steps": 19,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 1,
-  "save_steps": 25,
+  "save_steps": 8,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
@@ -291,7 +201,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 39501942396.0,
+  "total_flos": 3359425752.0,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0b40f89f203aa6181d2ecfcc97532f60812689752da3bd88fca13bc6a344c8f6
-size 5201
+oid sha256:a142cd858828160b38820775c31c8d19bc13769b29cdfea16c1835758c53a125
+size 5265