Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,164 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model:
+  - LLM-OS-Models/Fabliq-8B-Agent
+  - LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch
+  - LiquidAI/LFM2.5-8B-A1B
+base_model_relation: finetune
+tags:
+  - lfm
+  - liquid-ai
+  - moe
+  - agentic
+  - tool-use
+  - terminal
+  - reasoning
+  - fable-5
+  - withinus
+  - helio
+  - distillation
+  - sft
+language:
+  - en
+pipeline_tag: text-generation
+---
+
+# Fabliq-8B-Agent-Reasoning 🌊🧠
+
+> The **reasoning-expanded sibling** of [Fabliq-8B-Agent](https://huggingface.co/LLM-OS-Models/Fabliq-8B-Agent). Adds general + deep reasoning on top of the agentic foundation — broadens the model beyond pure terminal tool-use into multi-domain expert Q&A, mathematical reasoning, scientific analysis, and cybersecurity. Two-phase curriculum inspired by [Qwythos-9B](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M).
+
+## ✨ Why Fabliq-Reasoning?
+
+- **🐠 Same tiny footprint, broader reach.** Inherits LFM2.5-8B-A1B's MoE efficiency (~1B active params). Now also handles expert Q&A, math, science — not just terminal work.
+- **🛠 Still agentic.** Phase-1 tool-use capability is preserved — the model still reasons in `<think>` and emits native LFM tool calls when needed.
+- **🧠 Multi-domain reasoning.** Trained on WithinUs (6 categories: advanced coding, agentic planning, general QA, math reasoning, scientific analysis, cybersecurity) + Helio (Opus 4.8 deep-reasoning distillation).
+- **🎯 2-phase curriculum.** Phase-1 broad agentic distillation → Phase-2 focused reasoning expansion (Qwythos pattern).
+
+## 🧪 Model details
+
+| | |
+| --- | --- |
+| **Architecture** | Lfm2MoeForCausalLM (24 layers, 32 experts, 4 experts/token) |
+| **Parameters** | ~8B total / ~1B active (MoE) |
+| **Context** | 8,192 trained · 128K native (`rope_theta=5e6`) |
+| **Precision** | bfloat16 |
+| **Fine-tune type** | Full-parameter SFT, continuation from Fabliq-8B-Agent |
+| **License** | Apache 2.0 |
+
+## 📚 Training data (Phase-2 only)
+
+| Source | Rows | Description |
+| --- | --- | --- |
+| WithinUs (from `claude_mythos_distilled_25k`) | 135 | 6-category expert Q&A — coding, planning, math, science, cybersecurity. SHA-256 dedup (25k → 135 unique). |
+| Helio (`Fable-5-Distill-Reasoning-462x`) | 146 | Opus 4.8 deep-reasoning traces. Russian-language filter (Cyrillic <30%). |
+| **Total Phase-2** | **281** | |
+
+Preprocessing:
+- **WithinUs:** Category-balanced (max 350/cat), SHA-256 dedup, "Drawing from the autonomous..." template first-sentence removal → [`build_withinus_lfm_sft.py`](https://github.com/LLM-OS-Models/Terminal/blob/main/fable_distillation/build_withinus_lfm_sft.py)
+- **Helio:** Cyrillic ratio filter (<30%), `<think>` wrapping for reasoning, line 192 corruption skip → [`build_helio_lfm_sft.py`](https://github.com/LLM-OS-Models/Terminal/blob/main/fable_distillation/build_helio_lfm_sft.py)
+- Combined: [`build_phase2_reasoning`](https://github.com/LLM-OS-Models/Terminal/blob/main/fable_distillation/datasets/) (concat)
+
+## 🔧 Training procedure (Phase-2)
+
+| Hyperparameter | Value |
+| --- | --- |
+| Base | `LLM-OS-Models/Fabliq-8B-Agent` (Phase-1 final) |
+| Schedule | 4 epochs, constant LR |
+| Max sequence length | 8,192 |
+| Per-device batch size | 2 |
+| Gradient accumulation | 4 |
+| GPUs | 8× H200 (effective batch 64) |
+| Learning rate | 3e-7 (lower than Phase-1 — model already agentic-tuned, avoid forgetting) |
+| Precision | bf16 |
+| FSDP | `full_shard`, activation checkpointing, `Lfm2MoeDecoderLayer` auto-wrap |
+| Final train_loss | ~1.6 |
+| Train runtime | ~6 minutes (281 rows × 4 epochs) |
+| Global steps | 20 |
+
+## 💬 System prompts (per data source)
+
+**WithinUs (broad reasoning):**
+```
+You are a knowledgeable assistant. Provide rigorous, well-structured answers
+across coding, cybersecurity, mathematics, scientific analysis, agentic planning,
+and general expert topics. Be precise and thorough.
+```
+
+**Helio (deep reasoning):**
+```
+You are a deep-reasoning assistant. Think step by step inside <think>...</think>,
+then provide a clear, structured answer.
+```
+
+## 🚀 How to use
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "LLM-OS-Models/Fabliq-8B-Agent-Reasoning"
+tok = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id, dtype=torch.bfloat16, device_map="auto"
+)
+
+SYSTEM = (
+    "You are a deep-reasoning assistant. Think step by step inside <think>...</think>, "
+    "then provide a clear, structured answer."
+)
+
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": "Derive the time complexity of merge sort and explain when it beats quicksort."},
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+
+out = model.generate(
+    **inputs,
+    max_new_tokens=2048,
+    do_sample=False,
+    repetition_penalty=1.05,
+)
+print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
+```
+
+## 🎯 When to use which Fabliq?
+
+| Use case | Model |
+| --- | --- |
+| Pure terminal / coding agent (read, edit, run, verify) | [Fabliq-8B-Agent](https://huggingface.co/LLM-OS-Models/Fabliq-8B-Agent) |
+| Multi-domain expert Q&A + reasoning + still agentic | **Fabliq-8B-Agent-Reasoning** (this model) |
+| Local 16GB VRAM deployment with tool-use | Either — both fit comfortably |
+
+## ⚠️ Limitations
+
+- **Phase-2 dataset is small (281 rows).** Reasoning expansion is real but bounded — this is a delta on top of Phase-1, not a from-scratch reasoning model.
+- **WithinUs dedup surprise.** Source dataset claims 25k rows but after SHA-256 dedup of templated prompts, only 135 unique rows remain. Template overfitting in the source data was severe.
+- **Helio Russian filter.** Original 462 rows filtered to 146 rows after removing Cyrillic-dominant (Russian) traces. Non-English coverage is limited.
+- **No safety alignment.** Trained on raw reasoning traces; add your own guardrails for production.
+- **Max seq 8,192 at training.** Behavior beyond 8K context is unverified.
+- **English-centric.**
+
+## 📜 License
+
+Apache 2.0, inherited from the [LiquidAI LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B) base.
+
+## 🌳 Model tree
+
+This is a **fine-tune** (continuation SFT). Direct parent: [`LLM-OS-Models/Fabliq-8B-Agent`](https://huggingface.co/LLM-OS-Models/Fabliq-8B-Agent).
+
+```
+LiquidAI/LFM2.5-8B-A1B                                          (LiquidAI base)
+  └─ LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch  (ToolBench foundation)
+      └─ LLM-OS-Models/Fabliq-8B-Agent                           (Phase-1: Fable-5 agentic SFT)
+          └─ LLM-OS-Models/Fabliq-8B-Agent-Reasoning             ← this model (Phase-2: + WithinUs + Helio)
+```
+
+## 🙏 Acknowledgements
+
+- **Base:** [LiquidAI/LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B)
+- **Phase-1 parent:** [Fabliq-8B-Agent](https://huggingface.co/LLM-OS-Models/Fabliq-8B-Agent)
+- **Phase-2 data:** WithinUs (claude_mythos_distilled_25k), Helio (Fable-5-Distill-Reasoning-462x)
+- **Reference:** [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) — 2-phase curriculum pattern