Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,158 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: LiquidAI/LFM2.5-8B-A1B
+base_model_relation: finetune
+tags:
+  - lfm
+  - liquid-ai
+  - moe
+  - agentic
+  - tool-use
+  - terminal
+  - coding-agent
+  - fable-5
+  - distillation
+  - sft
+language:
+  - en
+pipeline_tag: text-generation
+---
+
+# LFM2.5-8B-A1B-Terminal-Fable5-Agentic-SFT-3Epoch
+
+An agentic coding fine-tune of **LiquidAI/LFM2.5-8B-A1B** (8B total / ~1B active MoE) trained on real Claude Code terminal traces distilled from the Fable-5 dataset. This model continues from `LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch` and adds 3 epochs of full-parameter SFT on 4,047 multi-turn agentic trajectories (read → reason → call tool → observe → continue).
+
+## Model details
+
+- **Base model:** [`LiquidAI/LFM2.5-8B-A1B`](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B) via [`LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch`](https://huggingface.co/LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch)
+- **Architecture:** Lfm2MoeForCausalLM (24 layers, 32 experts, 4 experts/token, hybrid conv+full-attention)
+- **Parameters:** ~8B total / ~1B active
+- **Fine-tune type:** Full-parameter SFT (bf16, FSDP full_shard)
+- **Context:** 8,192 tokens trained, 128K native via rope_theta=5e6
+
+## Training data
+
+| Source | Rows | Description |
+| --- | --- | --- |
+| [Glint-Research/Fable-5-traces](https://huggingface.co/datasets/Glint-Research/Fable-5-traces) | 4,047 | Real Claude Code terminal sessions: Bash, Edit, Read, Write, Glob, Grep, WebSearch tool-use trajectories |
+
+Preprocessing:
+- Parsed `context` → multi-turn `USER`/`ASSISTANT (message)` messages
+- Removed `<local-command-caveat>` / `<command-*>` slash-command metadata blocks
+- Converted `{tool, input}` structured output → LFM native tool-call format `<|tool_call_start|>[ToolName(arg='value')]<|tool_call_end|>`
+- Wrapped chain-of-thought in `<think>...</think>` tags for assistant reasoning training
+- Dropped 618 rows with insufficient context (<3 messages after parsing)
+- Max sequence length: 8,192 tokens (covers 98.6% of traces without truncation)
+
+## Training procedure
+
+| Hyperparameter | Value |
+| --- | --- |
+| Schedule | 3 epochs, constant LR (continuation SFT) |
+| Max sequence length | 8,192 |
+| Per-device batch size | 2 |
+| Gradient accumulation | 4 |
+| GPUs | 8× H200 (effective batch 64) |
+| Learning rate | 5e-7 (adamw_torch) |
+| Precision | bf16 |
+| FSDP | full_shard, activation_checkpointing, Lfm2MoeDecoderLayer auto_wrap |
+| Loss | NLL, assistant-only (chat-template masked) |
+| Final train_loss | 1.277 |
+| Train runtime | 831 seconds (~14 min) |
+| Global steps | 192 |
+
+## System prompt
+
+The model was trained with this system prompt:
+
+```
+You are an agentic coding assistant. Read the conversation history and tool results,
+think step by step inside <think>...</think>, then either call a tool using
+<|tool_call_start|>[ToolName(arg=value)]<|tool_call_end|> or respond with text.
+Use available tools (Bash, Edit, Read, Write, Glob, Grep, WebSearch, WebFetch, etc.)
+to accomplish the user's task. Be concise but thorough.
+```
+
+## How to use
+
+### transformers
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "LLM-OS-Models/LFM2.5-8B-A1B-Terminal-Fable5-Agentic-SFT-3Epoch"
+tok = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id, dtype=torch.bfloat16, device_map="auto"
+)
+
+SYSTEM = (
+    "You are an agentic coding assistant. Read the conversation history and tool results, "
+    "think step by step inside <think>...</think>, then either call a tool using "
+    "<|tool_call_start|>[ToolName(arg=value)]<|tool_call_end|> or respond with text. "
+    "Use available tools (Bash, Edit, Read, Write, Glob, Grep, WebSearch, WebFetch, etc.) "
+    "to accomplish the user's task. Be concise but thorough."
+)
+
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": "List the Python files in /tmp and report the line count of the largest one."},
+]
+text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(text, return_tensors="pt").to(model.device)
+
+out = model.generate(
+    **inputs,
+    max_new_tokens=1024,
+    do_sample=False,
+    repetition_penalty=1.05,
+)
+print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
+```
+
+Expected output format — the model reasons in `<think>...</think>`, then emits a tool call:
+
+```
+<think>
+I need to find Python files in /tmp. I'll use Bash with ls and wc -l.
+</think>
+
+<|tool_call_start|>[Bash(command='ls -1 /tmp/*.py 2>/dev/null | xargs wc -l 2>/dev/null | sort -n | tail -1')]<|tool_call_end|>
+```
+
+### vLLM
+
+```bash
+vllm serve LLM-OS-Models/LFM2.5-8B-A1B-Terminal-Fable5-Agentic-SFT-3Epoch \
+  --max-model-len 8192 --dtype bfloat16 --gpu-memory-utilization 0.9
+```
+
+## Intended use
+
+- Local coding agent on top of an MoE-efficient backbone (1B active params)
+- Terminal / file-system agentic loops (read, edit, run, verify)
+- Research on distillation from frontier closed models into open-weight MoE models
+
+## Limitations
+
+- **Agentic specialization.** This is a focused fine-tune for terminal/coding agentic work. General-knowledge breadth is inherited from the base and may be slightly below `LFM2.5-8B-A1B` on benchmarks outside the agentic domain.
+- **No safety alignment.** Trained on raw tool-use traces; add your own guardrails for production deployments.
+- **Tool-format lock-in.** The model emits LFM-native tool-call syntax. A harness that parses `<|tool_call_start|>...<|tool_call_end|>` and executes the call is required for the agentic loop to actually work.
+- **Max seq 8,192 at training.** Beyond 8K context, behavior is unverified for this checkpoint.
+- **English-centric.**
+
+## License
+
+Apache 2.0, inherited from the LiquidAI LFM2.5-8B-A1B base.
+
+## Citation / lineage
+
+```
+Base: LiquidAI/LFM2.5-8B-A1B
+  → LLM-OS-Models/LFM2.5-8B-A1B-Terminal-ToolBench-Full-SFT-1Epoch (ToolBench SFT)
+    → LLM-OS-Models/LFM2.5-8B-A1B-Terminal-Fable5-Agentic-SFT-3Epoch (this model, Fable-5 agentic SFT)
+```
+
+Training data: [Glint-Research/Fable-5-traces](https://huggingface.co/datasets/Glint-Research/Fable-5-traces)

chat_template.jinja

@@ -0,0 +1,115 @@
+{{- bos_token -}}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+
+{%- macro format_arg_value(arg_value) -%}
+    {%- if arg_value is string -%}
+        {{- "'" + arg_value + "'" -}}
+    {%- elif arg_value is mapping -%}
+        {{- arg_value | tojson -}}
+    {%- else -%}
+        {{- arg_value | string -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- macro parse_content(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- else -%}
+        {%- set _ns = namespace(result="") -%}
+        {%- for item in content -%}
+            {%- if item["type"] == "image" -%}
+                {%- set _ns.result = _ns.result + "<image>" -%}
+            {%- elif item["type"] == "text" -%}
+                {%- set _ns.result = _ns.result + item["text"] -%}
+            {%- else -%}
+                {%- set _ns.result = _ns.result + item | tojson -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {{- _ns.result -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_tool_calls(tool_calls) -%}
+    {%- set tool_calls_ns = namespace(tool_calls=[]) -%}
+    {%- for tool_call in tool_calls -%}
+        {%- set func_name = tool_call["function"]["name"] -%}
+        {%- set func_args = tool_call["function"]["arguments"] -%}
+        {%- set args_ns = namespace(arg_strings=[]) -%}
+        {%- for arg_name, arg_value in func_args.items() -%}
+            {%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
+        {%- endfor -%}
+        {%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
+    {%- endfor -%}
+    {{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
+{%- endmacro -%}
+
+{%- set ns = namespace(system_prompt="", last_user_index=-1) -%}
+{%- if messages[0]["role"] == "system" -%}
+    {%- if messages[0].get("content") -%}
+        {%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
+    {%- endif -%}
+    {%- set messages = messages[1:] -%}
+{%- endif -%}
+{%- if tools -%}
+    {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
+    {%- for tool in tools -%}
+        {%- if tool is not string -%}
+            {%- set tool = tool | tojson -%}
+        {%- endif -%}
+        {%- set ns.system_prompt = ns.system_prompt + tool -%}
+        {%- if not loop.last -%}
+            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns.system_prompt = ns.system_prompt + "]" -%}
+{%- endif -%}
+{%- if ns.system_prompt -%}
+    {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
+{%- endif -%}
+{%- for message in messages -%}
+    {%- if message["role"] == "user" -%}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- for message in messages -%}
+    {{- "<|im_start|>" + message.role + "\n" -}}
+    {%- if message.role == "assistant" -%}
+        {%- generation -%}
+        {%- if message.thinking is defined and (preserve_thinking or loop.index0 > ns.last_user_index) -%}
+            {{- "<think>" + message.thinking + "</think>" -}}
+        {%- endif -%}
+        {%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
+        {%- set _has_cfm = false -%}
+        {%- if message.content is defined -%}
+            {%- set content = parse_content(message.content) -%}
+            {%- if not (preserve_thinking or loop.index0 > ns.last_user_index) -%}
+                {%- if "</think>" in content -%}
+                    {%- set content = content.split("</think>")[-1] | trim -%}
+                {%- endif -%}
+            {%- endif -%}
+            {%- if message.tool_calls is defined and content.endswith(_cfm_tag) -%}
+                {%- set _has_cfm = true -%}
+                {%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
+                {{- content[:_trunc_len] -}}
+            {%- else -%}
+                {{- content -}}
+            {%- endif -%}
+        {%- endif -%}
+        {%- if message.tool_calls is defined -%}
+            {{- render_tool_calls(message.tool_calls) -}}
+        {%- endif -%}
+        {%- if _has_cfm -%}
+            {{- _cfm_tag -}}
+        {%- endif -%}
+        {{- "<|im_end|>\n" -}}
+        {%- endgeneration -%}
+    {%- else %}
+        {%- if message.get("content") -%}
+            {{- parse_content(message["content"]) -}}
+        {%- endif -%}
+        {{- "<|im_end|>\n" -}}
+    {%- endif %}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{- "<|im_start|>assistant\n" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,63 @@
+{
+  "architectures": [
+    "Lfm2MoeForCausalLM"
+  ],
+  "bos_token_id": 124894,
+  "conv_L_cache": 3,
+  "conv_bias": false,
+  "dtype": "bfloat16",
+  "eos_token_id": 124900,
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 7168,
+  "layer_types": [
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv"
+  ],
+  "max_position_embeddings": 128000,
+  "model_name": "LiquidAI/LFM2.5-8B-A1B",
+  "model_type": "lfm2_moe",
+  "moe_intermediate_size": 1792,
+  "norm_eps": 1e-05,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_dense_layers": 2,
+  "num_experts": 32,
+  "num_experts_per_tok": 4,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 8,
+  "pad_token_id": 124893,
+  "rope_parameters": {
+    "rope_theta": 5000000,
+    "rope_type": "default"
+  },
+  "routed_scaling_factor": 1.0,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.0",
+  "unsloth_version": "2026.4.8",
+  "use_cache": false,
+  "use_expert_bias": true,
+  "vocab_size": 128000
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 124894,
+  "eos_token_id": [
+    124900
+  ],
+  "max_length": 128000,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 124893,
+  "transformers_version": "5.5.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c33c49ed32b2b09bac161ef8e3fada604b46b80dacdb7f2e0c7a4b62ed1c3db
+size 16936006912

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
+size 17905598

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "is_local": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|pad|>",
+  "padding_side": "right",
+  "tokenizer_class": "TokenizersBackend",
+  "use_default_system_prompt": false
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a4f5ba72caf07528d9c009943e7645de49a73764877078b494afda8818ab561
+size 5265