File size: 3,228 Bytes
ca29f40 61076c6 ca29f40 61076c6 ca29f40 45054fb 61076c6 45054fb ca29f40 45054fb ca29f40 45054fb ca29f40 61076c6 ca29f40 61076c6 45054fb 61076c6 45054fb 61076c6 ca29f40 61076c6 45054fb 61076c6 ca29f40 45054fb 61076c6 45054fb ca29f40 45054fb ca29f40 45054fb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
license: apache-2.0
pipeline_tag: text-generation
language: en
tags:
- lfm2
- agent
- fine-tune
- tool-use
- liquid-ai
base_model: LiquidAI/LFM2.5-1.2B-Thinking
datasets:
- TeichAI/Ox-Alpha-Pi-Traces
metrics:
- loss
---
# LiquidAgent-1.2B
A supervised fine-tune of [LiquidAI/LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) on agent traces from [TeichAI/Ox-Alpha-Pi-Traces](https://huggingface.co/datasets/TeichAI/Ox-Alpha-Pi-Traces), with tool results properly included in the training data.
## What this is
The base LFM2.5-1.2B-Thinking model is a 1.17B-param hybrid architecture (10 LIV-conv + 6 GQA blocks, 32K context, 65536 vocab) with built-in thinking. This fine-tune adapts it for agentic tool-use: the model learns to emit tool calls, see the tool results, and reason about them in subsequent turns.
**Key fix over prior attempts:** The original parser for the TeichAI data silently dropped all `toolResult` messages (the `type` field is always `"message"` in the raw data, and tool results arrive as `message.role == "toolResult"` - a branch that did not exist). This fine-tune uses a fixed parser that recovers 100% of tool turns (verified: 562/562 on a 50-file sample).
## Training details
- **Base:** LiquidAI/LFM2.5-1.2B-Thinking (bfloat16)
- **Data:** 379 examples, ~1.34M assistant tokens (from Ox-Alpha-Pi-Traces, 400 session files, 397 usable)
- **Optimizer:** AdamW 8-bit, lr 2e-6 to 8e-6 (cosine warmup over 47 steps)
- **Hardware:** NVIDIA RTX 5090 (32 GB), gradient checkpointing, batch size 1
- **Duration:** 81 seconds (47 steps)
- **Final loss:** 0.5965 (step 40) - training completed at step 47
## Architecture
| Parameter | Value |
|-----------|-------|
| Params | 1,170,340,608 (tied embedding) |
| Layers | 16 (10 conv + 6 full attention) |
| Hidden size | 2048 |
| FF dim | 12288 |
| Attention heads | 32 Q / 8 KV |
| Vocab | 65,536 |
| Context | 32,768 (max_position_embeddings: 128,000) |
| Dtype | bfloat16 |
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Compactbot/liquidagent-1.2b",
torch_dtype="bfloat16",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Compactbot/liquidagent-1.2b")
messages = [{"role": "user", "content": "Write a Python function to parse a JSON config file."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
## Honest limitations
- Trained on only 379 examples (~1.3M tokens) - this is a very light adaptation, not a full agent training run.
- The model retains the base's thinking pattern (emits thinking blocks before responding).
- No independent benchmark evaluation was run; quality was verified by generation samples (coherent, on-topic, no degenerate repetition).
- The 32k sequence length requested by the requester was not used in this run (data was packed at shorter lengths to fit the 32GB GPU with 8-bit AdamW). |