agent-1.2e / README.md
alextripplet's picture
Upload folder using huggingface_hub
784c58a verified
|
Raw
History Blame Contribute Delete
2.52 kB
# Agent 1.2e
A linear weight merge, built 2026-07-08. The usable model lives in `model/` (Hugging Face format, bfloat16, ~988 MB).
## What was actually merged
Requested: 33/33/33 of Qwen2.5-0.5B-Instruct, SmolLM2-360M-Instruct, and Qwen2.5-0.5B (base).
**SmolLM2-360M-Instruct could not be included in the weight average.** Weight merging requires identical architectures, and SmolLM2 differs from Qwen2.5-0.5B in every dimension that matters:
| | Qwen2.5-0.5B | SmolLM2-360M |
|---|---|---|
| Hidden size | 896 | 960 |
| Layers | 24 | 32 |
| Vocab / tokenizer | 151,936 (Qwen BPE) | 49,152 (GPT-2 style) |
The tensors have different shapes, so averaging them is mathematically undefined β€” no merge tool (mergekit included) can do it.
So `model/` is the two compatible models merged at equal weight (33/33 renormalized to 50/50):
- 50% **Qwen/Qwen2.5-0.5B-Instruct**
- 50% **Qwen/Qwen2.5-0.5B** (base)
Tokenizer, chat template, and generation config come from the Instruct model. Verified: loads with `transformers` and follows chat-formatted instructions coherently.
## If you still want SmolLM2's conversational flavor
Options that actually work:
1. **Distillation** β€” fine-tune this merge on SmolLM2-generated conversations (LoRA on a few thousand samples is enough at this scale).
2. **Routing** β€” keep SmolLM2 (in `sources/`) as a separate model and route conversational traffic to it.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("model")
model = AutoModelForCausalLM.from_pretrained("model")
```
## Ollama
The model is registered in Ollama as `agent-1.2e` (q8_0 GGUF, hardcoded system
prompt with the Agent 1.2e identity and internal codename `agent1-iteration2-eco`):
```bash
ollama run agent-1.2e
```
To rebuild after editing `Modelfile` (system prompt, params):
`ollama create agent-1.2e -f Modelfile`
## MLX / LM Studio
`model-mlx/` is the MLX (bf16) conversion, installed in LM Studio at
`~/.lmstudio/models/local/Agent-1.2e-MLX` β€” it appears as **agent-1.2e-mlx**.
The Agent 1.2e identity + codename are baked into the chat template's default
system prompt (applies whenever no explicit system prompt is set; a user-set
system prompt overrides it).
## Folder contents
- `model/` β€” the merged Agent 1.2e model
- `sources/` β€” the three downloaded source models (~2.6 GB; safe to delete once you're happy with the merge)
- `merge.py` β€” the merge script (re-runnable; edit `SOURCES` weights to re-blend)