File size: 4,340 Bytes
00bb63e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | ---
base_model: Lazarus-Ai/ReAligned-Qwen3.5-4B
library_name: peft
license: apache-2.0
tags:
- lora
- sft
- summarization
- context-compaction
- agents
- egirl
---
# Compactor-Qwen3.5-4B
A context-compaction summariser for [egirl](https://github.com/Schneewolf-Labs/egirl): given a run
of agentic conversation about to be dropped, write the summary that lets the agent continue.
Meant to run as the **auxiliary model** β the operator keeps its slot and its context, and the
summary is produced by something small, deterministic, and trained for it.
## The failure it fixes
egirl compacts by dropping middle messages and replacing them with a summary. When that summary
loses the task, the agent does not get confused β **it invents a new task**:
```
Interior compaction: dropped 19 middle messages, kept head + 0 tail groups
Generated summary (396 chars) from 19 messages
```
Sixteen web searches of real research compressed to 396 characters, after which the agent answered
a question about agent-harness architecture with *"Fresh project scaffolded at
~/projects/fresh_project."* The request was gone from its context.
So the objective is not fluency. It is **preservation**: the task, the findings, the state.
## Measured
45 held-out compaction windows, from transcripts that contributed no training data. Deterministic
checks, no judge:
| | base | **+ Compactor** |
|---|---|---|
| has a `Task:` line | **0/45 (0%)** | **45/45 (100%)** |
| task overlap (word F1 vs reference) | 0.000 | **0.509** |
| **fact recall** (paths, identifiers, numbers, errors) | **0.157** | **0.661** |
| compression | 0.043 | **0.129** |
| mean output | 884 chars | 2,689 chars |
Reference summaries (GPT-5) compress to ~0.12Γ. The base compresses to 0.043Γ β it is not
summarising, it is discarding: **84% of the distinctive facts are gone**, and it never states the
task at all.
**Fact recall is the number to read.** A summary can be fluent, plausible, correctly shaped, and
still have the specifics sanded off β which is what a small model does by default, and what makes
the agent lose the thread three turns later.
## Use
```python
model = AutoModelForImageTextToText.from_pretrained(
"Lazarus-Ai/ReAligned-Qwen3.5-4B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "schneewolflabs/Compactor-Qwen3.5-4B")
```
Render prompts with `enable_thinking=False` β training used the pre-closed think block, and the
model answers directly rather than deliberating. Serving with thinking open puts it in a state it
never saw.
In egirl, point `[local.auxiliary]` at a server running this and compaction routes to it
automatically.
## Training
| | |
|---|---|
| base | `Lazarus-Ai/ReAligned-Qwen3.5-4B` |
| method | SFT, LoRA **r=64 Ξ±=128** dropout 0.05 |
| data | [egirl-compaction-SFT](https://huggingface.co/datasets/schneewolflabs/egirl-compaction-SFT) β 1,059 train / 45 val |
| tokens | 9.34M |
| schedule | 1 epoch, 133 steps, lr 1e-4 cosine |
| sequence | 16,384 β an 8192 cap would have dropped 55% of rows |
| hardware | one RTX A6000, 11h44m |
Train loss 0.738 β 0.584; held-out 0.754 β **0.561**, below train at every checkpoint. No
memorisation, despite r=64 on 1,059 samples β which the transcript-level split makes a meaningful
statement rather than an artefact.
**This architecture is expensive to train.** 265 s/step at 16k sequence, against 15.7 s/step for a
same-size Qwen3-VL under identical batch settings β the 24 linear-attention layers have no fused
kernel in transformers 5.3.0 and run a sequential scan whose cost grows with sequence length.
Inference is cheap; training is not.
## Limits
- **One epoch, one checkpoint.** No epoch-2 comparison, which has mattered elsewhere in this
family: a sibling model won overall at epoch 2 while losing a specific behaviour it had at
epoch 1.
- **Targets are one model's judgement.** GPT-5 decided what mattered in each window. Where it
dropped something, this learned to drop it.
- **Task overlap is 0.509, not 0.9.** The task line is always present and usually about the right
thing; it is not word-for-word the reference. Treat it as "states the task" rather than "states
it identically".
- **One operator's transcripts**, 120 conversations of agentic coding. Other domains are untested.
|