README.md

---
base_model: Lazarus-Ai/ReAligned-Qwen3.5-4B
library_name: peft
license: apache-2.0
tags:
  - lora
  - sft
  - summarization
  - context-compaction
  - agents
  - egirl
---

# Compactor-Qwen3.5-4B

A context-compaction summariser for [egirl](https://github.com/Schneewolf-Labs/egirl): given a run
of agentic conversation about to be dropped, write the summary that lets the agent continue.

Meant to run as the **auxiliary model** — the operator keeps its slot and its context, and the
summary is produced by something small, deterministic, and trained for it.

## The failure it fixes

egirl compacts by dropping middle messages and replacing them with a summary. When that summary
loses the task, the agent does not get confused — **it invents a new task**:

```
Interior compaction: dropped 19 middle messages, kept head + 0 tail groups
Generated summary (396 chars) from 19 messages
```

Sixteen web searches of real research compressed to 396 characters, after which the agent answered
a question about agent-harness architecture with *"Fresh project scaffolded at
~/projects/fresh_project."* The request was gone from its context.

So the objective is not fluency. It is **preservation**: the task, the findings, the state.

## Measured

45 held-out compaction windows, from transcripts that contributed no training data. Deterministic
checks, no judge:

| | base | **+ Compactor** |
|---|---|---|
| has a `Task:` line | **0/45 (0%)** | **45/45 (100%)** |
| task overlap (word F1 vs reference) | 0.000 | **0.509** |
| **fact recall** (paths, identifiers, numbers, errors) | **0.157** | **0.661** |
| compression | 0.043 | **0.129** |
| mean output | 884 chars | 2,689 chars |

Reference summaries (GPT-5) compress to ~0.12×. The base compresses to 0.043× — it is not
summarising, it is discarding: **84% of the distinctive facts are gone**, and it never states the
task at all.

**Fact recall is the number to read.** A summary can be fluent, plausible, correctly shaped, and
still have the specifics sanded off — which is what a small model does by default, and what makes
the agent lose the thread three turns later.

## Use

```python
model = AutoModelForImageTextToText.from_pretrained(
    "Lazarus-Ai/ReAligned-Qwen3.5-4B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "schneewolflabs/Compactor-Qwen3.5-4B")
```

Render prompts with `enable_thinking=False` — training used the pre-closed think block, and the
model answers directly rather than deliberating. Serving with thinking open puts it in a state it
never saw.

In egirl, point `[local.auxiliary]` at a server running this and compaction routes to it
automatically.

## Training

| | |
|---|---|
| base | `Lazarus-Ai/ReAligned-Qwen3.5-4B` |
| method | SFT, LoRA **r=64 α=128** dropout 0.05 |
| data | [egirl-compaction-SFT](https://huggingface.co/datasets/schneewolflabs/egirl-compaction-SFT) — 1,059 train / 45 val |
| tokens | 9.34M |
| schedule | 1 epoch, 133 steps, lr 1e-4 cosine |
| sequence | 16,384 — an 8192 cap would have dropped 55% of rows |
| hardware | one RTX A6000, 11h44m |

Train loss 0.738 → 0.584; held-out 0.754 → **0.561**, below train at every checkpoint. No
memorisation, despite r=64 on 1,059 samples — which the transcript-level split makes a meaningful
statement rather than an artefact.

**This architecture is expensive to train.** 265 s/step at 16k sequence, against 15.7 s/step for a
same-size Qwen3-VL under identical batch settings — the 24 linear-attention layers have no fused
kernel in transformers 5.3.0 and run a sequential scan whose cost grows with sequence length.
Inference is cheap; training is not.

## Limits

- **One epoch, one checkpoint.** No epoch-2 comparison, which has mattered elsewhere in this
  family: a sibling model won overall at epoch 2 while losing a specific behaviour it had at
  epoch 1.
- **Targets are one model's judgement.** GPT-5 decided what mattered in each window. Where it
  dropped something, this learned to drop it.
- **Task overlap is 0.509, not 0.9.** The task line is always present and usually about the right
  thing; it is not word-for-word the reference. Treat it as "states the task" rather than "states
  it identically".
- **One operator's transcripts**, 120 conversations of agentic coding. Other domains are untested.