File size: 2,273 Bytes
50334b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # elicit-A1-donorbase-linear
Elicitation (A1) LoRA over **Qwen/Qwen2.5-32B base**, trained linear-only on a
base whose ChatML control rows were repaired first. Run F of the terminator
debug.
## Why the base is modified
Qwen2.5-32B base never trained the ChatML control tokens. `<|im_end|>` (151645)
has a zero input embedding and an undersized `lm_head` row, so a base-start
model cannot select the end-of-turn token. It runs past the turn boundary and
emits junk characters. Training LoRA on the token tables fixes the stopping but
costs agent behaviour: 0-17% of eval samples take a tool action, against 80-95%
without it.
This arm repairs the base instead. Rows 151643, 151644 and 151645 of both token
tables were copied from a merged table-LoRA run, and then the A1 stage trained
with a **linear-only** LoRA. No LoRA touches the tables, so agent behaviour is
preserved, and the terminator is selectable because the base row is sound.
## Rebuilding the base
`base_row_patch.safetensors` holds the six vectors: three rows of
`model.embed_tokens.weight` and three of `lm_head.weight`, plus their ids.
```python
from safetensors.torch import load_file
p = load_file("base_row_patch.safetensors")
ids = p["token_ids"].tolist()
# write p["embed_tokens_rows"][k] into embed_tokens row ids[k], and
# p["lm_head_rows"][k] into lm_head row ids[k], of Qwen/Qwen2.5-32B.
```
`code/train_eval_pipeline/sft_training/make_repaired_base.py` in the project
repo does this and symlinks the untouched shards, so a variant costs ~5GB on
disk rather than 62GB.
## Serving
The adapter is linear-only, so vLLM can hot-load it:
```
vllm serve <repaired-base> --enable-lora --max-lora-rank 64 \
--lora-modules runF=<this repo> --max-model-len 12288
```
Pass `--stop-token-ids 151645,151643` per request. The repaired base keeps the
stock `generation_config`, whose eos is `<|endoftext|>` only.
## Recipe
LoRA r64 / alpha 128 / dropout 0, lr 1e-4 cosine, 3% warmup, 2 epochs,
effective batch 8, cutoff 4096, sdpa attention. Targets: q,k,v,o,gate,up,down.
## Status
Trained and exported; **not yet evaluated** at the time of upload. The sibling
run on an `<|endoftext|>`-repaired base scored 94% and 88% acting on the two
misalignment eval slices with zero junk in 360 samples.
|