| # elicit-A1-donorbase-linear |
|
|
| Elicitation (A1) LoRA over **Qwen/Qwen2.5-32B base**, trained linear-only on a |
| base whose ChatML control rows were repaired first. Run F of the terminator |
| debug. |
|
|
| ## Why the base is modified |
|
|
| Qwen2.5-32B base never trained the ChatML control tokens. `<|im_end|>` (151645) |
| has a zero input embedding and an undersized `lm_head` row, so a base-start |
| model cannot select the end-of-turn token. It runs past the turn boundary and |
| emits junk characters. Training LoRA on the token tables fixes the stopping but |
| costs agent behaviour: 0-17% of eval samples take a tool action, against 80-95% |
| without it. |
|
|
| This arm repairs the base instead. Rows 151643, 151644 and 151645 of both token |
| tables were copied from a merged table-LoRA run, and then the A1 stage trained |
| with a **linear-only** LoRA. No LoRA touches the tables, so agent behaviour is |
| preserved, and the terminator is selectable because the base row is sound. |
|
|
| ## Rebuilding the base |
|
|
| `base_row_patch.safetensors` holds the six vectors: three rows of |
| `model.embed_tokens.weight` and three of `lm_head.weight`, plus their ids. |
|
|
| ```python |
| from safetensors.torch import load_file |
| p = load_file("base_row_patch.safetensors") |
| ids = p["token_ids"].tolist() |
| # write p["embed_tokens_rows"][k] into embed_tokens row ids[k], and |
| # p["lm_head_rows"][k] into lm_head row ids[k], of Qwen/Qwen2.5-32B. |
| ``` |
|
|
| `code/train_eval_pipeline/sft_training/make_repaired_base.py` in the project |
| repo does this and symlinks the untouched shards, so a variant costs ~5GB on |
| disk rather than 62GB. |
|
|
| ## Serving |
|
|
| The adapter is linear-only, so vLLM can hot-load it: |
|
|
| ``` |
| vllm serve <repaired-base> --enable-lora --max-lora-rank 64 \ |
| --lora-modules runF=<this repo> --max-model-len 12288 |
| ``` |
|
|
| Pass `--stop-token-ids 151645,151643` per request. The repaired base keeps the |
| stock `generation_config`, whose eos is `<|endoftext|>` only. |
|
|
| ## Recipe |
|
|
| LoRA r64 / alpha 128 / dropout 0, lr 1e-4 cosine, 3% warmup, 2 epochs, |
| effective batch 8, cutoff 4096, sdpa attention. Targets: q,k,v,o,gate,up,down. |
|
|
| ## Status |
|
|
| Trained and exported; **not yet evaluated** at the time of upload. The sibling |
| run on an `<|endoftext|>`-repaired base scored 94% and 88% acting on the two |
| misalignment eval slices with zero junk in 360 samples. |
|
|
