instruct-mask-tools

Core tooling for instruct-preserving masked training and merging — the in-house method used by the rpDungeon org to train and merge style/prose adapters onto instruct models without destroying their instruction-following.

Shipped models built with this tooling: Gemma-4-E4B-Luchador and [Gemma-4-E4B-Luchador-Rudo](https://huggingface.co/rpDungeon/Gemma-4-E4B-Luchador-Rudo). Precomputed subspace masks / Fisher artifacts for several Gemma-4 checkpoints live in [rpDungeon/gemma-4-masks`](https://huggingface.co/rpDungeon/gemma-4-masks).

The method in one paragraph

The instruct model's behavior relative to its pretrained base lives, to first order, in a low-rank subspace of the weight delta W_IT − W_PT. At training time, LoRA gradients are projected out of the top-r SVD directions of that delta, so style training can't overwrite the instruction-following manifold. At merge time, incoming deltas are dampened per-parameter and per-layer by Fisher importance ((1 − F_param) · (1 − F_layer)), so the parameters the base model relies on most see the least change. A final spherical-linear (slerp) heal re-anchors the result.

Layout

• core/ — the pipeline scripts (default CLI paths assume this repo's sibling layout on the original box; override with flags)
  • Extraction: extract_subspace.py (IT−PT SVD subspace), extract_fisher.py / extract_head_fisher_31b.py (diagonal Fisher importance)
  • Training: train_e4b_lora.py (masked LoRA SFT/CPT — requires LOFT_INSTRUCT_MASK=1 or it silently runs unmasked), loft_chunked_nll_loss.py (chunked-NLL loss module), lora_ties_padded.py (rank-padded LoRA TIES helper)
  • Merging: ties_merge.py (LoRA-space TIES), merge_e4b_posthoc.py / merge_e4b_v2.py / merge_31b_posthoc.py / merge_31b_v2_full_gpu.py (post-hoc Fisher-gated merges), merge_e4b_lerp_slerp.py + build_v6_slerp.py (linear vs spherical interpolation), apply_subspace_no_sft.py
  • Utilities: transplant_embed_e4b_pt.py (embedding transplant — never LoRA-decompose embeddings)
  • Eval: vllm_ifeval.py (IFEval via vLLM server), score_eval.py, score_style_v3.py / score_style_v3_extended2.py (style/humanness scoring)
• docs/ — the guidelines corpus: technique reference + TL;DR (8 experimentally proven hard rules), training/eval/ablation/runpod/upload guidelines

Hard rules (the short list — see docs/INSTRUCT_MASKING_TLDR.md)

1. Always TIES-merge in LoRA space, never full-rank (full-rank TIES: −32 to −48 pp IFEval)
2. LOFT_INSTRUCT_MASK=1 env gate or training silently runs unmasked
3. fp32 for all SVD/Fisher math; downcast only at save
4. Focused Fisher (IFEval-only, ~200 prompts) beats multi-task Fisher
5. Use Fisher-derived layer_importance.json, not SVD-derived (they're uncorrelated)

Provenance

Recovered 2026-09-06 from development session logs after the original project directory was lost; each file header carries its recovery timestamp. Hard-coded default paths reference the original workstation layout — pass explicit paths via CLI flags when running elsewhere.