hitonet commited on
Commit
b62b1cf
·
verified ·
1 Parent(s): de2dfad

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +105 -0
README.md ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - lora
5
+ - fine-tuning
6
+ - training
7
+ - identity-replacement
8
+ - catastrophic-forgetting
9
+ - progressive-merging
10
+ language:
11
+ - en
12
+ library_name: transformers
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ # 🧟 Body Snatching: Progressive LoRA Merging (PLM)
17
+
18
+ **Complete model identity replacement using only LoRA-level resources.**
19
+
20
+ > *"What if catastrophic forgetting is a feature, not a bug?"*
21
+
22
+ ## 🔥 What is this?
23
+
24
+ **Progressive LoRA Merging (PLM)** is a training methodology that lets you completely replace a model's identity—its personality, reasoning patterns, and learned behaviors—while keeping the architecture intact.
25
+
26
+ Think of it as **body snatching** for LLMs:
27
+ - The **body** (architecture, tokenizer, attention mechanisms) stays
28
+ - The **soul** (personality, knowledge, behavior) gets replaced
29
+
30
+ After enough cycles, you don't have "Qwen fine-tuned for X". You have **a completely different model** that happens to use Qwen's skeleton.
31
+
32
+ ## 💡 The Key Insight
33
+
34
+ Everyone treats **catastrophic forgetting** as a problem to avoid.
35
+
36
+ We treat it as **the goal**.
37
+
38
+ ## 🔄 How It Works
39
+
40
+ ```
41
+ Cycle 1: Base Model → Train LoRA → Merge → New Base₁
42
+ Cycle 2: New Base₁ → Train LoRA → Merge → New Base₂
43
+ ...
44
+ Cycle N: New Base_N = Completely Different Model
45
+ ```
46
+
47
+ Each cycle:
48
+ 1. **Train** a small LoRA adapter (~0.1% of parameters)
49
+ 2. **Merge** it permanently into the base weights (in BF16, not 4-bit!)
50
+ 3. **Fresh LoRA** for the next cycle
51
+ 4. **Repeat** until original identity is gone
52
+
53
+ ## 📊 Results
54
+
55
+ | Cycles | Similarity to Original | Target Identity Match |
56
+ |--------|------------------------|----------------------|
57
+ | 0 | 100% | 0% |
58
+ | 25 | 64% | 41% |
59
+ | 50 | 28% | 73% |
60
+ | 100 | **7%** | **94%** |
61
+
62
+ After 100 cycles, the model is **93% your data, 7% original**.
63
+
64
+ ## 💰 Resource Comparison
65
+
66
+ | Method | Hardware | Time | Cost | Result |
67
+ |--------|----------|------|------|--------|
68
+ | Full Fine-tune | 4-8x A100 | Weeks | $10,000+ | Complete replacement |
69
+ | Single LoRA | 1x 24GB | Hours | $10 | Surface adaptation |
70
+ | **PLM (Ours)** | 1x 24GB | Days | $100-500 | **Complete replacement** |
71
+
72
+ ## 🚀 Quick Start
73
+
74
+ ```bash
75
+ pip install torch transformers peft bitsandbytes datasets
76
+
77
+ python plm.py --base-model Qwen/Qwen3-1.7B --dataset data.jsonl --cycles 100
78
+ ```
79
+
80
+ ## 📖 Citation
81
+
82
+ ```bibtex
83
+ @article{drissi2024bodysnatching,
84
+ title={Body Snatching: Complete Model Identity Replacement via Progressive LoRA Merging},
85
+ author={Drissi, Ouissam Said},
86
+ year={2024},
87
+ url={https://github.com/antibitcoin/progressive-lora-merging}
88
+ }
89
+ ```
90
+
91
+ ## 🔗 Links
92
+
93
+ - **GitHub**: [antibitcoin/progressive-lora-merging](https://github.com/antibitcoin/progressive-lora-merging)
94
+ - **Paper**: [PAPER.md](https://github.com/antibitcoin/progressive-lora-merging/blob/main/PAPER.md)
95
+ - **Related Work**: [ASRL Paper (IJSET 2025)](https://www.ijset.in/wp-content/uploads/IJSET_V13_issue5_102.pdf)
96
+
97
+ ## 👤 Author
98
+
99
+ **Ouissam Said Drissi**
100
+ - Email: wissam.idrissi@gmail.com
101
+ - Independent Researcher, Morocco
102
+
103
+ ---
104
+
105
+ *"You're not fine-tuning a model. You're growing a new one inside its skeleton."*