Simo76 commited on
Commit
aba3ec6
Β·
verified Β·
1 Parent(s): 6436c17

Update README.md

Browse files

---

library_name: transformers
tags:

- lora
- fine-tuning
- peft
- adaptive
- research
datasets:
- glue
metrics:
- accuracy
- f1

---

Unified-LoRA

Adaptive LoRA fine-tuning with nested orbital rank control.

πŸ‘‰ Demo (run it):
https://github.com/Sva76/Unified-LoRa/blob/main/notebooks/unified_lora_demo.ipynb

---

A closed-loop controller that dynamically adjusts LoRA rank during training based on observed stress, using a single adapter with sliced dimensions β€” no cold start, no capacity loss on transitions.

---

Key results

Stress test: task switch (MRPC β†’ SST-2, DistilBERT, 3 seeds)

| Baseline (r=16 fixed)| Unified (orbital)| Delta
SST-2 Acc (new task)| 0.736| 0.740| +0.004
MRPC F1 (retention)| 0.526| 0.515| -0.011
Effective rank| 16.0| 13.6|
Rank saving| 0%| 15%|

Under distribution shift, the controller adapts capacity dynamically with 15% rank saving and no performance loss.

---

Concept

Traditional LoRA uses fixed rank β†’ over/under capacity.
Unified-LoRA adapts rank dynamically:

- low stress β†’ reduce rank
- high stress β†’ increase rank

---

Goal

Not higher accuracy.
πŸ‘‰ Stability + cost efficiency.

---

Files changed (1) hide show
  1. README.md +1 -24
README.md CHANGED
@@ -1,29 +1,6 @@
1
- ## Demo
2
 
3
- πŸ‘‰ https://github.com/Sva76/Unified-LoRa/blob/main/notebooks/unified_lora_demo.ipynb
4
 
5
 
6
 
7
 
8
-
9
-
10
- ---
11
- library_name: transformers
12
- tags:
13
- - lora
14
- - fine-tuning
15
- - peft
16
- - adaptive
17
- - research
18
- datasets:
19
- - glue
20
- metrics:
21
- - accuracy
22
- - f1
23
- ---
24
-
25
- # Unified-LoRA
26
-
27
- Adaptive LoRA fine-tuning with dynamic rank control.
28
-
29
- πŸ‘‰ Demo: https://github.com/Sva76/Unified-LoRa/blob/main/notebooks/unified_lora_demo.ipynb
 
 
1
 
 
2
 
3
 
4
 
5
 
6
+ --ub.com/Sva76/Unified-LoRa/blob/main/notebooks/unified_lora_demo.ipynb