Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -5,6 +5,7 @@ library_name: peft
|
|
| 5 |
base_model: Qwen/Qwen2.5-1.5B
|
| 6 |
tags:
|
| 7 |
- lora
|
|
|
|
| 8 |
- cognitive-architecture
|
| 9 |
- progressive-learning
|
| 10 |
- magnitude-pruning
|
|
@@ -15,46 +16,82 @@ datasets:
|
|
| 15 |
pipeline_tag: text-generation
|
| 16 |
---
|
| 17 |
|
| 18 |
-
#
|
| 19 |
|
| 20 |
-
**
|
| 21 |
|
| 22 |
-
##
|
| 23 |
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
-
##
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
```python
|
| 39 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 40 |
from peft import PeftModel
|
| 41 |
|
| 42 |
-
base_model = AutoModelForCausalLM.from_pretrained(
|
|
|
|
|
|
|
| 43 |
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B")
|
| 44 |
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
-
|
| 49 |
-
|
|
|
|
|
|
|
| 50 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 51 |
```
|
| 52 |
|
| 53 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
-
|
|
|
|
| 59 |
|
| 60 |
Apache 2.0
|
|
|
|
|
|
| 5 |
base_model: Qwen/Qwen2.5-1.5B
|
| 6 |
tags:
|
| 7 |
- lora
|
| 8 |
+
- peft
|
| 9 |
- cognitive-architecture
|
| 10 |
- progressive-learning
|
| 11 |
- magnitude-pruning
|
|
|
|
| 16 |
pipeline_tag: text-generation
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# Architettura Cognitiva Progressiva β Progressive-LoRA con Magnitude Pruning (Italiano)
|
| 20 |
|
| 21 |
+
**Primo prototipo** β Qwen2.5-1.5B addestrato con architettura cognitiva progressiva a 4 fasi, usando **magnitude pruning** (azzeramento pesi piccoli). Successivamente sostituito da SVD Dream Pruning.
|
| 22 |
|
| 23 |
+
## π Risultati
|
| 24 |
|
| 25 |
+
| Metrica | Progressive-LoRA (questo) | Dream-LoRA | Flat-LoRA |
|
| 26 |
+
|---------|--------------------------|-----------|-----------|
|
| 27 |
+
| Accuratezza Esatta | 37.0% Β± 0.5 | 58.6% Β± 2.9 | 60.6% |
|
| 28 |
+
| Number Sense | 57.7% Β± 0.5 | 60.0% Β± 0.8 | 0.0% |
|
| 29 |
+
| Metacognizione | 98.5% | 100.0% | 0.0% |
|
| 30 |
|
| 31 |
+
## π§ Architettura
|
| 32 |
|
| 33 |
+
Training progressivo a 4 fasi su dati aritmetici italiani:
|
| 34 |
+
1. **Fondamenta** β Aritmetica esatta
|
| 35 |
+
2. **Consolidamento** β Magnitude pruning + fine-tuning su approssimazioni
|
| 36 |
+
3. **Delega** β Routing complessitΓ : calcolo interno vs. strumento
|
| 37 |
+
4. **Orchestrazione** β Pipeline completa: intuizione β routing β tool β validazione
|
| 38 |
|
| 39 |
+
## π§ Configurazione
|
| 40 |
|
| 41 |
+
| Parametro | Valore |
|
| 42 |
+
|-----------|--------|
|
| 43 |
+
| Modello Base | Qwen/Qwen2.5-1.5B |
|
| 44 |
+
| LoRA Rank | 16 |
|
| 45 |
+
| LoRA Alpha | 32 |
|
| 46 |
+
| Target LoRA | q_proj, k_proj, v_proj, o_proj |
|
| 47 |
+
| Tipo Pruning | Magnitude (azzeramento pesi piccoli) |
|
| 48 |
+
| Lingua Dati | Italiano |
|
| 49 |
+
|
| 50 |
+
## π Uso Rapido
|
| 51 |
|
| 52 |
```python
|
| 53 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 54 |
from peft import PeftModel
|
| 55 |
|
| 56 |
+
base_model = AutoModelForCausalLM.from_pretrained(
|
| 57 |
+
"Qwen/Qwen2.5-1.5B", device_map="auto", torch_dtype="auto"
|
| 58 |
+
)
|
| 59 |
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B")
|
| 60 |
|
| 61 |
+
model = PeftModel.from_pretrained(
|
| 62 |
+
base_model,
|
| 63 |
+
"dexmac/progressive-cognitive-lora",
|
| 64 |
+
subfolder="lora_adapters"
|
| 65 |
+
)
|
| 66 |
|
| 67 |
+
messages = [{"role": "user", "content": "Calcola: 342 * 67"}]
|
| 68 |
+
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 69 |
+
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
| 70 |
+
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
|
| 71 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 72 |
```
|
| 73 |
|
| 74 |
+
## π Modelli Correlati
|
| 75 |
+
|
| 76 |
+
- [**Dream-LoRA (IT)**](https://huggingface.co/dexmac/progressive-cognitive-dream-lora) β Versione migliorata con SVD Dream Pruning
|
| 77 |
+
- [Flat-LoRA (IT)](https://huggingface.co/dexmac/progressive-cognitive-baseline-lora) β Controllo senza fasi
|
| 78 |
+
- [1.5B Dream (EN)](https://huggingface.co/dexmac/progressive-cognitive-dream-lora-en) β Miglior modello (inglese)
|
| 79 |
+
- [GitHub](https://github.com/dexmac221/progressive-cognitive) β Codice sorgente completo
|
| 80 |
|
| 81 |
+
## π Citation
|
| 82 |
+
|
| 83 |
+
```bibtex
|
| 84 |
+
@software{progressive_cognitive_2026,
|
| 85 |
+
author = {Dex Mac},
|
| 86 |
+
title = {Progressive Cognitive Architecture for LLMs},
|
| 87 |
+
year = {2026},
|
| 88 |
+
url = {https://github.com/dexmac221/progressive-cognitive},
|
| 89 |
+
version = {1.0.0}
|
| 90 |
+
}
|
| 91 |
+
```
|
| 92 |
|
| 93 |
+
|
| 94 |
+
## π License
|
| 95 |
|
| 96 |
Apache 2.0
|
| 97 |
+
|