File size: 2,388 Bytes
0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 0e74c64 cfcec39 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | ---
base_model: unsloth/gemma-3-270m-it
library_name: peft
pipeline_tag: text-generation
license: gpl-3.0
tags:
- lora
- gemma
- resonate
- reasoning
- multilingual
language:
- en
- fr
- de
- ru
- he
- ar
- ja
- zh
---
# Gemma-3 270M-IT /resonate/ LoRA
**LoRA adapter** that teaches Gemma-3 270M-IT the `/resonate/` reasoning format — stream-of-consciousness thinking followed by a clean answer.
## What is /resonate/?
```
/resonate/
[free-form thinking — cynical, multilingual, associative, honest]
/resonated/
[clean, structured answer]
```
The model learns to THINK before answering. The `/resonate/` block is raw reasoning — it can switch languages, use metaphors, be irreverent. The `/resonated/` block is the distilled answer.
## Architecture
| | |
|---|---|
| Base | `unsloth/gemma-3-270m-it` (268.1M params) |
| Frozen | `embed_tokens` = 167.8M (63%) — **all 140 languages preserved** |
| LoRA | R=16, alpha=32, q_proj + v_proj only |
| Trainable | 0.74M (0.3% of total) |
| Training | 3 epochs, 6445 examples, 32 min on A100 |
| Best val loss | 2.9241 |
## Key insight
Freezing `embed_tokens` (63% of the model) preserves the multilingual embedding space. The LoRA adapter only modifies attention projections — teaching the model HOW to think, not WHAT languages to know.
## Languages verified working
English, French, German, Russian, Hebrew, Arabic, Japanese, Chinese — all generate coherent text with `/resonate/` format after fine-tuning.
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
tokenizer = AutoTokenizer.from_pretrained("ataeff/g")
base = AutoModelForCausalLM.from_pretrained("unsloth/gemma-3-270m-it", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "ataeff/g")
prompt = "<start_of_turn>user\nWhat is the meaning of life?<end_of_turn>\n<start_of_turn>model\n"
ids = tokenizer(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))
```
## Training data
- `resonance_yent_full.jsonl` — 6435 examples of /resonate/ format dialogues
- `resonance_gold_10.jsonl` — 10 hand-crafted gold examples (math, philosophy, code, multilingual)
## Part of the Arianna Method ecosystem
- [ariannamethod.ai](https://github.com/ariannamethod)
|