README.md

---
language:
- en
license: mit
library_name: transformers
tags:
- qlora
- smollm
- 360m
- cross-domain-transfer
- anime-isomorphism
- fine-tuned
pipeline_tag: text-generation
base_model: HuggingFaceTB/SmolLM2-360M-Instruct
---

# gnarp-m2

A 360M-parameter language model fine-tuned via QLoRA on 74,395 cross-domain isomorphism and verification-labeled instruction pairs. gnarp-m2 specializes in cross-domain structural transfer — mapping mechanisms from one domain (biology, physics, anime, economics, etc.) to software engineering constructs with concrete failure boundaries.

## Model Details

| Property | Value |
|----------|-------|
| Base model | HuggingFaceTB/SmolLM2-360M-Instruct |
| Method | QLoRA (4-bit NF4, double quantization) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable params | 8,683,520 (2.34% of total) |
| Total params | 370,504,640 |
| Adapter size | 34.8 MB (rank-16, alpha-32) |
| Merged model size | 1.4 GB |
| Architecture | LlamaForCausalLM |
| Max sequence length | 768 tokens |

## Training Data

**Corpus:** `clean_corpus_v5.jsonl` — 74,395 rows

| Source | Rows | Description |
|--------|------|-------------|
| isomorphism_sft.jsonl | 53,403 | Anime-to-software structural isomorphisms with failure_class from the 17,801-row moat |
| gpu_assay_verdicts.jsonl | 11,989 | GPU assay verification-labeled pairs |
| anime_metaphor_engine.jsonl | 4,523 | Cross-domain metaphor engine outputs |
| forge_bloom | 2,117 | Forge pipeline bloom outputs |
| capability/reasoning/seed.jsonl | 1,232 | Reasoning capability seed data |
| fleet_toolforge.jsonl | 187 | Fleet tool use pairs |
| Others (30+ sources) | 944 | Security, orchestration, calibration, refusal, compliance, etc. |

Positive + unlabeled rows only. Negative rows excluded from SFT targets.

## Training Configuration

| Parameter | Value |
|-----------|-------|
| Epochs | 1 |
| Learning rate | 1e-4 (cosine schedule, 5% warmup) |
| Batch size | 1 |
| Gradient accumulation | 8 |
| Effective batch size | 8 |
| Optimizer | AdamW (bf16) |
| Eval split | 10% held out |
| Eval strategy | Every 500 steps |
| Best model selection | eval_loss (load_best_model_at_end) |
| Seed | 7 |
| Training hardware | RTX 3080 Laptop (8 GB VRAM) |
| Training time | ~13 hours |

## Training Results

| Metric | Value |
|--------|-------|
| Final train loss | 2.613 |
| Final eval loss | 2.509 |
| Token accuracy | 55.55% |
| Perplexity (train) | 13.64 |

## Evaluation: Cross-Domain Transfer Benchmark v2

36 cross-domain transfer tasks spanning anime, biology, physics, economics, fiction, geography, music, cooking, ecology, martial arts, psychology, logistics, chemistry, sports, agriculture, linguistics, city planning, finance, navigation, and architecture.

**Scoring:** Heuristic rubric (keyword + structural analysis). Trust DELTAS between models on the same tasks, not absolutes.

| Model | Judge Mean | Delta vs Base | Avg Response (chars) | Avg Latency (s) |
|-------|-----------|---------------|---------------------|-----------------|
| **gnarp-m2** | **0.7839** | **+14.1%** | 1,108 | 4.9 |
| base (SmolLM2-360M-Instruct) | 0.6871 | — | 1,489 | 6.9 |

## Prior Model Lineage (heldout benchmark, qwen3:8b judge)

| Model | Transfer Score | Heldout Loss | Perplexity | Refusal Rate | Training Data |
|-------|---------------|-------------|-----------|-------------|---------------|
| base | 0.709 | 1.625 | 5.08 | 0.130 | — |
| v1 | 0.218 | 1.850 | 6.36 | 0.385 | ~2,152 rows |
| v2 | 0.713 | 1.800 | 6.05 | 0.340 | ~2,152 rows |
| v3 | 0.561 | 1.790 | 5.99 | 0.400 | ~2,152 rows |
| **m2** | **0.7839*** | — | — | — | **74,395 rows** |

*m2 scored on transfer_benchmark_v2 (heuristic-only), not the qwen3:8b-judged heldout benchmark. Cross-benchmark comparisons should be treated with caution.

## Limitations

1. **360M parameters.** Small model. Cannot match larger models on complex reasoning, long-form generation, or nuanced instruction following.
2. **Single GPU, single epoch.** Trained on consumer hardware (RTX 3080 8GB) for one epoch. More training could improve results but risks overfitting.
3. **Heuristic eval.** The transfer benchmark v2 uses keyword/structural heuristic scoring, not a strong LLM judge. The +14.1% delta is directionally meaningful but not precisely calibrated.
4. **Cross-benchmark caveat.** v1/v2/v3 were scored with a qwen3:8b judge; m2 was scored with heuristic-only. Direct numerical comparison across the two benchmarks is not valid.
5. **Domain-specific training data.** Over 71% of training data is isomorphism pairs. The model is optimized for cross-domain structural transfer and may underperform on general chat or coding tasks.
6. **No safety fine-tuning beyond refusal data.** The model includes 19 refusal pairs but is not extensively safety-tuned.

## How to Use

### With Ollama (recommended for local inference)

```bash
# Create the Modelfile
cat > Modelfile << 'EOF'
FROM ./model/merged_gnarpm2
TEMPLATE """### Instruction: {{ .Prompt }} ### Response: """
PARAMETER num_ctx 4096
PARAMETER temperature 0.3
PARAMETER num_predict 512
SYSTEM You are gnarp-m2, a cross-domain transfer specialist.
EOF

ollama create gnarp-m2 -f Modelfile
ollama run gnarp-m2
```

### With Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "gnarp/gnarp-m2"  # or local path
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "### Instruction:\nApply the concept of biological apoptosis to software deployment strategy.\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### With PEFT (adapter only)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")
model = PeftModel.from_pretrained(base, "path/to/adapter_gnarpm2")
model = model.merge_and_unload()
```

## License

- **Model weights and code:** MIT License
- **Training data (corpus):** COPL (Community Open Public License) — derived from the WaveMotionExpansion isomorphism engine
- **Base model:** Apache 2.0 (SmolLM2-360M-Instruct by HuggingFace)

## Citation

```bibtex
@model{gnarp-m2,
  title={gnarp-m2: Cross-Domain Transfer Fine-Tuned Language Model},
  author={WaveMotionExpansion},
  year={2026},
  base_model={HuggingFaceTB/SmolLM2-360M-Instruct},
  method={QLoRA},
  training_rows={74395},
  transfer_benchmark={0.7839}
}
```