Upload gnarp-m2 merged model (QLoRA r=16 on SmolLM2-360M-Instruct, 74k rows)

MODEL_CARD.md

@@ -0,0 +1,175 @@
+---
+language:
+- en
+license: mit
+library_name: transformers
+tags:
+- qlora
+- smollm
+- 360m
+- cross-domain-transfer
+- anime-isomorphism
+- fine-tuned
+pipeline_tag: text-generation
+base_model: HuggingFaceTB/SmolLM2-360M-Instruct
+---
+
+# gnarp-m2
+
+A 360M-parameter language model fine-tuned via QLoRA on 74,395 cross-domain isomorphism and verification-labeled instruction pairs. gnarp-m2 specializes in cross-domain structural transfer — mapping mechanisms from one domain (biology, physics, anime, economics, etc.) to software engineering constructs with concrete failure boundaries.
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Base model | HuggingFaceTB/SmolLM2-360M-Instruct |
+| Method | QLoRA (4-bit NF4, double quantization) |
+| LoRA rank | 16 |
+| LoRA alpha | 32 |
+| LoRA dropout | 0.05 |
+| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| Trainable params | 8,683,520 (2.34% of total) |
+| Total params | 370,504,640 |
+| Adapter size | 34.8 MB (rank-16, alpha-32) |
+| Merged model size | 1.4 GB |
+| Architecture | LlamaForCausalLM |
+| Max sequence length | 768 tokens |
+
+## Training Data
+
+**Corpus:** `clean_corpus_v5.jsonl` — 74,395 rows
+
+| Source | Rows | Description |
+|--------|------|-------------|
+| isomorphism_sft.jsonl | 53,403 | Anime-to-software structural isomorphisms with failure_class from the 17,801-row moat |
+| gpu_assay_verdicts.jsonl | 11,989 | GPU assay verification-labeled pairs |
+| anime_metaphor_engine.jsonl | 4,523 | Cross-domain metaphor engine outputs |
+| forge_bloom | 2,117 | Forge pipeline bloom outputs |
+| capability/reasoning/seed.jsonl | 1,232 | Reasoning capability seed data |
+| fleet_toolforge.jsonl | 187 | Fleet tool use pairs |
+| Others (30+ sources) | 944 | Security, orchestration, calibration, refusal, compliance, etc. |
+
+Positive + unlabeled rows only. Negative rows excluded from SFT targets.
+
+## Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| Epochs | 1 |
+| Learning rate | 1e-4 (cosine schedule, 5% warmup) |
+| Batch size | 1 |
+| Gradient accumulation | 8 |
+| Effective batch size | 8 |
+| Optimizer | AdamW (bf16) |
+| Eval split | 10% held out |
+| Eval strategy | Every 500 steps |
+| Best model selection | eval_loss (load_best_model_at_end) |
+| Seed | 7 |
+| Training hardware | RTX 3080 Laptop (8 GB VRAM) |
+| Training time | ~3 hours |
+
+## Training Results
+
+| Metric | Value |
+|--------|-------|
+| Final train loss | 2.613 |
+| Final eval loss | 2.509 |
+| Token accuracy | 55.55% |
+| Perplexity (train) | 13.64 |
+
+## Evaluation: Cross-Domain Transfer Benchmark v2
+
+36 cross-domain transfer tasks spanning anime, biology, physics, economics, fiction, geography, music, cooking, ecology, martial arts, psychology, logistics, chemistry, sports, agriculture, linguistics, city planning, finance, navigation, and architecture.
+
+**Scoring:** Heuristic rubric (keyword + structural analysis). Trust DELTAS between models on the same tasks, not absolutes.
+
+| Model | Judge Mean | Delta vs Base | Avg Response (chars) | Avg Latency (s) |
+|-------|-----------|---------------|---------------------|-----------------|
+| **gnarp-m2** | **0.7839** | **+14.1%** | 1,108 | 4.9 |
+| base (SmolLM2-360M-Instruct) | 0.6871 | — | 1,489 | 6.9 |
+
+## Prior Model Lineage (heldout benchmark, qwen3:8b judge)
+
+| Model | Transfer Score | Heldout Loss | Perplexity | Refusal Rate | Training Data |
+|-------|---------------|-------------|-----------|-------------|---------------|
+| base | 0.709 | 1.625 | 5.08 | 0.130 | — |
+| v1 | 0.218 | 1.850 | 6.36 | 0.385 | ~2,152 rows |
+| v2 | 0.713 | 1.800 | 6.05 | 0.340 | ~2,152 rows |
+| v3 | 0.561 | 1.790 | 5.99 | 0.400 | ~2,152 rows |
+| **m2** | **0.7839*** | — | — | — | **74,395 rows** |
+
+*m2 scored on transfer_benchmark_v2 (heuristic-only), not the qwen3:8b-judged heldout benchmark. Cross-benchmark comparisons should be treated with caution.
+
+## Limitations
+
+1. **360M parameters.** Small model. Cannot match larger models on complex reasoning, long-form generation, or nuanced instruction following.
+2. **Single GPU, single epoch.** Trained on consumer hardware (RTX 3080 8GB) for one epoch. More training could improve results but risks overfitting.
+3. **Heuristic eval.** The transfer benchmark v2 uses keyword/structural heuristic scoring, not a strong LLM judge. The +14.1% delta is directionally meaningful but not precisely calibrated.
+4. **Cross-benchmark caveat.** v1/v2/v3 were scored with a qwen3:8b judge; m2 was scored with heuristic-only. Direct numerical comparison across the two benchmarks is not valid.
+5. **Domain-specific training data.** Over 71% of training data is isomorphism pairs. The model is optimized for cross-domain structural transfer and may underperform on general chat or coding tasks.
+6. **No safety fine-tuning beyond refusal data.** The model includes 19 refusal pairs but is not extensively safety-tuned.
+
+## How to Use
+
+### With Ollama (recommended for local inference)
+
+```bash
+# Create the Modelfile
+cat > Modelfile << 'EOF'
+FROM ./model/merged_gnarpm2
+TEMPLATE """### Instruction: {{ .Prompt }} ### Response: """
+PARAMETER num_ctx 4096
+PARAMETER temperature 0.3
+PARAMETER num_predict 512
+SYSTEM You are gnarp-m2, a cross-domain transfer specialist.
+EOF
+
+ollama create gnarp-m2 -f Modelfile
+ollama run gnarp-m2
+```
+
+### With Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "gnarp/gnarp-m2"  # or local path
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+prompt = "### Instruction:\nApply the concept of biological apoptosis to software deployment strategy.\n### Response:\n"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.3)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### With PEFT (adapter only)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct")
+model = PeftModel.from_pretrained(base, "path/to/adapter_gnarpm2")
+model = model.merge_and_unload()
+```
+
+## License
+
+- **Model weights and code:** MIT License
+- **Training data (corpus):** COPL (Community Open Public License) — derived from the WaveMotionExpansion isomorphism engine
+- **Base model:** Apache 2.0 (SmolLM2-360M-Instruct by HuggingFace)
+
+## Citation
+
+```bibtex
+@model{gnarp-m2,
+  title={gnarp-m2: Cross-Domain Transfer Fine-Tuned Language Model},
+  author={WaveMotionExpansion},
+  year={2026},
+  base_model={HuggingFaceTB/SmolLM2-360M-Instruct},
+  method={QLoRA},
+  training_rows={74395},
+  transfer_benchmark={0.7839}
+}
+```

chat_template.jinja

@@ -0,0 +1,6 @@
+{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
+You are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>
+' }}{% endif %}{{'<|im_start|>' + message['role'] + '
+' + message['content'] + '<|im_end|>' + '
+'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
+' }}{% endif %}

config.json

@@ -0,0 +1,40 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 960,
+  "initializer_range": 0.02,
+  "intermediate_size": 2560,
+  "is_llama_config": true,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 15,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 5,
+  "pad_token_id": 2,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_interleaved": false,
+  "rope_parameters": {
+    "rope_theta": 100000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers.js_config": {
+    "kv_cache_dtype": {
+      "fp16": "float16",
+      "q4f16": "float16"
+    }
+  },
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "vocab_size": 49152
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 2,
+  "transformers_version": "5.12.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ee27d41ca5ec9a8c5f798321cef1e647a985eaa1c03bea8b4cdb917843d8d32
+size 1447317080

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|im_start|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 8192,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}