Add Delta-NAS fine-tuned LoRA adapter (22 cycles)

README.md

@@ -0,0 +1,95 @@
+---
+base_model: deepseek-ai/deepseek-coder-7b-instruct-v1.5
+library_name: peft
+license: mit
+pipeline_tag: text-generation
+tags:
+- lora
+- code-generation
+- neural-architecture-search
+- delta-nas
+- pytorch
+---
+
+# Delta-NAS DeepSeek-Coder-7B-Instruct LoRA Adapter
+
+This is a LoRA adapter for [DeepSeek-Coder-7B-Instruct-v1.5](https://huggingface.co/deepseek-ai/deepseek-coder-7b-instruct-v1.5), fine-tuned for **delta-based Neural Architecture Search (NAS)** — generating novel PyTorch image-classification architectures via unified code diffs.
+
+## Model Description
+
+This adapter is the result of 22 iterative fine-tuning cycles on the delta-NAS pipeline described in **"Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs"**. The model generates unified diffs that modify a baseline neural network architecture to produce new, functional PyTorch models.
+
+### Training Details
+
+- **Base model**: `deepseek-ai/deepseek-coder-7b-instruct-v1.5`
+- **Fine-tuning method**: LoRA (Low-Rank Adaptation)
+- **LoRA rank (r)**: 32
+- **LoRA alpha**: 32
+- **LoRA dropout**: 0.05
+- **Target modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, lm_head
+- **Training cycles**: 22 (iterative self-improvement)
+- **Total trained candidates**: 828
+- **Admitted novel architectures**: 83 (MinHash-Jaccard novelty filter + τ_acc ≥ 0.40)
+
+### Evaluation Datasets
+
+Models were evaluated on 6 LEMUR image-classification benchmarks:
+- CIFAR-10, CIFAR-100, MNIST, SVHN, ImageNette, CelebA-Gender
+
+### Key Results
+
+| Metric | Value |
+|--------|-------|
+| Trained candidates | 828 |
+| Valid rate (compiles + trains) | 49.5% |
+| Mean 1-epoch accuracy | 33.9% (±7.9% SD across cycles) |
+| ≥40% accuracy rate | 16.6% |
+| Novel architectures admitted to LEMUR | 83 |
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    "deepseek-ai/deepseek-coder-7b-instruct-v1.5",
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-7b-instruct-v1.5")
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "ABrain/Delta-NAS-DeepSeek-Coder-7B")
+
+# Generate a diff to modify a baseline architecture
+prompt = """Given the following PyTorch neural network baseline:
+[baseline code here]
+
+Generate a unified diff that creates a novel architecture variant."""
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Associated Resources
+
+- **Code**: [ABrain-One/nn-gpt](https://github.com/ABrain-One/nn-gpt)
+- **Generated models**: [ABrain-One/nn-dataset PR #204](https://github.com/ABrain-One/nn-dataset/pull/204) (197 del-* prefixed architectures)
+- **Paper**: "Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs" (submitted to CVPR 2026)
+
+## Citation
+
+```bibtex
+@article{deltanas2026,
+  title={Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs},
+  author={Adhikari, Santosh and Ignatov, Dmitry},
+  year={2026}
+}
+```
+
+## License
+
+MIT License (same as the base model and LEMUR dataset)

adapter_config.json

@@ -0,0 +1,74 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "deepseek-ai/deepseek-coder-7b-instruct-v1.5",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23
+  ],
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "lm_head",
+    "gate_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d9757f113952f89efa6eee54ff1c32fa59cbdad92948815defbfec57f0977a4
+size 1931261344