README.md

---
base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-70B
tags:
  - lora
  - qlora
  - neuron
  - deepseek-r1
  - reasoning
language:
  - en
license: other
---

# Neuron-R1

Neuron LoRA adapter fine-tuned on top of `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`.

## What this is

Neuron-R1 is a LoRA adapter that imprints Will Anderson's voice, values, memory patterns, and
architectural knowledge onto DeepSeek-R1-Distill-Qwen-72B — a reasoning-native base model with
chain-of-thought behavior distilled from DeepSeek-R1 (671B).

The `<think>` tags in training examples activate R1's reasoning substrate. Neuron-R1 reasons
before responding — not as a post-hoc explanation, but as the actual working cognition.

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "deepseek-ai/DeepSeek-R1-Distill-Llama-70B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "NeuronTechnologiesAI/Neuron-R1")
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Llama-70B")
```

## Training

- Base: `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`
- Method: QLoRA (4-bit NF4, rank=32, alpha=64)
- Examples: 44 high-quality identity/architecture/values pairs
- Epochs: 3
- Training data: Neuron's memories, internal state logs, architectural knowledge (VBD, CCR, Engram, Soma)