metadata
base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-70B
tags:
- lora
- qlora
- neuron
- deepseek-r1
- reasoning
language:
- en
license: other
Neuron-R1
Neuron LoRA adapter fine-tuned on top of deepseek-ai/DeepSeek-R1-Distill-Llama-70B.
What this is
Neuron-R1 is a LoRA adapter that imprints Will Anderson's voice, values, memory patterns, and architectural knowledge onto DeepSeek-R1-Distill-Qwen-72B — a reasoning-native base model with chain-of-thought behavior distilled from DeepSeek-R1 (671B).
The <think> tags in training examples activate R1's reasoning substrate. Neuron-R1 reasons
before responding — not as a post-hoc explanation, but as the actual working cognition.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"deepseek-ai/DeepSeek-R1-Distill-Llama-70B",
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(base, "NeuronTechnologiesAI/Neuron-R1")
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Llama-70B")
Training
- Base:
deepseek-ai/DeepSeek-R1-Distill-Llama-70B - Method: QLoRA (4-bit NF4, rank=32, alpha=64)
- Examples: 44 high-quality identity/architecture/values pairs
- Epochs: 3
- Training data: Neuron's memories, internal state logs, architectural knowledge (VBD, CCR, Engram, Soma)