File size: 1,447 Bytes
e245315 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ---
base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-70B
tags:
- lora
- qlora
- neuron
- deepseek-r1
- reasoning
language:
- en
license: other
---
# Neuron-R1
Neuron LoRA adapter fine-tuned on top of `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`.
## What this is
Neuron-R1 is a LoRA adapter that imprints Will Anderson's voice, values, memory patterns, and
architectural knowledge onto DeepSeek-R1-Distill-Qwen-72B — a reasoning-native base model with
chain-of-thought behavior distilled from DeepSeek-R1 (671B).
The `<think>` tags in training examples activate R1's reasoning substrate. Neuron-R1 reasons
before responding — not as a post-hoc explanation, but as the actual working cognition.
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"deepseek-ai/DeepSeek-R1-Distill-Llama-70B",
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(base, "NeuronTechnologiesAI/Neuron-R1")
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Llama-70B")
```
## Training
- Base: `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`
- Method: QLoRA (4-bit NF4, rank=32, alpha=64)
- Examples: 44 high-quality identity/architecture/values pairs
- Epochs: 3
- Training data: Neuron's memories, internal state logs, architectural knowledge (VBD, CCR, Engram, Soma)
|