README.md

---
language: en
license: apache-2.0
tags:
- test
- custom-architecture
- deepseek
---

# DeepSeek-R1-Channel-INT8_4L (4 Layers)

⚠️ **For Testing Purposes Only**  
This is a modified version of meituan/DeepSeek-R1-Channel-INT8 with **random weights**, used for architecture experiments.

## Key Modifications
- Reduced to **4 layers** 
- Contains:
  - First 3 layers: **MLA** (Multi-head Latent Attention)
  - Layer 4: **MoE** (Mixture of Experts)
- All weights randomly initialized (not performance-optimized)

## Usage
```python
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("MollyHexapotato/custom_DeepSeek-R1-Channel-INT8_4L")