RusNanoLM-800M / README.md
itismax0's picture
Add GGUF usage section (F16 variants for llama.cpp and Ollama)
d4caa02 verified
|
Raw History Blame Contribute Delete
6.87 kB
---
language:
- ru
- en
license: apache-2.0
tags:
- mera
- russian
- math
- reasoning
- qwen3.5
- linear-attention
- sub-1b
- sft
pipeline_tag: text-generation
library_name: transformers
metrics:
- accuracy
---
# RusNanoLM-800M
[English](#english-overview) | [Русский](#обзор-на-русском)
---
<a id="english-overview"></a>
## English Overview
**RusNanoLM-800M** is an open-weights **800M parameter bilingual (Russian / English) language model** built on a hybrid Linear Attention architecture (Qwen3.5 with Gated Delta Rule layers). It is evaluated on **MERA**, the Russian language understanding benchmark suite: on RuMultiAR a 0.8B model outperforms Qwen2.5-7B and GigaChat-2-10B.
### MERA Results (Pass@1, greedy, chat template)
| Benchmark | Tasks | RusNanoLM-0.8B | Qwen3.5-0.8B (baseline) | Δ |
|---|---|---|---|---|
| SimpleAR | 1000 | **83.2%** | 48.3% | +34.9 |
| RuMultiAR | 1039 | **33.5%** | 8.1% | +25.4 (×4.1) |
| MathLogicQA | 680 | **38.2%** | 33.9% | +4.4 |
| MaMuRaMu | 285 | **46.0%** | 43.6% | +2.4 |
| PARus | 500 | **53.4%** | 53.6% | -0.2 |
| RWSD | 810 | **49.9%** | 52.7% | -2.8 |
| Strict Constraints JSON | 12 | **58.3%** | - | - |
Evaluation date: 2026-11-17. Baseline numbers — official MERA submission of Qwen3.5-0.8B (submit id 16125).
### RuMultiAR vs larger models
| Model | Params | RuMultiAR |
|---|---|---|
| RuadaptQwen-32B | 32B | 43.3% |
| **RusNanoLM-0.8B (ours)** | **0.8B** | **33.5%** |
| GigaChat-2-10B | 10B | 31.5% |
| Qwen2.5-7B | 7B | 30.8% |
| Meta-Llama-3-8B | 8B | 24.8% |
| Llama-3.2-1B-Instruct | 1B | 16.2% |
| Qwen3.5-0.8B (baseline) | 0.8B | 8.1% |
### Visual Leaderboards
#### MERA Suite Summary
![MERA Suite Summary](assets/mera_suite_summary.png)
#### Baseline Progression
![Baseline Progression](assets/mera_baseline_progression.png)
#### SimpleAR
![SimpleAR](assets/mera_simplear.png)
#### RuMultiAR
![RuMultiAR](assets/mera_rumultiar.png)
#### MathLogicQA
![MathLogicQA](assets/mera_mathlogicqa.png)
#### MaMuRaMu
![MaMuRaMu](assets/mera_mamuramu.png)
---
### Quickstart & Inference
```python
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
MODEL_ID = "itismax0/RusNanoLM-800M"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForImageTextToText.from_pretrained(
MODEL_ID,
torch_dtype=torch.bfloat16,
device_map="auto"
)
model.eval()
messages = [
{"role": "user", "content": "Реши задачу: в корзине 5 яблок и 7 груш. Сколько всего фруктов?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
```
### Run with llama.cpp / Ollama (GGUF, full precision)
Two full-precision **F16 GGUF** files are included — no quantization loss. They differ only in tensor naming (llama.cpp renamed some Qwen3.5 tensors between b12257 and b12413):
| File | Use with |
|---|---|
| `RusNanoLM-800M-F16.gguf` | llama.cpp **>= b12413** (current master) |
| `RusNanoLM-800M-F16-ollama.gguf` | **Ollama** (all current versions) and other apps bundling llama.cpp <= b12257 (LM Studio, etc.) |
```bash
# llama.cpp (fresh build)
llama-cli -hf itismax0/RusNanoLM-800M:RusNanoLM-800M-F16.gguf --temp 0 -p "Реши: 13*7"
# Ollama — direct pull
ollama run hf.co/itismax0/RusNanoLM-800M:RusNanoLM-800M-F16-ollama.gguf "Вычисли: 13*7"
```
> Note: the model uses the Qwen3.5 hybrid linear-attention architecture. If you see `tensor 'blk.0.ssm_beta.weight' not found` in Ollama — use the `-ollama` GGUF variant. If LM Studio refuses to load, update its llama.cpp runtime (Settings → Runtimes).
---
<a id="обзор-на-русском"></a>
## Обзор на русском
**RusNanoLM-800M** — открытая двуязычная (русский и английский) модель на 800 млн параметров, построенная на гибридной архитектуре Qwen3.5 с линейным вниманием (Gated Delta Rule): 24 слоя, каждый четвёртый — полное внимание. Оценивалась на русскоязычном бенчмарке MERA (Pass@1, жадная генерация, chat template).
### Результаты MERA
| Бенчмарк | Задач | RusNanoLM-0.8B | Qwen3.5-0.8B (baseline) | Δ |
|---|---|---|---|---|
| SimpleAR | 1000 | **83.2%** | 48.3% | +34.9 |
| RuMultiAR | 1039 | **33.5%** | 8.1% | +25.4 (×4.1) |
| MathLogicQA | 680 | **38.2%** | 33.9% | +4.4 |
| MaMuRaMu | 285 | **46.0%** | 43.6% | +2.4 |
| PARus | 500 | **53.4%** | 53.6% | -0.2 |
| RWSD | 810 | **49.9%** | 52.7% | -2.8 |
| Strict Constraints JSON | 12 | **58.3%** | - | - |
### Главный результат: RuMultiAR против больших моделей
На арифметических рассуждениях (RuMultiAR) компактная модель 0.8B обгоняет модели в 8-12 раз тяжелее:
- **33.5%** — RusNanoLM-0.8B (наша)
- 31.5% — GigaChat-2-10B
- 30.8% — Qwen2.5-7B
- 24.8% — Meta-Llama-3-8B
- 8.1% — базовая Qwen3.5-0.8B (рост в 4.1 раза)
### Честные ограничения
- PARus (-0.2%) и RWSD (-2.8%) — на уровне или чуть ниже baseline: прирост получен за счёт арифметики и рассуждений, общее понимание текста не деградировало, но и не выросло.
- Strict Constraints JSON — всего 12 задач, выборка мала для сильных выводов.
### Графики
![Сводка MERA](assets/mera_suite_summary.png)
![Прогресс относительно baseline](assets/mera_baseline_progression.png)
![SimpleAR](assets/mera_simplear.png)
![RuMultiAR](assets/mera_rumultiar.png)
![MathLogicQA](assets/mera_mathlogicqa.png)
![MaMuRaMu](assets/mera_mamuramu.png)
### Архитектура и параметры
- Параметры: 800 млн (0.8B)
- Внимание: гибридное линейное внимание (Gated Delta Rule), сложность памяти состояния O(1)
- Слои: 24, каждый четвёртый — полное внимание
- Требования к видеопамяти: около 1.7 ГБ в формате BF16 (запускается на GPU с 4 ГБ VRAM)
- Шаблон диалога: ChatML (встроен в tokenizer_config)
- Лицензия: Apache 2.0