README.md

---
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
library_name: transformers
pipeline_tag: any-to-any
base_model: google/gemma-4-12B-it
tags:
  - gemma4
  - multimodal
  - text-generation
  - image-text-to-text
  - audio-text-to-text
  - video-text-to-text
  - long-context
  - lora
  - bf16
  - llama-cpp
  - zh
  - en
model_name: Anrn-12B-R1
---

# Anrn 12B R1

**Anrn 12B R1** is a 12B dense, natively multimodal Gemma 4 derivative distributed as a byte-identical BF16 base plus a project LoRA adapter applied at a tested scale of `0.45`. The release supports text, image, audio, and video inputs through the native Transformers service and includes a ready-to-run llama.cpp text deployment.

> **Release layout:** this repository intentionally keeps the original BF16 base and the trained adapter separate rather than destructively merging them. The included launchers compose both parts at runtime.

## Benchmark results

![Anrn 12B R1 benchmark results](assets/anrn-12b-r1-benchmarks.jpg)

| Benchmark | Anrn 12B R1 |
|---|---:|
| MMLU-Pro | **74.2** |
| GPQA Diamond | **77.8** |
| AIME 2026 (no tools) | **69.6** |
| LiveCodeBench v6 | **68.3** |
| Codeforces Elo | **1521** |
| BigBench Extra Hard | **48.2** |
| Tau2 Average | **63.2** |
| MMMU Pro | **62.1** |
| MRCR v2 (128K) | **42.6** |

The Anrn values above come from the project's August 2026 evaluation run. The comparison values shown in the chart are reproduced from the published benchmark results used by that report; values marked with `~` are approximate.

## Model details

| Field | Value |
|---|---|
| Architecture | Gemma 4 Unified, dense decoder-only multimodal model |
| Parameters | 11.95B |
| Base model | `google/gemma-4-12B-it` |
| Native precision | BF16 |
| Adaptation | LoRA, rank 8, alpha 16 |
| Tested adapter scale | `0.45` |
| Native modalities | Text, image, audio, video input; text output |
| Tested service context | 131,072 tokens |
| Model configuration maximum | 262,144 positions |
| Languages | Multilingual; optimized project behavior in Chinese and English |
| Default prompt | Project constitution embedded in `system_prompt.md` |

## Repository structure

```text
model/           Original BF16 Gemma 4 12B weights and processor files
adapter/         Anrn R1 PEFT/LoRA adapter
llama-model/     Q5_K_M GGUF base, Q8_0 GGUF LoRA, and embedded chat template
llama-runtime/   Verified Windows CUDA llama.cpp runtime
assets/          Benchmark figure
start.ps1        Native multimodal OpenAI-compatible service
serve.py         Native service implementation
Start-Llama-Service.bat
                 One-click llama.cpp text Web UI
```

## Quick start: native multimodal service

Requirements: Windows, Python, a recent CUDA-capable NVIDIA GPU, and PowerShell.

```powershell
python -m pip install -r requirements.txt
.\start.ps1 -Port 8091
```

The launcher loads `model/`, mounts `adapter/` at scale `0.45`, injects `system_prompt.md`, uses PyTorch SDPA for long-context inference, and automatically offloads language layers to system memory when required. The tested configuration used approximately 14 GB of GPU memory.

OpenAI-compatible endpoint:

```text
POST http://127.0.0.1:8091/v1/chat/completions
```

Example health check:

```powershell
Invoke-RestMethod http://127.0.0.1:8091/health
```

## Quick start: llama.cpp Web UI

On Windows, double-click:

```text
Start-Llama-Service.bat
```

The bootstrap checks the bundled llama.cpp runtime and, if it is missing, downloads a matching official Windows CUDA/CPU build. It then starts the text Web UI with:

- 131,072-token context
- LoRA scale `0.45`
- Flash Attention
- Q8_0 K/V cache
- the project system prompt embedded in the Jinja chat template

Web UI: `http://127.0.0.1:8091`

Use `Stop-Llama-Service.bat` to stop it. The llama.cpp route is text-only; use `start.ps1` for native image, audio, and video inputs.

## Prompt behavior

`system_prompt.md` contains the release's default project constitution. The native server injects it before caller-provided system content. A request may set:

```json
{"use_default_system_prompt": false}
```

to disable the default prompt for that request. The llama.cpp prompt is embedded in `llama-model/chat_template.prompt30k.jinja`.

## Verification

The release contains machine-readable verification artifacts covering:

- native text, image, audio, and video loading
- prompt hash and injection behavior
- llama.cpp health, 128K context, and live LoRA scale
- identity smoke tests
- package checksums and rollback scripts

Primary records: `verification.json`, `llama-bootstrap-verification.json`, `prompt30k-verification.json`, and `SHA256SUMS.txt`.

## Limitations

- Benchmark scores are sensitive to prompt templates, decoding parameters, judge versions, and harness revisions.
- The BF16 base is large; CPU offload improves accessibility but reduces generation speed.
- The included llama.cpp path is an auxiliary quantized text deployment and does not expose the native multimodal input path.
- Users should evaluate the model for their target language, domain, and deployment policy before production use.

## License

Apache 2.0. See the [Gemma 4 license information](https://ai.google.dev/gemma/docs/gemma_4_license) and the upstream model terms.