Add model card

README.md

@@ -0,0 +1,101 @@
+---
+base_model: mistralai/Leanstral-2603
+library_name: transformers
+tags:
+  - rotorquant
+  - kv-cache-quantization
+  - leanstral
+  - lean4
+  - formal-proofs
+  - theorem-proving
+  - quantized
+  - mistral
+  - moe
+license: apache-2.0
+---
+
+# Leanstral-RotorQuant
+
+**KV-cache quantized [Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) using [RotorQuant](https://github.com/scrya-com/rotorquant) for high-throughput Lean 4 formal proof generation.**
+
+Leanstral is the first open-source AI agent purpose-built for Lean 4 formal proofs -- generating both executable code and machine-checkable mathematical proofs. This variant applies RotorQuant KV-cache quantization, delivering **5.3x faster prefill** and **28% faster decode** compared to TurboQuant while preserving full BF16 model weights.
+
+## Overview
+
+This repository provides the **RotorQuant KV-cache-only** configuration of Leanstral-2603. The model weights remain at full precision; only the KV cache is quantized during inference using RotorQuant's rotation-aware quantization scheme.
+
+| Spec | Value |
+|------|-------|
+| Base model | [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) |
+| Architecture | Mistral MoE (~119B parameters, 7 consolidated shards) |
+| Compression | RotorQuant KV-cache quantization |
+| Weight precision | BF16 (unmodified) |
+| KV-cache precision | Mixed-precision quantized |
+| Prefill speedup | 5.3x vs TurboQuant |
+| Decode speedup | 28% vs TurboQuant |
+| License | Apache 2.0 |
+| Use case | Lean 4 formal verification, theorem proving, mathematical proofs |
+
+## Quickstart
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from turboquant import IsoQuantCache
+
+model_id = "majentik/Leanstral-RotorQuant"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype="auto",
+)
+
+# Enable RotorQuant KV-cache quantization
+cache = IsoQuantCache(model)
+
+prompt = "Prove that for all natural numbers n, n + 0 = n in Lean 4:"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    past_key_values=cache,
+)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## What is RotorQuant?
+
+[RotorQuant](https://github.com/scrya-com/rotorquant) is an advanced KV-cache quantization method that leverages rotation-aware quantization to achieve superior throughput compared to standard KV-cache compression. By exploiting the rotary positional embedding structure, RotorQuant achieves:
+
+- **5.3x faster prefill** -- critical for long Lean 4 proof contexts
+- **28% faster decode** -- faster token-by-token proof generation
+- Equivalent memory savings to TurboQuant with better computational efficiency
+
+This makes RotorQuant the preferred choice for interactive theorem proving sessions where latency matters.
+
+## Memory Estimates
+
+| Component | Estimate |
+|-----------|----------|
+| Model weights (BF16) | ~238 GB |
+| KV-cache savings | 2-4x reduction vs FP16 KV cache |
+| Recommended VRAM | 4x A100 80GB or equivalent |
+
+## Lean 4 Use Case
+
+Leanstral excels at:
+- **Formal verification** -- generating machine-checkable proofs of mathematical theorems
+- **Theorem proving** -- interactive and automated proof search in Lean 4
+- **Code generation** -- writing verified Lean 4 programs with correctness guarantees
+- **Proof repair** -- fixing incomplete or broken proof scripts
+
+## See Also
+
+- [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) -- Base model
+- [majentik/Leanstral-TurboQuant](https://huggingface.co/majentik/Leanstral-TurboQuant) -- TurboQuant KV-cache variant
+- [majentik/Leanstral-RotorQuant-MLX-4bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-4bit) -- MLX 4-bit + RotorQuant
+- [majentik/Leanstral-RotorQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-2bit) -- MLX 2-bit + RotorQuant
+- [majentik/Leanstral-RotorQuant-MLX-1bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-1bit) -- MLX 1-bit + RotorQuant
+- [RotorQuant repository](https://github.com/scrya-com/rotorquant)