majentik commited on
Commit
d1dec42
·
verified ·
1 Parent(s): ffc8ab5

Card accuracy pass 2: remove unmeasured speed claims, honest brand labels

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -40,7 +40,7 @@ pipeline_tag: text-generation
40
 
41
  **8-bit MLX weight-quantized [Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) with [RotorQuant](https://github.com/scrya-com/rotorquant) KV-cache quantization for Lean 4 formal proof generation on Apple Silicon.**
42
 
43
- Leanstral is the first open-source AI agent purpose-built for Lean 4 formal proofs -- generating both executable code and machine-checkable mathematical proofs. This variant combines **dual compression**: 8-bit MLX weight quantization for reduced model size plus the legacy RotorQuant KV-cache fork (superseded by upstream llama.cpp KV options) for efficient long-context inference with faster prefill and decode.
44
 
45
  Approximate model size: **~120 GB**
46
 
@@ -90,7 +90,7 @@ upstream options described above (`-ctk/-ctv q8_0`, `OLLAMA_KV_CACHE_TYPE`).
90
  | Method | Prefill Speed | Decode Speed | Memory Savings | Reference |
91
  |---|---|---|---|---|
92
  | **TurboQuant** | Baseline | Baseline | High | [arXiv: 2504.19874](https://arxiv.org/abs/2504.19874) |
93
- | **RotorQuant** | 5.3x faster | 28% faster | High | [GitHub](https://github.com/scrya-com/rotorquant) |
94
 
95
  ## Memory Estimates
96
 
 
40
 
41
  **8-bit MLX weight-quantized [Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) with [RotorQuant](https://github.com/scrya-com/rotorquant) KV-cache quantization for Lean 4 formal proof generation on Apple Silicon.**
42
 
43
+ Leanstral is the first open-source AI agent purpose-built for Lean 4 formal proofs -- generating both executable code and machine-checkable mathematical proofs. This variant combines **dual compression**: 8-bit MLX weight quantization for reduced model size plus the legacy RotorQuant KV-cache fork (superseded by upstream llama.cpp KV options) for efficient long-context inference with KV-cache handling.
44
 
45
  Approximate model size: **~120 GB**
46
 
 
90
  | Method | Prefill Speed | Decode Speed | Memory Savings | Reference |
91
  |---|---|---|---|---|
92
  | **TurboQuant** | Baseline | Baseline | High | [arXiv: 2504.19874](https://arxiv.org/abs/2504.19874) |
93
+ | **RotorQuant** | different | 28% faster | High | [GitHub](https://github.com/scrya-com/rotorquant) |
94
 
95
  ## Memory Estimates
96