Image-Text-to-Text
MLX
Safetensors
gemma4
rotorquant
kv-cache-quantization
gemma
multimodal
quantized
8bit
8-bit precision
Instructions to use majentik/gemma-4-31B-RotorQuant-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/gemma-4-31B-RotorQuant-MLX-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("majentik/gemma-4-31B-RotorQuant-MLX-8bit") config = load_config("majentik/gemma-4-31B-RotorQuant-MLX-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
docs: remove AWQ variants from family table (AWQ lane paused, repos private)
Browse files
README.md
CHANGED
|
@@ -115,13 +115,11 @@ This model requires approximately 31 GB of unified memory. Recommended hardware:
|
|
| 115 |
|
| 116 |
## Variants in this family
|
| 117 |
|
| 118 |
-
(Showing
|
| 119 |
|
| 120 |
| Variant | Runtime | Approx size | Use case |
|
| 121 |
|---|---|---|---|
|
| 122 |
| [RotorQuant](https://huggingface.co/majentik/gemma-4-31b-rotorquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 123 |
-
| [RotorQuant-AWQ-4bit](https://huggingface.co/majentik/gemma-4-31b-rotorquant-awq-4bit) | transformers | ~19 GB | GPU 4-bit (AutoAWQ) |
|
| 124 |
-
| [RotorQuant-AWQ-8bit](https://huggingface.co/majentik/gemma-4-31b-rotorquant-awq-8bit) | transformers | ~34 GB | GPU 8-bit (AutoAWQ) |
|
| 125 |
| [RotorQuant-GGUF-IQ4_XS](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-IQ4_XS) | llama.cpp | ~27 GB | Lossy 4-bit, low-RAM CPU/edge |
|
| 126 |
| [RotorQuant-GGUF-Q2_K](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-Q2_K) | llama.cpp | ~19 GB | Lossy, low-RAM CPU/edge |
|
| 127 |
| [RotorQuant-GGUF-Q3_K_M](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-Q3_K_M) | llama.cpp | ~24 GB | Smaller 3-bit, CPU-friendly |
|
|
@@ -132,8 +130,6 @@ This model requires approximately 31 GB of unified memory. Recommended hardware:
|
|
| 132 |
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-31b-rotorquant-mlx-4bit) | mlx-lm | ~19 GB | Apple Silicon balanced |
|
| 133 |
| **RotorQuant-MLX-8bit** | mlx-lm | ~37 GB | Apple Silicon reference |
|
| 134 |
| [TurboQuant](https://huggingface.co/majentik/gemma-4-31b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 135 |
-
| [TurboQuant-AWQ-4bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-awq-4bit) | transformers | ~19 GB | GPU 4-bit (AutoAWQ) |
|
| 136 |
-
| [TurboQuant-AWQ-8bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-awq-8bit) | transformers | ~34 GB | GPU 8-bit (AutoAWQ) |
|
| 137 |
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-2bit) | mlx-lm | ~9.9 GB | Apple Silicon, smallest |
|
| 138 |
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-4bit) | mlx-lm | ~19 GB | Apple Silicon balanced |
|
| 139 |
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-8bit) | mlx-lm | ~37 GB | Apple Silicon reference |
|
|
|
|
| 115 |
|
| 116 |
## Variants in this family
|
| 117 |
|
| 118 |
+
(Showing 14 sibling variants under `majentik/gemma-4-31b-*`. The current variant — `RotorQuant-MLX-8bit` — is **bolded**.)
|
| 119 |
|
| 120 |
| Variant | Runtime | Approx size | Use case |
|
| 121 |
|---|---|---|---|
|
| 122 |
| [RotorQuant](https://huggingface.co/majentik/gemma-4-31b-rotorquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
|
|
|
|
|
|
| 123 |
| [RotorQuant-GGUF-IQ4_XS](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-IQ4_XS) | llama.cpp | ~27 GB | Lossy 4-bit, low-RAM CPU/edge |
|
| 124 |
| [RotorQuant-GGUF-Q2_K](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-Q2_K) | llama.cpp | ~19 GB | Lossy, low-RAM CPU/edge |
|
| 125 |
| [RotorQuant-GGUF-Q3_K_M](https://huggingface.co/majentik/gemma-4-31b-rotorquant-gguf-Q3_K_M) | llama.cpp | ~24 GB | Smaller 3-bit, CPU-friendly |
|
|
|
|
| 130 |
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-31b-rotorquant-mlx-4bit) | mlx-lm | ~19 GB | Apple Silicon balanced |
|
| 131 |
| **RotorQuant-MLX-8bit** | mlx-lm | ~37 GB | Apple Silicon reference |
|
| 132 |
| [TurboQuant](https://huggingface.co/majentik/gemma-4-31b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
|
|
|
|
|
|
| 133 |
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-2bit) | mlx-lm | ~9.9 GB | Apple Silicon, smallest |
|
| 134 |
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-4bit) | mlx-lm | ~19 GB | Apple Silicon balanced |
|
| 135 |
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-31b-turboquant-mlx-8bit) | mlx-lm | ~37 GB | Apple Silicon reference |
|