Image-Text-to-Text
MLX
Safetensors
gemma4
rotorquant
kv-cache-quantization
gemma
multimodal
quantized
2bit
2-bit
Instructions to use majentik/gemma-4-E4B-RotorQuant-MLX-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/gemma-4-E4B-RotorQuant-MLX-2bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("majentik/gemma-4-E4B-RotorQuant-MLX-2bit") config = load_config("majentik/gemma-4-E4B-RotorQuant-MLX-2bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Remove links to retired duplicate brand variants
Browse files
README.md
CHANGED
|
@@ -114,7 +114,6 @@ This model requires approximately 1.2 GB of unified memory. Recommended hardware
|
|
| 114 |
- [google/gemma-4-E4B](https://huggingface.co/google/gemma-4-E4B) -- Base model
|
| 115 |
- [majentik/gemma-4-E4B-RotorQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-E4B-RotorQuant-MLX-8bit) -- MLX 8-bit variant
|
| 116 |
- [majentik/gemma-4-E4B-RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-E4B-RotorQuant-MLX-4bit) -- MLX 4-bit variant
|
| 117 |
-
- [majentik/gemma-4-E4B-TurboQuant-MLX-2bit](https://huggingface.co/majentik/gemma-4-E4B-TurboQuant-MLX-2bit) -- TurboQuant MLX 2-bit variant
|
| 118 |
- [RotorQuant GitHub](https://github.com/scrya-com/rotorquant)
|
| 119 |
- [MLX Framework](https://github.com/ml-explore/mlx)
|
| 120 |
|
|
@@ -147,6 +146,3 @@ This model requires approximately 1.2 GB of unified memory. Recommended hardware
|
|
| 147 |
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-e4b-rotorquant-mlx-4bit) | mlx-lm | ~2.5 GB | Apple Silicon balanced |
|
| 148 |
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-e4b-rotorquant-mlx-8bit) | mlx-lm | ~4.7 GB | Apple Silicon reference |
|
| 149 |
| [TurboQuant](https://huggingface.co/majentik/gemma-4-e4b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 150 |
-
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/gemma-4-e4b-turboquant-mlx-2bit) | mlx-lm | ~1.3 GB | Apple Silicon, smallest |
|
| 151 |
-
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-e4b-turboquant-mlx-4bit) | mlx-lm | ~2.5 GB | Apple Silicon balanced |
|
| 152 |
-
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-e4b-turboquant-mlx-8bit) | mlx-lm | ~4.7 GB | Apple Silicon reference |
|
|
|
|
| 114 |
- [google/gemma-4-E4B](https://huggingface.co/google/gemma-4-E4B) -- Base model
|
| 115 |
- [majentik/gemma-4-E4B-RotorQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-E4B-RotorQuant-MLX-8bit) -- MLX 8-bit variant
|
| 116 |
- [majentik/gemma-4-E4B-RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-E4B-RotorQuant-MLX-4bit) -- MLX 4-bit variant
|
|
|
|
| 117 |
- [RotorQuant GitHub](https://github.com/scrya-com/rotorquant)
|
| 118 |
- [MLX Framework](https://github.com/ml-explore/mlx)
|
| 119 |
|
|
|
|
| 146 |
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/gemma-4-e4b-rotorquant-mlx-4bit) | mlx-lm | ~2.5 GB | Apple Silicon balanced |
|
| 147 |
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/gemma-4-e4b-rotorquant-mlx-8bit) | mlx-lm | ~4.7 GB | Apple Silicon reference |
|
| 148 |
| [TurboQuant](https://huggingface.co/majentik/gemma-4-e4b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
|
|
|
|
|
|
|
|