File size: 1,749 Bytes
b3d6d01 0330d66 b3d6d01 0330d66 b3d6d01 0330d66 32c1f7e 0330d66 32c1f7e 0330d66 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
base_model: ricdomolm/mini-coder-1.7b
library_name: mlx
tags:
- mlx
- quantized
- 4-bit
- code-generation
---
# Mini-Coder 1.7B - MLX 4-bit
This is the [ricdomolm/mini-coder-1.7b](https://huggingface.co/ricdomolm/mini-coder-1.7b) model quantized into **4-bit MLX format** for native, ultra-fast execution on Apple Silicon devices (M1/M2/M3/M4 chips).
The conversion was performed to ensure the best trade-off between inference speed and the quality of the generated code, while keeping the unified RAM footprint to a minimum. I got 86 tps on MacBook Pro M4 16GB by using this model in LMStudio.
## How to use it with MLX
You can load and run this model directly in Python using the official `mlx-lm` library.
### 1. Installation
If you haven't already, install the necessary package:
```bash
pip install mlx-lm
```
### 2. Execution (Inference)
Here is a quick Python script to generate code:
```python
from mlx_lm import load, generate
# Loading the model from your Hugging Face hub
model_path = "fabriziosalmi/mini-coder-1.7b-mlx-4bit"
model, tokenizer = load(model_path)
prompt = "Write a Python function to calculate the Fibonacci sequence."
# If the model uses a specific chat template, apply it:
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(
model,
tokenizer,
prompt=prompt,
max_tokens=512,
verbose=True,
temp=0.2 # Keep the temperature low for better code generation
)
```
## Quantization Details
* **Framework:** MLX
* **Bits:** 4
* **Base Model:** ricdomolm/mini-coder-1.7b
|