Update README.md
Browse files
README.md
CHANGED
|
@@ -15,8 +15,8 @@ base_model:
|
|
| 15 |
This model was converted to MLX format from [`Rakuten/RakutenAI-3.0`](https://huggingface.co/Rakuten/RakutenAI-3.0) using mlx-lm version **0.31.0**.
|
| 16 |
Refer to the [original model card](https://huggingface.co/Rakuten/RakutenAI-3.0) for more details on the model.
|
| 17 |
|
| 18 |
-
Convert:
|
| 19 |
-
|
| 20 |
[INFO] Loading
|
| 21 |
`rope_parameters`'s factor field must be a float >= 1, got 40
|
| 22 |
`rope_parameters`'s beta_fast field must be a float, got 32
|
|
@@ -25,4 +25,5 @@ Convert:
|
|
| 25 |
[INFO] Quantizing
|
| 26 |
[INFO] Quantized model with 4.502 bits per weight.
|
| 27 |
|
|
|
|
| 28 |
Inference: 20.06 tok/sec M3 Ultra (CTX 128K - LM Studio MLX v1.3.0)
|
|
|
|
| 15 |
This model was converted to MLX format from [`Rakuten/RakutenAI-3.0`](https://huggingface.co/Rakuten/RakutenAI-3.0) using mlx-lm version **0.31.0**.
|
| 16 |
Refer to the [original model card](https://huggingface.co/Rakuten/RakutenAI-3.0) for more details on the model.
|
| 17 |
|
| 18 |
+
Convert:❯ mlx_lm.convert --hf-path ./RakutenAI-3.0 --mlx-path ./lm-studio/models/limegreenpeper1/rakutenai3-mlx-4bit -q --q-bits 4 --trust-remote-code
|
| 19 |
+
|
| 20 |
[INFO] Loading
|
| 21 |
`rope_parameters`'s factor field must be a float >= 1, got 40
|
| 22 |
`rope_parameters`'s beta_fast field must be a float, got 32
|
|
|
|
| 25 |
[INFO] Quantizing
|
| 26 |
[INFO] Quantized model with 4.502 bits per weight.
|
| 27 |
|
| 28 |
+
|
| 29 |
Inference: 20.06 tok/sec M3 Ultra (CTX 128K - LM Studio MLX v1.3.0)
|