Spaces:
Running
Running
| title: README | |
| emoji: 📚 | |
| colorFrom: green | |
| colorTo: indigo | |
| sdk: static | |
| pinned: false | |
| # MLX Community | |
| A community org for [MLX](https://github.com/ml-explore/mlx) model weights that run on Apple Silicon. This organization hosts ready-to-use models compatible with: | |
| - [mlx-lm](https://github.com/ml-explore/mlx-lm) - A Python package for LLM text generation and fine-tuning with MLX. | |
| - [mlx-swift-examples](https://github.com/ml-explore/mlx-swift-examples) – a Swift package to run MLX models. | |
| - [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) – A Python package for inference and fine-tuning of Vision Language Models (VLMs) using MLX. | |
| - [mlx-audio](https://github.com/Blaizzy/mlx-audio) - A Python and Swift Package for text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) for MLX. | |
| These are pre-converted weights, ready to use in the example scripts or integrate in your apps. | |
| # Quick start for LLMs | |
| Install `mlx-lm`: | |
| ``` | |
| pip install mlx-lm | |
| ``` | |
| You can use `mlx-lm` from the command line. For example: | |
| ``` | |
| mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --prompt "hello" | |
| ``` | |
| This will download a Mistral 7B model from the Hugging Face Hub and generate | |
| text using the given prompt. | |
| To chat with an LLM use: | |
| ```bash | |
| mlx_lm.chat | |
| ``` | |
| This will give you a chat REPL that you can use to interact with the LLM. The | |
| chat context is preserved during the lifetime of the REPL. | |
| For a full list of options run `--help` on the command of your interest, for example: | |
| ``` | |
| mlx_lm.chat --help | |
| ``` | |
| ## Conversion and Quantization | |
| To quantize a model from the command line run: | |
| ``` | |
| mlx_lm.convert --hf-path mistralai/Mistral-7B-Instruct-v0.3 -q | |
| ``` | |
| For more options run: | |
| ``` | |
| mlx_lm.convert --help | |
| ``` | |
| You can upload new models to Hugging Face by specifying `--upload-repo` to | |
| `convert`. For example, to upload a quantized Mistral-7B model to the | |
| [MLX Hugging Face community](https://huggingface.co/mlx-community) you can do: | |
| ``` | |
| mlx_lm.convert \ | |
| --hf-path mistralai/Mistral-7B-Instruct-v0.3 \ | |
| -q \ | |
| --upload-repo mlx-community/my-4bit-mistral | |
| ``` | |
| Models can also be converted and quantized directly in the | |
| [mlx-my-repo](https://huggingface.co/spaces/mlx-community/mlx-my-repo) Hugging | |
| Face Space. | |
| For more details on the API checkout the full [README](https://github.com/ml-explore/mlx-lm/tree/main) | |
| ### Other Examples: | |
| For more examples, visit the [MLX Examples](https://github.com/ml-explore/mlx-examples) repo. The repo includes examples of: | |
| - Image generation with Flux and Stable Diffusion | |
| - Parameter efficient fine tuning with LoRA | |
| - Speech recognition with Whisper | |
| - Multimodal models such as CLIP and LLaVA | |
| - Many other examples of different machine learning applications and algorithms |