mlx-vlm Setup on macOS (Apple Silicon)

This document outlines the steps taken to successfully install and run the mlx-vlm package on a macOS machine with Apple Silicon, resolving potential build issues related to dependencies like scipy.

Prerequisites

• Homebrew: Package manager for macOS. (https://brew.sh/)
• Python: A recent version of Python 3 (e.g., 3.10+). The log shows Python 3.13 was used.
• Git: For cloning repositories (implied, standard development tool).
• Build Tools & Libraries:
  • gcc: The GNU Compiler Collection.
  • libomp: OpenMP runtime library.
  • openblas: Optimized BLAS library (required by scipy). Although not explicitly installed in the final successful command sequence, the environment variables set indicate it's a necessary dependency that must be present (likely installed previously or as a dependency).

Setup Steps

1. Create and Activate Virtual Environment:
   It's recommended to use a virtual environment to manage dependencies.

   # Assuming you have cloned or created a project directory
   # cd your_project_directory
   python3 -m venv mlx-vlm
   source mlx-vlm/bin/activate
   

   (Replace mlx-vlm with your preferred environment name if desired)

2. Install Dependencies via Homebrew:
   Ensure gcc, libomp, and openblas are installed. If you encounter issues, make sure you have the latest versions or the specific versions required.

   brew install gcc libomp openblas
   

   (The log shows these were already installed, but this command ensures they are present.)

3. Configure Environment Variables for Build:
   This was the crucial step to resolve the scipy build error related to finding OpenBLAS. These variables tell the build system where to find the compilers and necessary libraries/headers installed by Homebrew.
   Note: The specific gcc version (gcc-14) was used. Adjust if your installed version differs.

   # Specify the GCC compilers from Homebrew
   export CC=$(brew --prefix gcc)/bin/gcc-14
   export CXX=$(brew --prefix gcc)/bin/g++-14
   
   # Set Linker flags to find OpenBLAS and libomp libraries
   export LDFLAGS="-L$(brew --prefix openblas)/lib -L$(brew --prefix libomp)/lib"
   
   # Set C Preprocessor flags to find OpenBLAS and libomp headers
   export CPPFLAGS="-I$(brew --prefix openblas)/include -I$(brew --prefix libomp)/include"
   
   # Set the pkg-config path to help find OpenBLAS configuration
   export PKG_CONFIG_PATH="$(brew --prefix openblas)/lib/pkgconfig"
   

   Using $(brew --prefix <formula>) makes the paths dynamic based on your Homebrew installation location.

4. Install mlx-vlm:
   With the environment configured, install the package using pip.

   pip install mlx-vlm
   

   (This command triggered the download and successful build/installation of mlx-vlm and its numerous dependencies, including scipy, mlx, transformers, etc.)

Usage Example

After successful installation, you can run the model inference command:

python -m mlx_vlm.generate --model mlx-community/gemma-3-4b-it-4bit --max-tokens 100 --temperature 0.0 --prompt "Your prompt here" --image your_image.png

(The example command used in the log was:

python -m mlx_vlm.generate --model mlx-community/gemma-3-4b-it-4bit --max-tokens 100 --temperature 0.0 --prompt "How many points to travel Saturday to Saturday" --image mexico-va.png

which successfully fetched the model and generated output.)

Troubleshooting Notes
The primary issue encountered was scipy failing to build due to not finding the `

All together:
brew install gcc libomp
export CC=gcc-14
export CXX=g++-14
export LDFLAGS="-L/opt/homebrew/opt/openblas/lib -L/opt/homebrew/opt/libomp/lib"
export CPPFLAGS="-I/opt/homebrew/opt/openblas/include -I/opt/homebrew/opt/libomp/include"
export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"
pip install mlx-vlm