README_MAC.md

Mintoak RAG Assistant - macOS Setup Guide

This guide describes how to set up and run the Mintoak RAG Knowledge Assistant locally on macOS (optimized for Apple Silicon Apple M1/M2/M3/M4 chips using MLX).

---

Prerequisites

1. macOS: Ventura (13.0) or higher recommended.
2. Python: Python 3.9, 3.10, or 3.11 installed. (Check with python3 --version).
3. Git LFS: Install Git Large File Storage to manage weights/vectors:

   brew install git-lfs
   git lfs install
   

---

1. Environment Setup

It is highly recommended to use a dedicated virtual environment for MLX:

# Create a virtual environment named 'mlx-env'
python3 -m venv mlx-env

# Activate the virtual environment
source mlx-env/bin/activate

# Upgrade pip
pip install --upgrade pip

---

2. Installation of Dependencies

Install the requirements from requirements.txt along with the Apple-specific MLX inference framework (mlx-lm):

# Install core dependencies
pip install -r requirements.txt

# Install MLX framework (Apple Silicon acceleration)
pip install mlx-lm

---

3. Running the Assistant

There are two ways to run the assistant locally on macOS:

Option A: Local MLX Chat Server (Recommended for Apple Silicon)

This runs the local MLX-optimized version of the chat interface using 4-bit quantized Qwen 2.5 weights.

# Ensure your environment is active
source mlx-env/bin/activate

# Start the local server
python3 scripts/mintoak/chat_server.py

• Once started, open http://localhost:5001 in your browser.
• The local database will automatically populate with 900+ chunks on first run.

Option B: PyTorch/Transformers Server (Standard App)

This runs the CPU/GPU PyTorch version of the application (matching the Hugging Face Spaces deployment):

# Start the production Flask app
python3 app.py

• Access the interface at http://localhost:7860.

---

4. Running the Evaluation Suite

To test the RAG grounding performance, compliance filters (e.g. banned phrase checking), and out-of-scope refusals, run the evaluation script:

# Run tests on a batch of queries
python3 scripts/mintoak/evaluate_rag.py --max_cases 25

---

Troubleshooting

• ModuleNotFoundError: No module named 'mlx_lm': Make sure you have activated the virtual environment (source mlx-env/bin/activate) before running scripts.
• ChromaDB Issues: If you encounter SQLite or Chroma database conflicts, reset the local DB cache:

  rm -rf data/mintoak/chroma_db
  

  The database will automatically rebuild from mintoak_chunks.json on the next server start.