docs/MINICPM_OMNI_SETUP.md

MiniCPM-o 4.5 Setup

⚠️ Optional voice mode — language warning. MiniCPM-o is not the default backend. The default is the text-only llama_cpp_server provider running OpenBMB MiniCPM4.1-8B (see .env.example), because the MiniCPM-o omni model frequently hallucinates and drifts into Chinese — especially in its TTS/audio branch — even when prompted in English. Use MiniCPM-o only if you need synthesized witness voices, and expect occasional Chinese output despite the English-only mitigations in llm/omni_client.py and app.py.

Phantom Grid expects three external components. Keep these outside the project so model weights and compiled binaries are not copied into source control.

1. Download the GGUF snapshot

Install the Hugging Face CLI and preserve the repository's nested module folders:

py -m pip install -U huggingface_hub
huggingface-cli download openbmb/MiniCPM-o-4_5-gguf --local-dir D:\Models\MiniCPM-o-4_5-gguf

The directory must contain one or more root LLM quantizations and all companion modules:

MiniCPM-o-4_5-gguf/
  MiniCPM-o-4_5-Q4_K_M.gguf
  audio/
  tts/
  token2wav-gguf/
  vision/

The Settings model scan lists root quantizations only and reports whether the audio, TTS, and Token2Wav modules are present.

2. Build llama.cpp-omni

The official Comni integration currently uses the feat/web-demo branch:

git clone https://github.com/tc-mb/llama.cpp-omni.git D:\Tools\llama.cpp-omni
Set-Location D:\Tools\llama.cpp-omni
git checkout feat/web-demo
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --target llama-server -j

The Comni branch expects build\bin\llama-server (or build\bin\Release\llama-server.exe for multi-config Windows builds). The packaged first-run installer uses project-local CMake, Ninja, and Zig for a CPU-capable build without requiring Visual Studio.

3. Install the Comni gateway

git clone https://github.com/OpenBMB/MiniCPM-o-Demo.git D:\Tools\MiniCPM-o-Demo
Set-Location D:\Tools\MiniCPM-o-Demo
git checkout Comni
py -3.10 -m venv .venv\base
.\.venv\base\Scripts\python.exe -m pip install -U pip
.\.venv\base\Scripts\python.exe -m pip install "torch==2.8.0" "torchaudio==2.8.0"
.\.venv\base\Scripts\python.exe -m pip install -r requirements.txt
Copy-Item config.example.json config.json

The Phantom Grid launcher updates config.json at launch with the selected model, context length, GPU layers, ports, and external paths. It starts one worker and an HTTP gateway at 127.0.0.1:8006 by default.

4. Configure Phantom Grid

Open Settings and fill in:

• Comni checkout: D:\Tools\MiniCPM-o-Demo
• llama.cpp-omni root: D:\Tools\llama.cpp-omni
• MiniCPM model directory: D:\Models\MiniCPM-o-4_5-gguf
• Quantization: choose a scanned root GGUF
• Context: 4096 to 32768
• GPU layers: auto, 0, or a non-negative integer

Press Start MiniCPM-o. First model load can take a minute or more. The browser will refuse to create or advance an AI case until the gateway health check succeeds.

Context adaptation

The selected context is also the game's memory budget. Smaller contexts retain fewer recent story segments and interview turns, while older events are compacted into a continuity synopsis. Larger contexts preserve more recent detail. Story decisions, observable facts, and persisted case history are never discarded.

Reference voices

Development reference WAVs live in data/voices. Each witness receives a stable voice ID and that WAV is supplied to MiniCPM-o for TTS and live interviews. Review data/voices/README.md before distributing a build.

Upstream references

• https://github.com/OpenBMB/MiniCPM-o-Demo/tree/Comni
• https://github.com/tc-mb/llama.cpp-omni/tree/feat/web-demo
• https://huggingface.co/openbmb/MiniCPM-o-4_5-gguf