apoapps/apochat-gemma4-e2b-litert-conversion

Apochat-tuned Gemma 4 E2B → LiteRT export

This folder contains a script that converts the public Apochat-tuned MLX model into a .litertlm package that the iOS/macOS app can run with the LiteRT backend.

Do not run this on the 16 GB local Mac — the conversion peaks at more than 16 GB of memory. Run it on a machine with at least:

• 32 GB of CPU RAM, or
• A GPU with 24 GB+ VRAM.

Quick start on Hugging Face

1. Create a GPU Space / Notebook (or any cloud VM) with Python 3.10+.
2. Clone this conversion repo:

   git clone https://huggingface.co/apoapps/apochat-gemma4-e2b-litert-conversion
   cd apochat-gemma4-e2b-litert-conversion
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Run the export:

   python export_apochat_litert.py \
       --mlx-repo apoapps/apochat-gemma4-e2b-apochat-tuned-v1 \
       --upload-repo apoapps/apochat-gemma4-e2b-apochat-tuned-v1-litert
   

The script will:

• download the MLX-q4 fused snapshot,
• dequantize it to bfloat16 PyTorch safetensors,
• run litert convert with weight-only int4 quantization,
• upload the resulting .litertlm to the --upload-repo.

Output

When it finishes, the artifact will be available at:

https://huggingface.co/apoapps/apochat-gemma4-e2b-apochat-tuned-v1-litert

Use that URL/revision/SHA to add the entry to Apochat/Sources/LocalAI/Catalog/BundledCatalog.swift.