README.md

# Apochat-tuned Gemma 4 E2B → LiteRT export

This folder contains a script that converts the public Apochat-tuned MLX model into a
`.litertlm` package that the iOS/macOS app can run with the LiteRT backend.

**Do not run this on the 16 GB local Mac** — the conversion peaks at more than 16 GB of
memory. Run it on a machine with at least:

- 32 GB of CPU RAM, or
- A GPU with 24 GB+ VRAM.

## Quick start on Hugging Face

1. Create a **GPU Space / Notebook** (or any cloud VM) with Python 3.10+.
2. Clone this conversion repo:
   ```bash
   git clone https://huggingface.co/apoapps/apochat-gemma4-e2b-litert-conversion
   cd apochat-gemma4-e2b-litert-conversion
   ```
3. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```
4. Run the export:
   ```bash
   python export_apochat_litert.py \
       --mlx-repo apoapps/apochat-gemma4-e2b-apochat-tuned-v1 \
       --upload-repo apoapps/apochat-gemma4-e2b-apochat-tuned-v1-litert
   ```

The script will:
- download the MLX-q4 fused snapshot,
- dequantize it to bfloat16 PyTorch safetensors,
- run `litert convert` with weight-only int4 quantization,
- upload the resulting `.litertlm` to the `--upload-repo`.

## Output

When it finishes, the artifact will be available at:

```
https://huggingface.co/apoapps/apochat-gemma4-e2b-apochat-tuned-v1-litert
```

Use that URL/revision/SHA to add the entry to `Apochat/Sources/LocalAI/Catalog/BundledCatalog.swift`.