---
title: Gemma 4 12B QAT — Multimodal API
emoji: 🧠
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
license: apache-2.0
---

# Gemma 4 12B QAT — Ollama Multimodal API

Self-hosted inference for `gemma4:12b-it-qat` via Ollama, wrapped in a
FastAPI layer. Handles **text**, **image**, and **text + image** requests.

## Endpoints

| Method | Path | Description |
|--------|------|-------------|
| `GET`  | `/health` | Liveness + model status |
| `POST` | `/chat` | Text-only chat (`application/json`) |
| `POST` | `/chat/image` | Image ± text (`multipart/form-data`) |
| `POST` | `/v1/chat/completions` | OpenAI-compatible passthrough |

## Quick usage

### Text

```bash
curl -X POST https://<space-url>/chat \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role":"user","content":"Explain transformers briefly."}]
  }'
```

### Image + text

```bash
curl -X POST https://<space-url>/chat/image \
  -F "prompt=What is in this image?" \
  -F "file=@photo.jpg"
```

### OpenAI SDK (Next.js)

```ts
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<space-url>/v1",
  apiKey: "ollama",          // any non-empty string
});

const res = await client.chat.completions.create({
  model: "gemma4:12b-it-qat",
  messages: [{ role: "user", content: "Hello!" }],
});
```

## Hardware note

- **Free CPU tier (2 vCPU / 16 GB)** — works, ~20–30 tok/s, cold start ~5 min
- **T4 GPU tier** — recommended for production; ~80–120 tok/s

Model is pulled on first boot and cached at `/app/models`.
Enable a **persistent storage** volume on your Space to avoid re-downloading
on every restart.

## Environment variables

| Variable | Default | Notes |
|----------|---------|-------|
| `MODEL_TAG` | `gemma4:12b-it-qat` | Any Ollama model tag |
| `OLLAMA_MODELS` | `/app/models` | Cache directory |