VertexAGI/prism-caption-1-micro

Prism Caption 1 Micro

A tiny, single-purpose model that titles chats -- distilled from Nemotron-3-Nano-Omni into Gemma 3 1B

---

Overview

Prism Caption 1 Micro does exactly one thing: given a user's first message in a new conversation, it produces a short, specific chat title -- the same "New chat" auto-naming behavior ChatGPT, Claude, and similar assistants provide. Nothing else. It's built to be small and fast enough to run this single narrow task cheaply and locally rather than routing it to a larger general-purpose model.

Part of the Prism family of models.

Training

• Base model: mlx-community/gemma-3-1b-it-4bit
• Teacher: nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free (via OpenRouter's free tier)
• Dataset: 339 examples of realistic first-messages across ~50 everyday topics (coding, travel, cooking, finance, health, and more), paired with teacher-generated titles
• Method: LoRA fine-tuning (rank 8, 8 layers), 1,500 iterations
• Best validation loss: 0.163 (reached at iteration 800; the run was essentially converged from iteration ~300 onward)

Evaluation

A held-out base-vs-tuned comparison (20 fresh first-messages) shows the base Gemma-3-1B-it model is already fairly capable at this simple task -- the real gain from fine-tuning is format reliability:

Metric	Base Gemma-3-1B-it	Prism Caption 1 Micro
Bad-format outputs (too long / multiline / malformed)	3/20	0/20
Avg word count	4.0	4.0

Example outputs (base -> tuned):

• "Explaining the difference between TCP and UDP" -> "TCP vs UDP Explanation" -> "TCP vs UDP Overview"
• "How to start composting in a small apartment" -> "Start Small Apartment Composting" -> "Small Apartment Composting Guide"
• "Debugging a React component that re-renders infinitely" -> "React Component Re-renders" -> "React Component Re-render Issue"

Important inference note: Gemma 3's tokenizer registers <eos> (id 1) as its only default stop token, but the chat template actually ends each turn with <end_of_turn> (id 106). If your inference code doesn't also treat <end_of_turn> as a stop condition, generation will run past the correct title into garbage/padding tokens. See usage examples below.

Formats available

This repo includes both:

Format	File	Notes
MLX (4-bit)	model.safetensors + config	For Apple Silicon via mlx-lm
GGUF (Q4_K_M)	prism_caption_1_micro_Q4_K_M.gguf	For llama.cpp and compatible runtimes (LM Studio, Ollama, etc.)

Usage -- MLX

from mlx_lm import load, generate

model, tokenizer = load("VertexAGI/prism-caption-1-micro")
tokenizer.eos_token_ids.add(106)  # <end_of_turn> -- see inference note above

prompt = "How do I fix a leaking kitchen faucet?"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

title = generate(model, tokenizer, prompt=text, max_tokens=30)
print(title)

Usage -- GGUF (llama.cpp)

llama.cpp's own chat template handling for Gemma 3 already stops correctly at <end_of_turn>, so no extra flags are needed:

llama-cli -hf VertexAGI/prism-caption-1-micro -m prism_caption_1_micro_Q4_K_M.gguf -st -p "How do I fix a leaking kitchen faucet?"

System prompt

For best results, use the system prompt the model was trained with:

You name chat conversations. Given the user's first message, reply with ONLY a short, specific chat title (3-6 words, title case, no quotes, no punctuation at the end, no preamble). Nothing else -- just the title.

Limitations

This is a 1B-parameter model fine-tuned via LoRA on a small (339-example) dataset for one narrow task -- it will not perform well on anything outside chat-titling. Titles for highly ambiguous or very short first messages may be generic. Not intended as a general-purpose chat model.

License

Apache 2.0, inherited from the Gemma 3 base model.