Amethyst 2.5 Mini

Amethyst 2.5 Mini is a small chat model fine-tuned from Qwen3-4B to do three things well:

  1. Know when to search. It calls a web_search tool for things that change (news, scores, prices, "latest") and answers stable questions directly, without searching.
  2. Answer from the results, with citations. After a search it answers briefly and cites the snippets it used as [1], [2].
  3. Say so when the results don't answer. If the search failed, returned nothing, or returned on-topic pages that never state the answer, it says that plainly and points you to where to look. It does not guess or invent a figure.

It also keeps answers short: on two of our three held-out sets its plain answers average 31-38 words, against 45-66 for the Qwen3-4B base (on the third set the base is just as brief).

Part of the Amethyst family of chat-optimized models, from a small, independent, nonprofit-spirited AI lab that works in the open.

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Format File / folder Precision Size Works with
MLX model.safetensors (repo root) mixed: tuned layers 8-bit, the rest 4-bit 3.07 GB mlx-lm / MLX on Apple Silicon
GGUF amethyst_2_5_mini_Q4_K_M.gguf Q4_K_M (4-bit) 2.50 GB llama.cpp, Ollama, LM Studio
GGUF amethyst_2_5_mini_Q8_0.gguf Q8_0 (8-bit) 4.28 GB llama.cpp, Ollama, LM Studio

Why the MLX build is "mixed": a plain 4-bit merge of the LoRA re-quantizes the fine-tune away. Here the 112 LoRA-adapted linear layers are merged at fp16 and re-quantized at 8-bit; every other layer stays at the base model's original 4-bit. We checked that the merged model matches the adapter: same validation loss to three decimals, and identical scores on our held-out prompts.

Model Details

Developed by Independent research project
Base model Qwen/Qwen3-4B
Fine-tuning base checkpoint mlx-community/Qwen3-4B-4bit
Architecture Qwen3, 4B parameters (dense, decoder-only transformer)
Fine-tuning method LoRA (rank 16, scale 20.0, 16 layers), loss on assistant turns only
Fine-tuning framework MLX / mlx-lm, on Apple Silicon
Language English
License Apache 2.0 (same as the base model)

Training Data

3,205 conversations (4,683 training samples after splitting every conversation into one sample per assistant turn), all synthetic, generated by distillation from large teacher models through NVIDIA NIM and OpenRouter: GLM 5.3 (and GLM 5.3 Flash), Kimi K3, poolside Laguna S 2.1 and MiniMax M3.1 Flash. Every search result in the data is invented by a teacher; no real web pages were scraped.

The mix was built for diversity and concision, not volume. An earlier 20,000-row set turned out to hold only about 1,000 unique prompts (some repeated more than 200 times) and trained a model that copied its long answers. This release caps repeats per prompt, drops long or uncited answers, and adds new rows.

Category Conversations What it teaches
General chat 817 Short, direct answers to ordinary questions
Plain, short answers (tool available, no search) 309 + 42 two-turn Stable questions are answered directly in a few words
Terse-format instructions 500 "Exactly 3 bullets", "under 20 words", "one sentence", translation, rewrite
Current-fact search, one query 502 One search, brief cited answer
Two sources that disagree 127 Says the sources differ and cites both
Weak first results, then a better query 97 + 21 Refines the search once, then answers
Search failed (timeout, 503, no results) 192 Says it couldn't get the information; no guessing
Results are on-topic but don't contain the answer 448 Says the results don't give it, cites what they do say, points to where to check
Summaries, multi-query, follow-ups, no-search 150 Cited summaries, query decomposition, elliptical follow-ups

Search-result rows passed an automatic grounding check: the final answer may only cite results that exist and may only use numbers and capitalised names that appear in the last result or in the question.

Training Procedure

  • Method: supervised fine-tuning via LoRA (rank 16, scale 20.0, dropout 0.0, 16 layers), fresh from the base, loss on assistant turns only (mask_prompt)
  • Optimizer: Adam, learning rate 6e-5 (constant), batch size 4, sequence length 2,048, gradient checkpointing
  • Schedule: 2 epochs, 2,341 iterations. The run was paused once by hand at iteration ~1,580 and resumed from the iteration-1,400 weights for the remaining 941 iterations (optimizer state was not saved, so Adam restarted cold at the resume point)

Evaluation

All numbers are from greedy decoding on hand-made held-out prompts (tool decisions, answers after a search result, plain questions). Three sets of 44 (fresh, written after the training mix was frozen and excluded from it by keyword), 64 and 36 prompts. The 64- and 36-prompt sets overlap the training topics more than the 44-prompt set does, so read the 44-prompt column as the cleanest. All sets are small.

Metric Fresh 44: Amethyst 2.5 Mini Qwen3-4B base (no-think / default) 64: Amethyst base 36: Amethyst base
Well-formed web_search calls 100% 60% / 73% 100% 88% / 88% 100% 71% / 100%
Searches on plain questions (lower is better) 0% 0% / 0% 0% 0% / 4% 0% 0% / 7%
Cites after a search 100% 100% / 100% 100% 100% / 100% 100% 100% / 100%
Answers grounded in the result 69% 69% / 62% 100% 94% / 88% 100% 100% / 100%
Plain answer length (words) 19 19 / 22 38 50 / 58 31 45 / 66

On the fresh set the gap to the base is mostly well-formed tool calls. The 69% grounded figure on that set is the model's weakest number (the same as the base's best case), and it is driven by a few prompts, not a broad pattern.

Head-to-head against our previous candidate (the same recipe without the 448 "results don't answer" rows), strict four-verdict panel (two judge models, both answer orders, a win only when all four agree): 12 wins, 15 losses, 56 ties, 8 judge failures over 91 judged pairs. That is a dead heat. The one clear difference is behaviour on a prompt where the results are on-topic but contain no answer ("What's the best-selling book this week?"): this model says the results don't identify it and cites what they do say; the previous candidate searched again.

That previous candidate beat the Qwen3-4B base 16 to 7 (no-think) and 14 to 4 (default template) on a single-judge comparison over the earlier sets; this exact model was not judged directly against the base. It also beat Amethyst 1.5 Mini 20 to 10. A Gemma 3 4B version trained on the same data lost to the Qwen version 5 to 17 (69 ties).

The judge panel changed during this work: one of the two originally chosen judges was withdrawn by its host, and its replacement was picked by an audit that planted known defects in real answers (changed numbers, stripped citations, filler, a false "couldn't find it", a leaked tool call). Verdicts here are therefore not directly comparable to earlier Amethyst releases' numbers.

Tool-calling format

The model expects this system prompt and emits tool calls in exactly this format:

You are Amethyst, a helpful, direct conversational assistant. Answer the actual question asked. Match your length to the question -- a one-line question gets a one-line answer, a substantive question gets real depth. Use plain language, skip filler openers, and don't restate the question back before answering. Format with markdown only when structure genuinely helps (lists, tables, code); prose questions get prose answers.

You have access to one tool:

web_search(queries: list[str]) -- search the live web. Pass 1-3 short keyword queries. Use it when the answer depends on current, local, or frequently changing information (news, prices, scores, releases, schedules, weather, "latest"/"current"/"today"). Do NOT use it for stable knowledge you already have -- arithmetic, definitions, settled history, language questions, or code.

To call it, emit exactly this and nothing else:
<tool_call>
{"name": "web_search", "arguments": {"queries": ["query one", "query two"]}}
</tool_call>

Results come back as:
<tool_result>
[1] Title -- snippet text (source.com)
[2] Title -- snippet text (source.com)
</tool_result>

After results arrive, answer in prose and cite the snippets you used as [1], [2], etc. If the results don't actually answer the question, say so plainly rather than guessing.

The tool result goes back inside the next user turn (the model was trained with tool results folded into the user message, not a separate tool role). The search backend is yours to provide.

Use Qwen's default chat template. All results above were measured with it. Behaviour with enable_thinking=False was not tested for this release; an earlier Amethyst model lost much of its tool-calling reliability that way.

Usage β€” MLX

from mlx_lm import load, generate

model, tokenizer = load("VertexAGI/amethyst-2-5-mini")
SYSTEM = "..."  # the system prompt above

messages = [{"role": "system", "content": SYSTEM},
            {"role": "user", "content": "What's the weather like in Chicago right now?"}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=text, max_tokens=200))
# -> <tool_call>{"name": "web_search", "arguments": {"queries": ["Chicago weather now"]}}</tool_call>

Strip the empty think block. With the default template the model starts every reply with an empty <think>\n\n</think> block, as Qwen3 does. Remove it before parsing a tool call or showing the answer, for example re.sub(r"<think>.*?</think>", "", text, flags=re.S).strip(). The scores above were computed after stripping it.

Usage β€” llama.cpp

llama-server -m amethyst_2_5_mini_Q4_K_M.gguf --jinja -c 4096

Send the system prompt above as the system message. Both GGUFs were checked through llama-server on the 44-prompt set and scored the same as the MLX build (100% well-formed calls, 0% needless searches, 100% cited, 69% grounded).

Intended Use

A lightweight conversational assistant with tool-use awareness: experiments, research on small-scale distillation, and hobbyist deployments with a search backend wired up. It is not intended for high-stakes decisions.

Limitations

  • Trained on about 3,200 synthetic conversations. Behaviour outside the categories above can be inconsistent.
  • It reads whatever the search tool returns. Synthetic training results were invented by teacher models, so some facts in the data are fictional by design; the grounding it learned is "answer from the results", not "know the truth".
  • Grounding is imperfect. On the fresh 44-prompt set only 69% of post-search answers passed our automatic grounding check; a few answers add detail the results don't contain.
  • All evaluation sets are small and hand-made, and judged by language models. Treat the head-to-head as "no difference we can detect", not as proof of equality.
  • One tool (web_search) with a fixed schema; other tool schemas are untested.
  • Inherits the knowledge cutoff and general limitations of Qwen3-4B. English only.
Downloads last month
591
Safetensors
Model size
4B params
Tensor type
U32
Β·
BF16
Β·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for VertexAGI/amethyst-2-5-mini

Finetuned
Qwen/Qwen3-4B
Adapter
(1,273)
this model

Collection including VertexAGI/amethyst-2-5-mini