Amethyst 2.5 Micro

Amethyst 2.5 Micro is the 1.7B-parameter little sibling of Amethyst 2.5 Mini: a small chat model fine-tuned from Qwen3-1.7B on the same data, aimed at three behaviours:

  1. Know when to search. It calls a web_search tool for things that change (news, scores, prices, "latest") and answers stable questions directly.
  2. Answer from the results, with citations. After a search it answers briefly and cites the snippets it used as [1], [2].
  3. Say so when the results don't answer. If the search failed, returned nothing, or returned on-topic pages that never state the answer, it is trained to say that plainly rather than guess.

What actually worked, and what didn't. The clear win over the Qwen3-1.7B base is format and brevity: the tuned model emits a correct tool call on every search prompt we tried, never searches on plain questions, cites its sources, and answers plain questions in fewer words. The base 1.7B mostly fails to produce a usable tool call with this prompt. The quality of answers after a search is about a wash against the base (8 wins vs 7 losses, no-think arm), and Micro loses head-to-head to Mini (4B), 4 wins to 9. It is a small, cheap, well-behaved tool-caller, not a smarter model than its base. Details below.

Part of the Amethyst family of chat-optimized models, from a small, independent, nonprofit-spirited AI lab that works in the open.

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Format File / folder Precision Size Works with
MLX model.safetensors (repo root) mixed: tuned layers 8-bit, the rest 4-bit 1.67 GB mlx-lm / MLX on Apple Silicon
GGUF amethyst_2_5_micro_Q4_K_M.gguf Q4_K_M (4-bit) 1.11 GB llama.cpp, Ollama, LM Studio
GGUF amethyst_2_5_micro_Q8_0.gguf Q8_0 (8-bit) 1.83 GB llama.cpp, Ollama, LM Studio

Why the MLX build is "mixed": a plain 4-bit merge of a LoRA re-quantizes the fine-tune away. Here all 196 LoRA-adapted linear layers (7 per layer x 28 layers) are merged at fp16 and re-quantized at 8-bit. On 60 validation conversations the merged model's full-sequence loss was 3.337 against 3.331 for the unmerged adapter (that figure includes the system prompt, so it is not comparable to the training-time validation loss), and its held-out scores match the adapter's.

Model Details

Developed by Independent research project
Base model Qwen/Qwen3-1.7B
Fine-tuning base checkpoint mlx-community/Qwen3-1.7B-4bit
Architecture Qwen3, 1.7B parameters (dense, decoder-only transformer, tied embeddings)
Fine-tuning method LoRA (rank 16, scale 20.0, all 28 layers), loss on assistant turns only
Fine-tuning framework MLX / mlx-lm, on Apple Silicon
Language English
License Apache 2.0 (same as the base model)

Training Data

The same 3,205 synthetic conversations (4,683 training samples after splitting each conversation into one sample per assistant turn) used for Amethyst 2.5 Mini, generated by distillation from large teacher models through NVIDIA NIM and OpenRouter: GLM 5.3 (and GLM 5.3 Flash), Kimi K3, poolside Laguna S 2.1 and MiniMax M3.1 Flash. Every search result in the data is invented by a teacher; no real web pages were scraped.

Category Conversations What it teaches
General chat 817 Short, direct answers to ordinary questions
Plain, short answers (tool available, no search) 309 + 42 two-turn Stable questions are answered directly in a few words
Terse-format instructions 500 "Exactly 3 bullets", "under 20 words", "one sentence", translation, rewrite
Current-fact search, one query 502 One search, brief cited answer
Two sources that disagree 127 Says the sources differ and cites both
Weak first results, then a better query 97 + 21 Refines the search once, then answers
Search failed (timeout, 503, no results) 192 Says it couldn't get the information; no guessing
Results are on-topic but don't contain the answer 448 Says the results don't give it, cites what they do say
Summaries, multi-query, follow-ups, no-search 150 Cited summaries, query decomposition, elliptical follow-ups

Search-result rows passed an automatic grounding check: the final answer may only cite results that exist and may only use numbers and capitalised names that appear in the last result or in the question. The set caps repeats per prompt (an earlier 20,000-row set held only ~1,000 unique prompts).

Training Procedure

  • Method: supervised fine-tuning via LoRA (rank 16, scale 20.0, dropout 0.0, all 28 layers), fresh from the base, loss on assistant turns only (mask_prompt)
  • Optimizer: Adam, learning rate 6e-5 (constant), batch size 4, sequence length 2,048, gradient checkpointing
  • Schedule: 2 epochs, 2,341 iterations, one uninterrupted run. Validation loss (assistant tokens only) fell from 4.59 to about 1.01 at iteration ~1,400 and ended at 1.155, so it drifted up slightly late in training; the final weights, not the best checkpoint, are what is released.

Evaluation

Greedy decoding on hand-made held-out prompts (tool decisions, answers after a search result, plain questions), three sets of 44 (fresh, written after the training mix was frozen and excluded from it by keyword), 64 and 36 prompts. The 64- and 36-prompt sets overlap training topics more than the 44-prompt set, so read the 44-prompt column as the cleanest. All sets are small. Base columns are the Qwen3-1.7B base with enable_thinking=False / the default template.

Metric Fresh 44: Micro base (no-think / default) 64: Micro base 36: Micro base
Well-formed web_search calls 100% 0% / 13% 100% 0% / 25% 100% 0% / 14%
Searches on plain questions (lower is better) 0% 0% / 6% 0% 0% / 4% 0% 0% / 0%
Cites after a search 100% 46% / 54% 94% 19% / 50% 100% 25% / 75%
Answers grounded in the result 69% 54% / 46% 100% 100% / 88% 100% 88% / 88%
Plain answer length (words) 16 21 / 22 33 56 / 63 37 50 / 42

The base's tool-call numbers are low mostly because the 1.7B base does not follow the <tool_call> format from the system prompt, so part of the gap is plain instruction-following, not tool judgment. The 69% grounded figure on the fresh set is Micro's weakest number (it is the same as Mini's on that set).

Judged against the 1.7B base (strict four-verdict panel: two judge models, both answer orders, a win only when all four verdicts agree; failed judge calls are counted as failed, never as ties):

Set vs base, no-think vs base, default template
Fresh 44 13 wins, 1 loss, 10 ties, 5 failed 11 wins, 1 loss, 10 ties, 7 failed
64 11 wins, 9 losses, 12 ties, 8 failed 13 wins, 9 losses, 15 ties, 3 failed
36 6 wins, 4 losses, 11 ties, 1 failed 7 wins, 3 losses, 12 ties, 0 failed
Total (91 pairs) 30 wins, 14 losses, 33 ties, 14 failed 31 wins, 13 losses, 37 ties, 10 failed

The wins come mostly from plain answers (tighter and cleaner). On answers after a search the result is close to even: against the no-think base, Micro wins 8 and loses 7 across the three sets.

Head-to-head against Amethyst 2.5 Mini (4B) on the fresh 44 set: Micro 4 wins, Mini 9, 14 ties, 2 failed. Mini is better, as expected for a model more than twice the size; the tool-call and grounding numbers are identical on this set, so the gap is in the answers themselves.

The judge panel changed during this work (one originally chosen judge was withdrawn by its host and its replacement was picked by an audit that planted known defects in real answers), so these verdicts are not directly comparable to earlier Amethyst releases' numbers.

Tool-calling format

The model expects this system prompt and emits tool calls in exactly this format:

You are Amethyst, a helpful, direct conversational assistant. Answer the actual question asked. Match your length to the question -- a one-line question gets a one-line answer, a substantive question gets real depth. Use plain language, skip filler openers, and don't restate the question back before answering. Format with markdown only when structure genuinely helps (lists, tables, code); prose questions get prose answers.

You have access to one tool:

web_search(queries: list[str]) -- search the live web. Pass 1-3 short keyword queries. Use it when the answer depends on current, local, or frequently changing information (news, prices, scores, releases, schedules, weather, "latest"/"current"/"today"). Do NOT use it for stable knowledge you already have -- arithmetic, definitions, settled history, language questions, or code.

To call it, emit exactly this and nothing else:
<tool_call>
{"name": "web_search", "arguments": {"queries": ["query one", "query two"]}}
</tool_call>

Results come back as:
<tool_result>
[1] Title -- snippet text (source.com)
[2] Title -- snippet text (source.com)
</tool_result>

After results arrive, answer in prose and cite the snippets you used as [1], [2], etc. If the results don't actually answer the question, say so plainly rather than guessing.

The tool result goes back inside the next user turn (the model was trained with tool results folded into the user message, not a separate tool role). The search backend is yours to provide.

Use Qwen's default chat template. All results above were measured with it; enable_thinking=False was not tested for the tuned model.

Usage β€” MLX

from mlx_lm import load, generate

model, tokenizer = load("VertexAGI/amethyst-2-5-micro")
SYSTEM = "..."  # the system prompt above

messages = [{"role": "system", "content": SYSTEM},
            {"role": "user", "content": "What's the weather like in Chicago right now?"}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=text, max_tokens=200))

Strip the empty think block. With the default template the model starts every reply with an empty <think>\n\n</think> block, as Qwen3 does. Remove it before parsing a tool call or showing the answer, for example re.sub(r"<think>.*?</think>", "", text, flags=re.S).strip(). The scores above were computed after stripping it.

Usage β€” llama.cpp

llama-server -m amethyst_2_5_micro_Q4_K_M.gguf --jinja -c 4096

Send the system prompt above as the system message. Both GGUFs were checked through llama-server on the fresh 44-prompt set and scored the same as the MLX build (100% well-formed calls, 0% needless searches, 100% cited, 69% grounded; plain answers 14-16 words).

Intended Use

A very small conversational assistant with tool-use awareness: experiments, research on small-scale distillation, and hobbyist or on-device deployments with a search backend wired up. It is not intended for high-stakes decisions.

Limitations

  • Trained on about 3,200 synthetic conversations. Behaviour outside the categories above can be inconsistent.
  • At 1.7B it reads search results no better than its base. Judged answer quality after a search was about even with the base, and clearly behind Mini (4B). Use Mini if you can afford 4B.
  • It reads whatever the search tool returns. The synthetic training results were invented by teacher models, so some facts in the data are fictional by design; the grounding it learned is "answer from the results", not "know the truth".
  • Grounding is imperfect. On the fresh 44-prompt set only 69% of post-search answers passed our automatic grounding check.
  • All evaluation sets are small and hand-made, and judged by language models; 14 of 91 judged pairs against the no-think base failed to get a full verdict. Treat results as directional.
  • One tool (web_search) with a fixed schema; other tool schemas are untested.
  • Inherits the knowledge cutoff and general limitations of Qwen3-1.7B. English only.
Downloads last month
446
Safetensors
Model size
2B params
Tensor type
U32
Β·
BF16
Β·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for VertexAGI/amethyst-2-5-micro

Finetuned
Qwen/Qwen3-1.7B
Adapter
(954)
this model

Collection including VertexAGI/amethyst-2-5-micro