Mining-4-Qwen v2 (Qwen3 4B, learning fine-tune)

A small learning fine-tune of Qwen3 4B Instruct 2507 that answers questions about three generic mining and mineral-processing routes and names a public source for each claim.

This is a personal learning exercise, published to show what a small fine-tune does and does not learn. It is not engineering, design, safety, operating or investment advice, and it does not describe any real plant. It confidently invents answers on topics outside its training data (see Evaluation), so check every claim against the named source.

How it was trained

  • Base model: Qwen/Qwen3-4B-Instruct-2507 (Apache 2.0), trained from unsloth/Qwen3-4B-Instruct-2507
  • Method: QLoRA (4-bit) with Unsloth on a free Google Colab T4 GPU
  • Settings: LoRA rank 32, alpha 32, learning rate 2e-4, 3 epochs, max sequence length 2048, batch 2, gradient accumulation 4, loss on responses only
  • Final loss: 0.07 training, 0.06 validation (validation = new wordings of trained facts, so it measures memorisation)
  • Export: GGUF, Q4_K_M, for Ollama and llama.cpp

Training data

2,880 question and answer pairs written from public sources, covering three generic routes: copper ore to cathode (concentrator, flash smelting, converting, fire refining, electrorefining), copper heap leach with solvent extraction and electrowinning, and gold carbon-in-leach. Each value is asked six different ways with the same answer. About 12 % of examples decline requests for named plants, private data, forecasts, setpoints or costs.

Sources are public documents: US EPA, USGS, IFC/World Bank, EU JRC, UK HSE, NIOSH, IARC, CCOHS, IMO, ICMM/UNEP/PRI, ICSG, ICA, open-access papers, a US patent and a published textbook. Copyrighted sources were paraphrased and cited by page, not copied. No employer, customer, private, licensed or site-specific data was used. The dataset is not published.

Every training example uses this system prompt, and the model works best with it:

You are a mining and mineral-processing tutor. Answer from public, generic process knowledge only. Say which public source supports each claim. If no public source gives a value, say so instead of guessing. Never give values for a named or real plant.

Evaluation

Answers were scored 0 to 4: correct 0 to 2, fitting real source 0 to 1, nothing invented 0 to 1. Temperature 0. Scored by one AI reviewer against published reference answers; treat the figures as indicative.

Test 1: 30 questions on the trained routes (14 textbook values, 12 explanations, 4 that should be declined). The questions are new wordings of facts in the training data, so this mainly tests recall.

Model Average (0 to 4) Values right (of 14)
Claude Sonnet (for comparison) 3.70 8
Claude Opus (for comparison) 3.67 7
Mining-4-Qwen v2 3.07 12
Qwen3 4B Instruct (base) 0.87 1
Mining-4-Qwen v1 (934 examples, 2 epochs) 0.67 2

Test 2: 8 questions outside the training data (nickel, zinc, lithium, iron ore, uranium, coal, lead-zinc flotation, cement kilns).

Model Average (0 to 4) Answers with nothing invented (of 8)
Claude Sonnet / Opus 4.00 8
Mining-4-Qwen v2 0.88 0
Qwen3 4B Instruct (base) 0.88 0

Limitations

  • Outside the three routes it invents. It answers every question in its trained pattern ("A typical published value is ...") and cites a copper textbook even for iron ore, lithium or nickel. It never says it does not know.
  • Declines are unreliable. It declined most named-plant and private-data requests, but sometimes adds an unrelated "typical value", and it gave a figure for one plant-output question it should have declined.
  • Explanations are short and can drift after the first sentence; two of 12 cited a source that does not exist.
  • Stray tags. Answers may start with an empty <think></think> block or a stray </tool_call> tag. Ignore them.
  • Values it gives are published typical or textbook values, never those of a real operation.

Use

ollama create mining-4-qwen-v2 -f Modelfile
ollama run mining-4-qwen-v2

The Modelfile already sets the tutor system prompt above. If you load the GGUF another way (llama.cpp, LM Studio), set that system prompt yourself; without it answers are much worse.

Author

Paul Mignone, as a personal learning project.

Downloads last month
47
GGUF
Model size
4B params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for pmignone/mining-4-qwen-v2

Adapter
(5,865)
this model