Industrial-Instruction Qwen3-4B (Qwen-generated data)

Qwen3-4B-Instruct-2507 fully fine-tuned on the panasonic_qa_v1 config of Industrial-Instruction — the split generated with the open-weight Qwen3-30B-A3B-Instruct model.

A companion model trained on the Claude-Opus-4.6-generated split is available at Parssky/industrial-instruction-qwen4b-claude.

Paper

Industrial-Instruction: An End-to-End Framework for Building Instruction-Tuning and Benchmark Datasets from Industrial Technical Reports

Base Model

Qwen3-4B-Instruct-2507

Results

Panasonic benchmark (panasonic_qa_v1 test split, 1,000 items):

Configuration F1 Jaccard Set-Match Acc.
Base, with RAG 46.57% 41.57% 28.50%
Fine-tuned, with RAG 63.48% 57.95% 42.00%
Base, no RAG 46.63% 41.61% 28.50%
Fine-tuned, no RAG 63.14% 57.60% 41.70%

General knowledge (MMLU, 57 subjects, 14,042 questions): 72.13% base → 70.87% fine-tuned (−1.26).

FailureSensorIQ: AccOrgIBM 34% → 27%, F1-Macro 40% → 43%, F1-Micro 66% → 74%.

Limitations

  • Accuracy on perturbed (rephrased) FailureSensorIQ questions is 0%, both before and after fine-tuning. The training data contains no paraphrased question variants, so this model should not be relied on where question phrasing varies.
  • Fine-tuning costs 1.26 points of MMLU accuracy, concentrated in Humanities (moral-reasoning subjects drop ~10 points).
  • Source documentation comes from a single manufacturer; terminology may not transfer.

Intended Use

Research and benchmarking for industrial retrieval-augmented generation (RAG), evidence integration, and technical-domain QA.

Notes

  • Inference-ready weights and tokenizer files only; training checkpoints and optimizer states are intentionally excluded.

Source Code

GitHub repository: https://github.com/parssky/industrial-instruction

Downloads last month
-
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Parssky/industrial-instruction-qwen4b

Finetuned
(1946)
this model

Dataset used to train Parssky/industrial-instruction-qwen4b

Collection including Parssky/industrial-instruction-qwen4b

Paper for Parssky/industrial-instruction-qwen4b