tonythethompson/Phi-4-Mini-Instruct-GPU-INT4-ONNX

Phi-4 Mini Instruct — GPU INT4 (ONNX Runtime GenAI)

This repository repackages the gpu/gpu-int4-rtn-block-32 configuration from Microsoft's official ONNX release microsoft/Phi-4-mini-instruct-onnx. It is not a newly trained model. This is an unofficial community repackage; Microsoft is the original author and license holder.

Source

Field	Value
Upstream model	microsoft/Phi-4-mini-instruct-onnx
Upstream source revision	fc04c8f93df696602fd9f300a30d1bf2e3081347
Export tool/script	Microsoft ONNX Runtime GenAI model builder (upstream Phi ONNX bundle)
Quantization recipe	ONNX Runtime GenAI RTN INT4 (gpu-int4-rtn-block-32)

Files

All files are under gpu/gpu-int4-rtn-block-32/:

File	Size	Description
model.onnx	~280 KB	INT4 ONNX graph
model.onnx.data	~3 GB	External weights
genai_config.json	~2 KB	ONNX Runtime GenAI session config
config.json	~2 KB	Model config
configuration_phi3.py	~11 KB	Phi-3/4 config class
tokenizer.json + tokenizer_config.json	~15 MB	Tokenizer
vocab.json + merges.txt	~6 MB	Vocabulary / merges
added_tokens.json + special_tokens_map.json	<1 KB	Special tokens

Intended Use

A 4-bit, GPU-targeted build of Phi-4-Mini-Instruct for local text generation and translation. Targets NVIDIA CUDA and DirectML (Windows) execution paths where a capable GPU is available. A CPU variant is in tonythethompson/phi-4-mini-instruct-cpu-int4-onnx.

Runtime Notes

• Designed for ONNX Runtime GenAI compatible runtimes.
• GPU execution: CUDA (onnxruntime-genai-cuda) or DirectML (onnxruntime-genai-directml).
• Context length: 128K tokens (inherited from Phi-4-Mini-Instruct).
• Validate on the target execution provider before production use.

Precision and Packaging

Export tooling, precision, and quantization are recorded in the Source table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use.

Limitations

• English-dominant training; quality on lower-resource languages may degrade.
• INT4 quantization introduces approximation error relative to the base FP16/FP32 model.
• No repository-specific translation quality evaluation is documented here.
• Validate on your target language pair before production use.

License

MIT — inherited from microsoft/Phi-4-mini-instruct-onnx. This packaging repo adds no new license terms.