Phi-4-mini-instruct — ONNX (INT4)

INT4-quantized ONNX export of Phi-4-mini-instruct, a 3.8B-parameter lightweight language model from Microsoft. Optimized for CPU inference with INT4 RTN block-32 quantization in onnxruntime-genai format.

Mirrored for use with inference4j, an inference-only AI library for Java.

Original Source

• Repository: Microsoft
• License: mit

Usage with inference4j

try (TextGenerator gen = TextGenerator.builder()
        .model(ModelSources.phi4Mini())
        .build()) {
    GenerationResult result = gen.generate("What is Java in one sentence?");
    System.out.println(result.text());
}

Model Details

Property	Value
Architecture	Phi-4-mini (3.8B parameters, 32 layers, 3072 hidden)
Task	Text generation / chat
Context length	128k tokens
Quantization	INT4 RTN block-32 acc-level-4
Original framework	PyTorch (transformers)

License

This model is licensed under the MIT License. Original model and ONNX conversion by Microsoft.