Renesas/Llama-3.2-3B-Instruct-ONNX

Llama-3.2-3B-Instruct (ONNX FP32) – Renesas X5H

Introduction

This repository hosts the Llama‑3.2‑3B‑Instruct model exported to ONNX (FP32), serving as a reference representation for analysis and optimization targeting Renesas Gen5 platforms. The ONNX model is used as an intermediate format for functional validation and performance estimation. The format, precision, and representation used for final hardware execution are subject to the chosen deployment flow.

Supported & Targeted Precisions

Precision	Status	Notes
FP32 (ONNX)	✅ Provided	Reference / functional baseline
W4A16	🔧 Estimated	Weights INT4, Activations INT16 (accuracy‑oriented) auto cast by the toolchain

Note
FP16 & W4A16 numbers below are performance estimates, derived from internal Gen5 measurements and scaling trends. Final results depend on compiler, tiling, memory layout, and runtime configuration.

---

Performance (Estimated – PPA Estimator)

Important:
The model variants listed below have not been executed on Renesas X5H hardware.
All performance numbers are estimates generated using the Renesas PPA Estimator tool, based on model characteristics, target precision, and hardware configuration assumptions. For hardware executed performances check https://huggingface.co/Renesas/Llama-3.2-3B-Instruct-GGUF

Token Generation Throughput (Estimated)

Model	Precision	Device	Tokens / sec
Llama‑3.2‑3B‑Instruct	FP16	X5H – 1xNPX - 50GB/s	7.30 (PPA estimate)
Llama‑3.2‑3B‑Instruct	FP16	X5H – 1xNPX - 80GB/s	11.33 (PPA estimate)
Llama‑3.2‑3B‑Instruct	W4A16	X5H – 1xNPX - 50GB/s	26.04 (PPA estimate)
Llama‑3.2‑3B‑Instruct	W4A16	X5H – 1xNPX - 80GB/s	38.10 (PPA estimate)

---

Estimation Methodology

• Performance values are derived from the Renesas PPA Estimator tool
• Estimates are based on:
  • Model size and operator mix
  • Target precision (FP32 / W4A16)
  • Single‑cluster NPX configuration
  • Assumed memory bandwidth and compute utilization
• No measurements were taken on physical silicon or emulator for these results

---

Disclaimer

These results are indicative only and do not represent measured performance on real hardware.
Actual performance may vary depending on compiler version, graph partitioning, tiling strategy, runtime configuration, and memory bandwidth constraints.
Hardware‑validated benchmarks may be published separately.