Hugging Face's logo

amd-shark
/
sdxl-quant-int8

Transformers
Model card Files
xet
 Community
2

Updates to minimal quantization script.

#1
by nickfraser - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+648
-27
Fix (minimal): Move to CPU before export13b90945
 Feat (script): Added calibration size argument.8324b6e6
 Fix (script): print output directory.e6b40f3f
 Fix (script): Quantize output of QuantLora layers.77d84b5e
 Reorg (minimal): moved minimal script to sub-directory.de290379
 Fix (script/env): added pandas requirement.e01fcf24
 Feat (script/env): Added conda environment YAMLsbffa16d9
 Fix (script): Updated dependencies.62e0e7b2
 Feat (script): Added option to validate on MLPerf validation set & to load a pre-quantized checkpoint.34b0078d
 Feat (script): Added initial reproduction README.bdf45c37
 Feat (script): Added pre-generated `captions.tsv`b2f369f7
nickfraser
 AMD SHARK org
•
edited Jun 28, 2024

The following updates to the minimal quantization script:

  • Reorg files (move to subdirectory)
  • Added optional calibration set size (small size is useful to detect algorithmic changes)
  • Added validation with optional size (small size is useful to detect algorithmic changes)
  • Fix: added quantization to the output of QuantLoraLinear layers (when --quantize-sdp enabled)
  • Updated dependencies, added conda environment files
  • Added pre-generated captions (from Nvidia's previous submission)
  • Added basic README for reproducing results
  • Checked that the right quantized structure is generated for Int8+FP8 mode
  • Checked that the right quantized structure is generated for Int8 mode
  • Checked that the compliant accuracy is achieved for Int8+FP8
  • Checked that the compliant accuracy is achieved for Int8
  • Checked that dataset setup instructions (from MLPerf) work & produce the correct directories / files
GiusFra changed pull request status to open
GiusFra changed pull request status to merged

Sign up or log in to comment