fp8 inference

#26

by Melody32768 - opened Aug 30, 2024

May I ask whether the input is quantified during inference

Please let me know ,.how to inference fp8

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment