RedHatAI
/

granite-3.1-8b-instruct-FP8-dynamic

compressed-tensors MLA support requires fp8 activations and weights in group 'group_0',

#1 opened over 1 year ago by