baseten
/

7b-fp8-dynamic

compressed-tensors

Model card Files Files and versions

7b-fp8-dynamic / recipe.yaml

htrivedi99's picture

Upload folder using huggingface_hub

00d7a93 verified 8 months ago

history blame contribute delete

194 Bytes

	default_stage:
	default_modifiers:
	QuantizationModifier:
	targets: [Linear]
	ignore: ['re:.lm_head', 're:vision_tower.', 're:multi_modal_projector.*']
	scheme: FP8_DYNAMIC