How to use SumanthRH/zephr_flex_packing_bsz_4 with Transformers:
# Load model directly from transformers import AutoTokenizer, MistralForCausalLMFlexAttn tokenizer = AutoTokenizer.from_pretrained("SumanthRH/zephr_flex_packing_bsz_4") model = MistralForCausalLMFlexAttn.from_pretrained("SumanthRH/zephr_flex_packing_bsz_4", device_map="auto")
fxwang-anyscale