TevunahAi
/

NextCoder-14B-FP8

Text Generation

text-generation-inference

compressed-tensors

Model card Files Files and versions

rockylynnstein commited on Dec 8, 2025

Commit

ed96669

·

verified ·

1 Parent(s): 7f6d813

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -48,7 +48,7 @@ import torch
 # Load model with FP8 quantization
 model = AutoModelForCausalLM.from_pretrained(
     "TevunahAi/NextCoder-14B-FP8",
-    torch_dtype=torch.float8_e4m3fn,  # FP8 dtype
     device_map="auto",
     low_cpu_mem_usage=True,
 )

 # Load model with FP8 quantization
 model = AutoModelForCausalLM.from_pretrained(
     "TevunahAi/NextCoder-14B-FP8",
+    torch_dtype=torch.bfloat16,
     device_map="auto",
     low_cpu_mem_usage=True,
 )