unsloth

/

DeepSeek-V4-Pro

like 60

Follow

Unsloth AI 27.3k

GPT-5 mini:
Calculating parameter sizes

I need to provide a concise explanation of estimating sizes. The calculation involves 865GB times the bytes per parameter ratio. The exact size really depends on the original data type and specific parameters. If we assume the original was likely fp16 or fp32—common in many HF safetensors—the fp32 to fp8 ratio is 4 to 1, resulting in about 216GB if originally fp32. If it's fp16, then it's about 432GB. I'll ensure my response addresses whether fp8 will be approximately 285GB depending on assumptions.Clarifying parameter assumptions

I need to explain my assumptions about the effective bytes being around 2.97 times the size of fp8. It’s best to keep it concise, suggesting approximately 285GB if it’s a mixed original. I’ll also provide a range of 216–432GB based on the original data type and additional tensors. It could be helpful to check the converted size after conversion. I might recommend using the huggingface_hub to list file sizes and inspect safetensors metadata for accuracy.Approximately — but not guaranteed. 285 GB is a reasonable estimate if the original checkpoint’s average parameter storage is ~3× larger than fp8 (865 GB × 0.33 ≈ 285 GB). Actual result depends on the original dtype and non‑expert tensors.

Quick guide:

• If original params are mostly fp32 (4 bytes) → fp8 (~1 byte) ≈ 865 × (1/4) ≈ 216 GB.
• If original params are mostly fp16 (2 bytes) → fp8 ≈ 865 × (1/2) ≈ 432 GB.
• Real checkpoints mix tensors and have some non‑parameter overhead, so expect a converted size between ≈216–432 GB; 285 GB is a plausible mid‑point for mixed formats.

How to know before converting:

• Inspect HF filesizes/metadata (safetensors headers or torch dtype) to see original dtype.
• Or run convert on a small subset / dry-run to measure compression ratio.

If you want, tell me the HF file types (safetensors vs .pt) and I’ll give exact commands to inspect dtypes and predict converted size.

GPT-5 mini:
Estimating converted sizes

The user is asking for estimated converted sizes based on previous calculations. Since they mentioned "estimated," I think they might want estimates for fp8 and fp4 conversions for safetensor files, which usually are either f32 or f16.

I’ll base these estimates on our earlier note that HF has a total of 865GB across 64 shards. Providing the estimates: for mostly f32, fp8 is about 216GB and fp4 around 108GB; for mostly f16, fp8 is roughly 432GB and fp4 around 216GB. With mixed types, I’ll go with midpoint estimates. It’s wise to recommend that fp4 fits nicely in 768GB.Using your 865 GB original safetensors as the baseline, estimated converted sizes:

• If originals are mostly f32 (4 bytes/param):

  • fp8 ≈ 865 × (1/4) ≈ 216 GB
  • fp4 ≈ 865 × (1/8) ≈ 108 GB

• If originals are mostly f16 (2 bytes/param):

  • fp8 ≈ 865 × (1/2) ≈ 432 GB
  • fp4 ≈ 865 × (1/4) ≈ 216 GB

• Mixed/realistic case (reasonable mid estimate):

  • fp8 ≈ 285 GB
  • fp4 ≈ 142 GB

Notes:

• Add ~5–20% overhead for metadata/runtime buffers; final sizes may vary.
• With your 768 GB RAM, fp4 (≈108–216 GB) is safe; fp8 mixed-case (~285 GB) also fits but leaves less headroom.