README: reframe bf16 weights — reason is flash_attn half-precision + runtime-bf16 (not download size)

README.md

@@ -32,11 +32,12 @@ itself, so no runtime patching or post-load hooks are needed.
 
 ## Behavior
 
-- **Loads bf16 by default.** Weights are stored bf16 (half the download: 274 MB vs 547 MB) and
-  `config.json` declares `torch_dtype: bfloat16`. The upstream custom `from_pretrained` silently
-  dropped `torch_dtype` and always loaded fp32; the copy in this repo honors it, so the model loads
-  bf16 natively, like any normal HF model. Pass `torch_dtype=torch.float32` to load fp32 (note: the
-  stored weights are bf16-precision, so this only widens the dtype, not the precision).
+- **Loads bf16 by default.** `flash_attn` requires half precision and the model runs bf16 in any
+  real serving setup, so the weights are stored bf16 and `config.json` declares `torch_dtype:
+  bfloat16`. The upstream custom `from_pretrained` silently dropped `torch_dtype` and always loaded
+  fp32; the copy in this repo honors it, so the model loads bf16 natively, like any normal HF
+  model. Pass `torch_dtype=torch.float32` to load fp32 (note: the stored weights are bf16-precision,
+  so this only widens the dtype, not the precision).
 - **CUDA + `flash_attn`** → flash-varlen path (fast, low VRAM). Attention tensors are cast to bf16
   internally as a safety net; with the default bf16 load this is a no-op.
 - **CPU, or no `flash_attn`** → the original eager attention algorithm runs unchanged. Because the
@@ -78,7 +79,11 @@ also scales to far larger batches (bs=256 fits in ~7.6 GB) where eager O(seq²)
 
 ## What changed vs the source repo
 
-1. **Weights**: fp32 → bf16 (lossy cast, parity-neutral). Halves the download.
+1. **Weights**: fp32 → bf16. `flash_attn` only accepts half precision and the model runs bf16 in
+   any real serving configuration, so the weights are stored bf16 and (via the load fix below)
+   arrive bf16 — which is simply how this model is used, and removes the need for a post-load dtype
+   cast. Parity-neutral (cosine 0.9986 vs the fp32 original); the smaller download is incidental,
+   not the reason.
 2. **`from_pretrained` dtype fix**: the upstream custom `from_pretrained` instantiated the model
    fp32 and `load_state_dict`-ed the checkpoint into fp32 params, **ignoring `torch_dtype`**. The
    copy here adds the standard transformers dtype resolution (explicit arg → `config.torch_dtype` →