Mem-0-m1mix-RMBench / checkpoint /README_REASSEMBLE.md
qiuly's picture
Add files using upload-large-folder tool
1dd1ae9 verified
|
Raw
History Blame Contribute Delete
1.9 kB

Reassembling the checkpoint

The full training checkpoint m1_mix_final_step50000.pt (15.3 GB) is uploaded as 9 byte-split parts. The upload path (an egress gateway) both capped single files at ~9.7 GB and throttled cumulative bytes per connection window, so the file is split into small parts that each fit a window: the first two parts are 4 GiB (uploaded before the throttle was understood) and the remaining seven are ≤1 GiB. Concatenating all parts in sorted order reproduces the checkpoint bit-for-bit.

part size
m1_mix_final_step50000.pt.part00 4.0 GiB
m1_mix_final_step50000.pt.part01 4.0 GiB
m1_mix_final_step50000.pt.part02 1.0 GiB
m1_mix_final_step50000.pt.part03 1.0 GiB
m1_mix_final_step50000.pt.part04 1.0 GiB
m1_mix_final_step50000.pt.part05 1.0 GiB
m1_mix_final_step50000.pt.part06 1.0 GiB
m1_mix_final_step50000.pt.part07 1.0 GiB
m1_mix_final_step50000.pt.part08 ~0.25 GiB

1. Reassemble

cat m1_mix_final_step50000.pt.part?? > m1_mix_final_step50000.pt

(part?? expands to part00 part01 … part08 in sorted order — the command is unchanged regardless of how many parts there are.)

2. Verify integrity

sha256sum -c m1_mix_final_step50000.pt.sha256
# expected:  m1_mix_final_step50000.pt: OK

The expected SHA-256 of the full file is:

df149a0d27c85ad76217552c4005bee0094f6068fdb92bd9f6b256190f5aa09a

3. Load

Load as usual — the deploy loader reads payload["model_state_dict"] and calls load_state_dict(..., strict=False). See the top-level README.md for the full inference / evaluation instructions and the optional inference-only slimming snippet.