Text Generation
PyTorch
Polish
English
polish
causal-lm
from-scratch
Maggio33 commited on
Commit
db4857f
·
verified ·
1 Parent(s): 64bc745

Card: link to intermediate checkpoints repo; second external stop at 19330 resumed from 19250

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -22,7 +22,9 @@ replayed from the 1.06 data), at the same constant learning rate and global batc
22
 
23
  **Status: training in progress.** The planned end is step 22327. This card reports intermediate training-time
24
  evaluations up to step 18590 and will be replaced by the final card when training ends. No 1.07 weights are
25
- published in this repository yet.
 
 
26
 
27
  - Previous version: [SlayerLab/Slayer1B-1.06](https://huggingface.co/SlayerLab/Slayer1B-1.06) (step 13848,
28
  14 520 680 448 tokens, one full pass over [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b))
@@ -44,7 +46,8 @@ published in this repository yet.
44
  - Planned length: steps 13848 → 22327 (8 479 steps, one pass over the new data; each window is used once).
45
  - Training was interrupted on 4 November at step 17130 by an external stop of the process and resumed on 5 November
46
  from the saved checkpoint at step 17000. After the resume the loss at step 17010 was identical to the first run
47
- (2.3177), so the resumed run reproduces the original one.
 
48
 
49
  ## Training data of 1.07
50
 
 
22
 
23
  **Status: training in progress.** The planned end is step 22327. This card reports intermediate training-time
24
  evaluations up to step 18590 and will be replaced by the final card when training ends. No 1.07 weights are
25
+ published in this repository yet. Intermediate training checkpoints (raw PyTorch training state `ckpt_N.pt`, not a
26
+ ready-to-use model) are published file by file in
27
+ [SlayerLab/Slayer1B-1.07-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.07-checkpoints), with sha256 sums.
28
 
29
  - Previous version: [SlayerLab/Slayer1B-1.06](https://huggingface.co/SlayerLab/Slayer1B-1.06) (step 13848,
30
  14 520 680 448 tokens, one full pass over [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b))
 
46
  - Planned length: steps 13848 → 22327 (8 479 steps, one pass over the new data; each window is used once).
47
  - Training was interrupted on 4 November at step 17130 by an external stop of the process and resumed on 5 November
48
  from the saved checkpoint at step 17000. After the resume the loss at step 17010 was identical to the first run
49
+ (2.3177), so the resumed run reproduces the original one. A second external stop on 5 November at step 19330 was
50
+ resumed within two minutes from the checkpoint at step 19250; the loss at step 19260 was again identical (2.3105).
51
 
52
  ## Training data of 1.07
53