fyaronskiy commited on
Commit
d4d5967
·
verified ·
1 Parent(s): 9b95901

Add checkpoints README

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - ru
5
+ - en
6
+ tags:
7
+ - sentence-transformers
8
+ - code-retrieval
9
+ - training-checkpoints
10
+ - rumodernbert
11
+ ---
12
+
13
+ # code_retriever training checkpoints
14
+
15
+ Full Hugging Face Trainer / SentenceTransformer checkpoints for the
16
+ [code_retriever](https://github.com/fedor28/code_retriever) project.
17
+
18
+ Each checkpoint directory contains everything needed to resume training:
19
+ `model.safetensors`, `optimizer.pt`, `scheduler.pt`, `rng_state.pth`,
20
+ `trainer_state.json`, `training_args.bin`, tokenizer files, and pooling config.
21
+
22
+ ## Contents
23
+
24
+ | Run | Checkpoints | Notes |
25
+ |-----|-------------|-------|
26
+ | `RuModernBERT-base_bs64_lr_2e-05` | `checkpoint-12400`, `checkpoint-33600`, `checkpoint-46400`, `checkpoint-82600` | 1st epoch, batch size 64 |
27
+ | `RuModernBERT-base_bs128_lr_2e-05_2nd_epoch` | `checkpoint-27200`, `checkpoint-45400` | 2nd epoch, batch size 128 |
28
+
29
+ Base model: [`deepvk/RuModernBERT-base`](https://huggingface.co/deepvk/RuModernBERT-base)
30
+
31
+ ## Download all checkpoints
32
+
33
+ ```bash
34
+ huggingface-cli download fyaronskiy/code_retriever-saved-checkpoints \
35
+ --repo-type model \
36
+ --local-dir models/saved_checkpoints
37
+ ```
38
+
39
+ ## Download a single checkpoint
40
+
41
+ ```bash
42
+ huggingface-cli download fyaronskiy/code_retriever-saved-checkpoints \
43
+ --repo-type model \
44
+ --include "RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600/*" \
45
+ --local-dir models/saved_checkpoints
46
+ ```
47
+
48
+ ## Resume training
49
+
50
+ 1. Download the desired run folder or checkpoint.
51
+ 2. In `train/train.py`, point `resume_checkpoint` to the checkpoint path and
52
+ set `model_dir` to the corresponding run directory under `models/`.
53
+
54
+ ```python
55
+ run_name = "RuModernBERT-base_bs64_lr_2e-05"
56
+ model_dir = f"../models/{run_name}"
57
+ resume_checkpoint = "../models/saved_checkpoints/RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600"
58
+ do_resume_train = True
59
+ auto_resume = False
60
+ ```
61
+
62
+ 3. Launch training as usual, e.g. `bash train/train_accelerate.sh`.
63
+
64
+ ## Load for inference only
65
+
66
+ ```python
67
+ from sentence_transformers import SentenceTransformer
68
+
69
+ model = SentenceTransformer(
70
+ "fyaronskiy/code_retriever-saved-checkpoints/RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600"
71
+ )
72
+ ```
73
+
74
+ For production inference, prefer the published model:
75
+ [`fyaronskiy/code_retriever_ru_en`](https://huggingface.co/fyaronskiy/code_retriever_ru_en).