Instructions to use fyaronskiy/code_retriever-saved-checkpoints with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use fyaronskiy/code_retriever-saved-checkpoints with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("fyaronskiy/code_retriever-saved-checkpoints") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Add checkpoints README
Browse files
README.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- ru
|
| 5 |
+
- en
|
| 6 |
+
tags:
|
| 7 |
+
- sentence-transformers
|
| 8 |
+
- code-retrieval
|
| 9 |
+
- training-checkpoints
|
| 10 |
+
- rumodernbert
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# code_retriever training checkpoints
|
| 14 |
+
|
| 15 |
+
Full Hugging Face Trainer / SentenceTransformer checkpoints for the
|
| 16 |
+
[code_retriever](https://github.com/fedor28/code_retriever) project.
|
| 17 |
+
|
| 18 |
+
Each checkpoint directory contains everything needed to resume training:
|
| 19 |
+
`model.safetensors`, `optimizer.pt`, `scheduler.pt`, `rng_state.pth`,
|
| 20 |
+
`trainer_state.json`, `training_args.bin`, tokenizer files, and pooling config.
|
| 21 |
+
|
| 22 |
+
## Contents
|
| 23 |
+
|
| 24 |
+
| Run | Checkpoints | Notes |
|
| 25 |
+
|-----|-------------|-------|
|
| 26 |
+
| `RuModernBERT-base_bs64_lr_2e-05` | `checkpoint-12400`, `checkpoint-33600`, `checkpoint-46400`, `checkpoint-82600` | 1st epoch, batch size 64 |
|
| 27 |
+
| `RuModernBERT-base_bs128_lr_2e-05_2nd_epoch` | `checkpoint-27200`, `checkpoint-45400` | 2nd epoch, batch size 128 |
|
| 28 |
+
|
| 29 |
+
Base model: [`deepvk/RuModernBERT-base`](https://huggingface.co/deepvk/RuModernBERT-base)
|
| 30 |
+
|
| 31 |
+
## Download all checkpoints
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
huggingface-cli download fyaronskiy/code_retriever-saved-checkpoints \
|
| 35 |
+
--repo-type model \
|
| 36 |
+
--local-dir models/saved_checkpoints
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
## Download a single checkpoint
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
huggingface-cli download fyaronskiy/code_retriever-saved-checkpoints \
|
| 43 |
+
--repo-type model \
|
| 44 |
+
--include "RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600/*" \
|
| 45 |
+
--local-dir models/saved_checkpoints
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
## Resume training
|
| 49 |
+
|
| 50 |
+
1. Download the desired run folder or checkpoint.
|
| 51 |
+
2. In `train/train.py`, point `resume_checkpoint` to the checkpoint path and
|
| 52 |
+
set `model_dir` to the corresponding run directory under `models/`.
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
run_name = "RuModernBERT-base_bs64_lr_2e-05"
|
| 56 |
+
model_dir = f"../models/{run_name}"
|
| 57 |
+
resume_checkpoint = "../models/saved_checkpoints/RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600"
|
| 58 |
+
do_resume_train = True
|
| 59 |
+
auto_resume = False
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
3. Launch training as usual, e.g. `bash train/train_accelerate.sh`.
|
| 63 |
+
|
| 64 |
+
## Load for inference only
|
| 65 |
+
|
| 66 |
+
```python
|
| 67 |
+
from sentence_transformers import SentenceTransformer
|
| 68 |
+
|
| 69 |
+
model = SentenceTransformer(
|
| 70 |
+
"fyaronskiy/code_retriever-saved-checkpoints/RuModernBERT-base_bs64_lr_2e-05/checkpoint-82600"
|
| 71 |
+
)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
For production inference, prefer the published model:
|
| 75 |
+
[`fyaronskiy/code_retriever_ru_en`](https://huggingface.co/fyaronskiy/code_retriever_ru_en).
|