Instructions to use onlplab/alephbert-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use onlplab/alephbert-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="onlplab/alephbert-base")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("onlplab/alephbert-base") model = AutoModelForMaskedLM.from_pretrained("onlplab/alephbert-base") - Inference
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -45,7 +45,7 @@ To optimize training time we split the data into 4 sections based on max number
|
|
| 45 |
3. 64 <= num tokens < 128 (10M sentences)
|
| 46 |
4. 128 <= num tokens < 512 (70M sentences)
|
| 47 |
|
| 48 |
-
Each section was first trained for 5 epochs with an initial learning rate set to 1e-4. Then each
|
| 49 |
|
| 50 |
Total training time was 8 days.
|
| 51 |
|
|
|
|
| 45 |
3. 64 <= num tokens < 128 (10M sentences)
|
| 46 |
4. 128 <= num tokens < 512 (70M sentences)
|
| 47 |
|
| 48 |
+
Each section was first trained for 5 epochs with an initial learning rate set to 1e-4. Then each section was trained for another 5 epochs with an initial learning rate set to 1e-5, for a total of 10 epochs.
|
| 49 |
|
| 50 |
Total training time was 8 days.
|
| 51 |
|