Hugging Face's logo

Wikidepia
/
IndoT5-small

Transformers
PyTorch
Indonesian
t5
text2text-generation
text-generation-inference
Model card Files
xet
 Community
1
Wikidepia commited on
Commit
e26406f
·
1 Parent(s): cd9fc4a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -0
README.md CHANGED
@@ -6,10 +6,13 @@ datasets:
6
  ---
7
  # Indonesian T5 Small
8
 
 
9
  T5 (Text-to-Text Transfer Transformer) model pretrained on Indonesian mC4 with [extra filtering](https://github.com/Wikidepia/indonesian_datasets/tree/master/dump/mc4). This model is pre-trained only and needs to be fine-tuned to be used for specific tasks.
10
 
11
  ## Pretraining Details
12
 
 
 
13
  Trained for 1M steps following [`google/t5-v1_1-small`](https://huggingface.co/google/t5-v1_1-small).
14
 
15
  ## Model Performance
 
6
  ---
7
  # Indonesian T5 Small
8
 
9
+
10
  T5 (Text-to-Text Transfer Transformer) model pretrained on Indonesian mC4 with [extra filtering](https://github.com/Wikidepia/indonesian_datasets/tree/master/dump/mc4). This model is pre-trained only and needs to be fine-tuned to be used for specific tasks.
11
 
12
  ## Pretraining Details
13
 
14
+ **Apparently I do not use the same learning rate as the original T5-smallv1.1, I will retrain it soon :)**
15
+
16
  Trained for 1M steps following [`google/t5-v1_1-small`](https://huggingface.co/google/t5-v1_1-small).
17
 
18
  ## Model Performance