put MODEL_DIR export separately

reorder "import jax" to avoid the stuck on importing it, change the dataset

added run finetuning

updated minimal line length

delete .idea folder

update <|endoftext|> tokenizer id from 50257 to 50256

refactor tokenizer related files with eos token

change endoftext value to len of tokenizer vocab size

fixed mismatched vocab_size between model and tokenizer

Add bias analysis

Changed very long line to short multi lines to make it easier for editing in IDE.

Update README.md

Merge branch 'main' of https://huggingface.co/flax-community/gpt2-medium-indonesian into main

model udpate

Update README.md

Update README.md

Update README.md

model update

model update

added text collection

updated the models

updated the model

udpated the model and script to load local data

Saving weights and logs of step 65000

Saving weights and logs of step 60000

Saving weights and logs of step 55000

Saving weights and logs of step 50000

Saving weights and logs of step 45000

Saving weights and logs of step 40000

Saving weights and logs of step 35000

Saving weights and logs of step 30000

Saving weights and logs of step 25000

Saving weights and logs of step 20000

Saving weights and logs of step 15000

Saving weights and logs of step 10000

Saving weights and logs of step 5000

add tokenizers files

update jax converter

updated pytorch model

remove wandb, add gitignore

Saving weights and logs of step 15000

updated the model

Saving weights and logs of step 10000

added th epytorch model

Saving weights and logs of step 5000

Saving weights and logs of step 100

Saving weights and logs of step 90

Saving weights and logs of step 80

Saving weights and logs of step 70

Saving weights and logs of step 60