kdirgul
/

smartcore-v1

Model card Files Files and versions

smartcore-v1 / code /kod /faz1_01_tokenize_shard.py

Commit History

faz1_01: --n_shards/--shard_index paralelize (ds.shard, ~8x)

9db6cba
verified

kdirgul commited on 14 days ago

faz1_01: --target_tokens bilimsel gosterim kabul (5e7/7.5e9)

6117cbf
verified

kdirgul commited on 14 days ago

faz1_01: tr_tc100b kaynagi (TC-100B ~103B TR, data_dir=pretrain) — v1.5b ana TR; 4-tuple data_dir destegi

69dc500
verified

kdirgul commited on 14 days ago

kod (data hariç) Colab için

9aed7c4
verified

kdirgul commited on Jun 5