Thierry Meyer commited on
Commit
40bc32b
·
1 Parent(s): 6f214d0

added tokenized datasets to the repository

Browse files
{ar → ckpts/ar}/qar_medium_xtrafo.ckpt RENAMED
File without changes
{ar → ckpts/ar}/qar_small_xtrafo.ckpt RENAMED
File without changes
{rq → ckpts/rq}/ResVQ_MLP_u20M_unkdc_snrdi_H1248_E3D3_K512xD8xQ8_EMA0.99_KM_b0.20_lr1e-5_Adam_bf16_tf32_ST3000k_epoch=66.ckpt RENAMED
File without changes
tokenized_datasets/spq_k512_Q8_D8/model_config.yaml ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ # Config file with details about the quantizers/tokenizers used to in the tokenization process
2
+ model_type: spq
3
+
4
+ checkpoints:
5
+ particles: /u/thmeyer/ptmp/particle_quantization/artifacts/ckpts/spq/ResVQ_MLP_u20M_unkdc_snrdi_H1248_E3D3_K512xD8xQ8_EMA0.99_KM_b0.20_lr1e-5_Adam_bf16_tf32_ST3000k_epoch=66.ckpt
6
+ energy_deposition: /u/thmeyer/ptmp/lego-eval/checkpoints/edep/edep_quantile_n2048.pt
tokenized_datasets/spq_k512_Q8_D8/rp_lqar_epg_re_ds_1.5M_290626_quantized.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8240e0b2283db982bb3f8f93a97baa1e8eb46d5bd608280b91057a07cc571d5
3
+ size 3780002625
tokenized_datasets/spq_k512_Q8_D8/rp_lqar_epg_re_ds_20M_290626_quantized.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c58a03ca86baecd06a08deffa4160bcf8173e2dd5c46369db1e7495aa60e0760
3
+ size 48640002732
tokenized_datasets/spq_k512_Q8_D8/rp_lqar_epg_re_ds_250K_020726_quantized.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58ee22845cce64ffa35ac6d6bacfd9975bbd31aeefe755a3003d4f95d6a3d2ed
3
+ size 553002625