gsaltintas commited on
Commit
a8ae434
·
verified ·
1 Parent(s): b044774

Upload mod_tokenizers_zero_padded/flexitok--mod-tokenizers-zero-padded-ltr_4digit_overlap.json with huggingface_hub

Browse files
mod_tokenizers_zero_padded/flexitok--mod-tokenizers-zero-padded-ltr_4digit_overlap.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"2": {"ratio_to_total_tokens": 0.08404506632745776, "expected_training_ratio_in_superset": 0.018676681406101722, "num_tokens": 925}, "4": {"ratio_to_total_tokens": 0.8181900781391968, "expected_training_ratio_in_superset": 0.3636400347285319, "num_tokens": 9005}, "6": {"ratio_to_total_tokens": 0.0880428856987098, "expected_training_ratio_in_superset": 0.0586952571324732, "num_tokens": 969}, "8": {"ratio_to_total_tokens": 0.008177357804833727, "expected_training_ratio_in_superset": 0.007268762493185535, "num_tokens": 90}, "9": {"ratio_to_total_tokens": 0.0015446120298019262, "expected_training_ratio_in_superset": 0.0015446120298019262, "num_tokens": 17}, "total_training_compared_to_full_model": 0.4498253477900943}