mod-tokenizers-zero-padded-ltr_2digit / mod_tokenizers_zero_padded /flexitok--mod-tokenizers-zero-padded-ltr_2digit_overlap.json
gsaltintas's picture
Upload mod_tokenizers_zero_padded/flexitok--mod-tokenizers-zero-padded-ltr_2digit_overlap.json with huggingface_hub
4a0c881 verified
raw
history blame contribute delete
319 Bytes
{"8": {"ratio_to_total_tokens": 0.8411214953271028, "expected_training_ratio_in_superset": 0.7476635514018691, "num_tokens": 90}, "9": {"ratio_to_total_tokens": 0.1588785046728972, "expected_training_ratio_in_superset": 0.1588785046728972, "num_tokens": 17}, "total_training_compared_to_full_model": 0.9065420560747663}