GooGooLM / scripts /01_data

240 kB

Ctrl+K

1 contributor

History: 1 commit

XiaoyanLi

Initial upload: code, training data, tokenizers, notes

83112d8 verified 4 months ago

analyze_blimp_coverage.py

4.4 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
find_oov_words.py

2.75 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
run_all.py

7.11 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage1_noise_filter.py

6.33 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage2_oov_filter.py

5.7 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage3_dedup.py

3.63 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage4_preprocess.py

14.9 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage5_rule_label.py

62.3 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage5a_vocab_build.py

42.9 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage6_qwen_label.py

12.2 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage7a_extract_eval.py

8.73 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage8_config.py

9.31 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage8_sample_A.py

4.46 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage8_sample_B.py

7.9 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage8_sample_C.py

7.34 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage8_sample_D.py

20 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
stage9_paraphrase.py

16.5 kB
Initial upload: code, training data, tokenizers, notes 4 months ago
vocab_coverage.py

3.35 kB
Initial upload: code, training data, tokenizers, notes 4 months ago