49.4 MB

Ctrl+K

1 contributor

History: 15 commits

ModerRAS

Add parser diagnostics and inference debugging

b57780c 7 days ago

colab
Add Codex Colab training workflow 7 days ago
data
重建词表：从632K dmhy_weak.jsonl统计频率取top8000，覆盖96.2% 7 days ago
datasets
Add character-token DMHY training path 7 days ago
exports
重新训练模型，补充训练结果 7 days ago
model
重建词表：从632K dmhy_weak.jsonl统计频率取top8000，覆盖96.2% 7 days ago
.gitattributes

1.57 kB
重新训练模型，补充训练结果 7 days ago
.gitignore

171 Bytes
Add AniFileBERT model and training project 12 days ago
.gitmodules

118 Bytes
Add AniFileBERT model and training project 12 days ago
AGENTS.md

6.67 kB
Add Codex Colab training workflow 7 days ago
ANDROID.md

1.87 kB
Document AniFileBERT maintenance workflow 12 days ago
MAINTENANCE.md

2.6 kB
Document AniFileBERT maintenance workflow 12 days ago
README.md

8.06 kB
Add Codex Colab training workflow 7 days ago
check_f1.py

1.42 kB
Add AniFileBERT model and training project 12 days ago
colab_client.py

6.75 kB
Add Codex Colab training workflow 7 days ago
colab_train.py

19.2 kB
Add Codex Colab training workflow 7 days ago
colab_worker.py

16.9 kB
Add Codex Colab training workflow 7 days ago
config.json

1.39 kB
Add AniFileBERT model and training project 12 days ago
config.py

2.04 kB
重建词表：从632K dmhy_weak.jsonl统计频率取top8000，覆盖96.2% 7 days ago
convert_to_char_dataset.py

7.59 kB
Add character-token DMHY training path 7 days ago
data_generator.py

29.4 kB
Add AniFileBERT model and training project 12 days ago
dataset.py

7.16 kB
Add AniFileBERT model and training project 12 days ago
diagnose_pipeline.py

27.6 kB
Add parser diagnostics and inference debugging 7 days ago
diagnostics_report.md

11.4 kB
Add parser diagnostics and inference debugging 7 days ago
diagnostics_report_word.md

42.5 kB
Add parser diagnostics and inference debugging 7 days ago
dmhy_dataset.py

19.6 kB
完成整个数据集的整理 8 days ago
export_onnx.py

5.01 kB
Add AniFileBERT model and training project 12 days ago
inference.py

22.2 kB
Add parser diagnostics and inference debugging 7 days ago
llm_labeler.py

10.3 kB
完成整个数据集的整理 8 days ago
mix_datasets.py

2.32 kB
Add AniFileBERT model and training project 12 days ago
model.py

2.04 kB
Add AniFileBERT model and training project 12 days ago
model.safetensors

15.9 MB
xet

Add AniFileBERT model and training project 12 days ago
requirements.txt

162 Bytes
Add AniFileBERT model and training project 12 days ago
semantic_labeler.py

12 kB
完成整个数据集的整理 8 days ago
smoke_test.py

1.62 kB
Add AniFileBERT model and training project 12 days ago
test_train_small.py

3.79 kB
Add AniFileBERT model and training project 12 days ago
tokenizer.py

14.1 kB
完成整个数据集的整理 8 days ago
tokenizer_config.json

948 Bytes
Add AniFileBERT model and training project 12 days ago
train.py

16.7 kB
Add Codex Colab training workflow 7 days ago
training_args.bin
Detected Pickle imports (10)
- "transformers.trainer_utils.HubStrategy",
- "transformers.trainer_utils.SchedulerType",
- "accelerate.utils.dataclasses.DistributedType",
- "torch.device",
- "transformers.trainer_utils.IntervalStrategy",
- "transformers.trainer_pt_utils.AcceleratorConfig",
- "transformers.training_args.TrainingArguments",
- "transformers.trainer_utils.SaveStrategy",
- "accelerate.state.PartialState",
- "transformers.training_args.OptimizerNames"
How to fix it?
5.27 kB
xet

Add AniFileBERT model and training project 12 days ago
validate_fix.py

2.75 kB
Add AniFileBERT model and training project 12 days ago
verify_data.py

1.41 kB
Add AniFileBERT model and training project 12 days ago
vocab.char.json

91.6 kB
Mirror character vocab in model repo 7 days ago
vocab.json

144 kB
重建词表：从632K dmhy_weak.jsonl统计频率取top8000，覆盖96.2% 7 days ago

Detected Pickle imports (10)