requirements.txt

numpy
omegaconf
msgspec
rouge-score
sacrebleu
sentencepiece
tiktoken
fsspec
blobfile
wandb
viztracer
scipy
pynvml
datatrove
orjson
pyarrow
transformers
pyahocorasick