PD03 commited on
Commit
1dd709a
·
verified ·
1 Parent(s): 7ee1876

Update requirements.txt

Browse files
Files changed (1) hide show
  1. requirements.txt +1 -2
requirements.txt CHANGED
@@ -3,5 +3,4 @@ pandas
3
  duckdb
4
  transformers
5
  torch
6
- tiktoken
7
- blobfile
 
3
  duckdb
4
  transformers
5
  torch
6
+ sentencepiece # ← add this so the slow SP tokenizer works