Add tiktoken dependency for model tokenizer
Browse files- requirements.txt +2 -1
requirements.txt
CHANGED
|
@@ -2,4 +2,5 @@ transformers>=4.30.0
|
|
| 2 |
torch>=2.0.0
|
| 3 |
accelerate>=0.20.0
|
| 4 |
huggingface-hub>=0.16.0
|
| 5 |
-
gradio>=3.50.0
|
|
|
|
|
|
| 2 |
torch>=2.0.0
|
| 3 |
accelerate>=0.20.0
|
| 4 |
huggingface-hub>=0.16.0
|
| 5 |
+
gradio>=3.50.0
|
| 6 |
+
tiktoken>=0.5.0
|