bdstar commited on
Commit
6fe0230
·
verified ·
1 Parent(s): aebe41a

Add tokenizers

Browse files
Files changed (1) hide show
  1. requirements.txt +1 -0
requirements.txt CHANGED
@@ -11,6 +11,7 @@ onnxruntime==1.23.2
11
 
12
  # --- LLM fallback for Spaces (since no local Ollama) ---
13
  transformers==4.44.2
 
14
  accelerate
15
  safetensors
16
  sentencepiece
 
11
 
12
  # --- LLM fallback for Spaces (since no local Ollama) ---
13
  transformers==4.44.2
14
+ tokenizers==0.19.1 ✅ IMPORTANT FIX
15
  accelerate
16
  safetensors
17
  sentencepiece