Cursor Agent
commited on
Commit
·
a930d30
1
Parent(s):
ffe15e5
fix: use huggingface-hub 0.17.3 for tokenizers compatibility
Browse filesCRITICAL FIX for tokenizers dependency:
- tokenizers 0.14.x requires huggingface-hub<0.18
- Previous 0.19.4 was too new
- Now using 0.17.3 (highest compatible version)
Version constraints satisfied:
- transformers 4.35.0: ✅ (needs >=0.16.4,<1.0)
- datasets 2.14.5: ✅ (needs >=0.14.0,<1.0)
- tokenizers 0.14.x: ✅ (needs >=0.16.4,<0.18)
This should resolve immediately without backtracking.
- requirements.txt +6 -3
requirements.txt
CHANGED
|
@@ -37,9 +37,12 @@ watchdog==6.0.0
|
|
| 37 |
dnspython==2.7.0
|
| 38 |
|
| 39 |
# HuggingFace (optional - for AI models)
|
| 40 |
-
#
|
| 41 |
-
#
|
| 42 |
-
huggingface-hub
|
|
|
|
|
|
|
|
|
|
| 43 |
datasets==2.14.5
|
| 44 |
|
| 45 |
# Utilities
|
|
|
|
| 37 |
dnspython==2.7.0
|
| 38 |
|
| 39 |
# HuggingFace (optional - for AI models)
|
| 40 |
+
# CRITICAL VERSION CONSTRAINTS:
|
| 41 |
+
# - transformers 4.35.0 requires: huggingface-hub>=0.16.4,<1.0
|
| 42 |
+
# - datasets 2.14.5 requires: huggingface-hub>=0.14.0,<1.0.0
|
| 43 |
+
# - tokenizers 0.14.x requires: huggingface-hub>=0.16.4,<0.18 ← THE CONSTRAINT!
|
| 44 |
+
# Compatible version: 0.16.4 <= huggingface-hub < 0.18
|
| 45 |
+
huggingface-hub==0.17.3 # Maximum version compatible with tokenizers
|
| 46 |
datasets==2.14.5
|
| 47 |
|
| 48 |
# Utilities
|