Spaces:

lemms
/

openllm

Runtime error

App Files Files Community

46.4 kB

Ctrl+K

Ctrl+K

1 contributor

History: 24 commits

lemms's picture

Fix: Use sentencepiece.SentencePieceProcessor() like local training code instead of AutoTokenizer

d6ce9fb verified 8 months ago

.gitattributes

1.52 kB
initial commit 8 months ago
README.md

2.81 kB
Update README for Gradio 4.44.1 and complete training features 8 months ago
app.py

26 kB
Fix: Use sentencepiece.SentencePieceProcessor() like local training code instead of AutoTokenizer 8 months ago
install_dependencies.py

3.84 kB
Add manual dependency installation script to fix sentencepiece import issues 8 months ago
openllm_tokenizer_fix.py

1.81 kB
Add OpenLLM custom tokenizer test script 8 months ago
requirements.txt

2.48 kB
Update requirements: Add comprehensive dependencies including sentencepiece for OpenLLM model compatibility 8 months ago
test_dependencies.py

3.45 kB
Add dependency test script to diagnose import issues 8 months ago
test_sentencepiece.py

4.43 kB
Add sentencepiece-specific test script to diagnose tokenizer issues 8 months ago