[feature request] Open LLM Leaderboardにはuse_remote_codeを使用しない
#3
by ke99L - opened
https://www.preferred.jp/ja/news/pr20230928/
モデルアーキテクチャの変更はないみたい、なぜuse_remote_codeを使用だろう？？
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
-> Submit here!
モデルにuse_remote_code=Trueが必要な場合、Open LLM Leaderboardはまだこのオプションをサポートしていません
revision branchやsafetensorsおすすめ
ELYZA 7b
mistral 7b
なとのもリストで評価できる
Thank you for your interest!
As we have outlined in our tech blog, the model architecture and tokenizer have been updated compared to the original transformers' implementation. Therefore, use_remote_code is required.
dhigurashi changed discussion status to closed