Spaces:
Running
Running
Query regarding closed source models
#1
by atul-incept - opened
Hi guys, just wanted to check whether you support evaluation of closed source models as well?
hi @atul-incept thanks for your interest!
Currently, we support open-source models only as most of the benchmarks rely on log-probs which are not accessible through closed source models. However, if you want to evaluate any closed source model on generative tasks, you can still use our github repo https://github.com/tiiuae/QIMMA-leaderboard