Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
📢‼️Official Message from Leaderboard Maintainers: Updates on MCMR
#176
by MINGYISU - opened
An official message from MMEB Leaderboard managers:
MCMR is a new Image Retrieval dataset, added in MMEB-V3. Previously it was not on the leaderboard by mistake, and has now been added to the leaderboard. All the overall scores are now computed with MCMR included, except Image-Overall(V2) which explicitly excludes MCMR to retain earlier scores from V2 submissions.
Existing V3 submissions might be missing MCMR scores, so if you see a score drop or a 0 for MCMR, please use the updated script to re-generate your model's scores sheets and ensure MCMR exists in the sheet.
Thank you for your cooperations and support for MMEB! If you still have any questions or believe there exists a bug, please let us know by creating a new discussion.
MINGYISU changed discussion title from Official Message from Leaderboard Maintainers: Updates on MCMR to ‼️Official Message from Leaderboard Maintainers: Updates on MCMR
MINGYISU changed discussion title from ‼️Official Message from Leaderboard Maintainers: Updates on MCMR to 📢‼️Official Message from Leaderboard Maintainers: Updates on MCMR