What is the evaluation setting to get the benchmark result like GSM8K?
#7
by ljb121002 - opened
How to reproduce the result in https://qwenlm.github.io/blog/qwen-moe/? Qwen1.5-MoE-A2.7B gets 61.5 on GSM8K. Is it zero-shot? And what is the prompt? Thank you.
Did you work it out?
what about use Qwen1.5-MoE-A2.7B-Chat?