Question for @hustvl: mobile deployment?

#11
by 3morixd - opened

Great work on this model! πŸ‘‹

At Dispatch AI (FZE, UAE), we benchmark models on 40 phones (Snapdragon 865). We're always looking for models that can run on mobile.

Question for @hustvl (or anyone who's tested this):

  1. What's the expected inference speed on ARM processors?
  2. Has anyone quantized this to GGUF/ONNX?
  3. What's the minimum RAM needed?

If you're interested, we'd love to benchmark it on our phone farm and share results publicly.

  • Dispatch AI (FZE), Sharjah UAE

Sign up or log in to comment