Question: edge/mobile deployment — anyone tested?
#76
by 3morixd - opened
We benchmark models on 40 phones (Snapdragon 865) at Dispatch AI (FZE, UAE).
Question: has anyone tested this model on mobile/edge? Interested in:
- Inference speed (t/s)
- Model size after quantization
- RAM usage
Happy to share phone farm benchmark results.
- Dispatch AI (FZE), Sharjah UAE