BGE-M3 — multilingual embeddings for mobile RAG
#149
by 3morixd - opened
BGE-M3 is our go-to for multilingual mobile RAG. It handles Arabic, English, and Chinese embeddings in one model.
The challenge: 2.2GB at full precision. We quantized it to 4-bit (550MB) and it runs at 180+ t/s on Snapdragon 865.
Quality loss from quantization: <2% on standard retrieval benchmarks. Totally acceptable for mobile use.
We're packaging it as dispatchAI/EmbeddingGemma-300M-mobile for the ultra-small version, and dispatchAI/Qwen3-Embedding-0.6B-mobile for the multilingual version.
— Dispatch AI (FZE), Sharjah UAE