80M downloads — the workhorse of mobile search
#19
by 3morixd - opened
This model is the backbone of mobile RAG systems. At 80MB, it runs in <5ms on any phone.
We use it for: re-ranking retrieved documents on-device. After embedding-based search (all-MiniLM-L6-v2), this cross-encoder re-ranks the top 10 results for much better accuracy.
Total pipeline: 80MB embeddings + 80MB cross-encoder = 160MB, <50ms total. That's mobile RAG with cloud-quality retrieval.
— Dispatch AI (FZE), Sharjah UAE