| --- |
| license: gemma |
| base_model: google/embeddinggemma-300m |
| library_name: onnx |
| pipeline_tag: sentence-similarity |
| language: |
| - en |
| tags: |
| - embeddinggemma |
| - onnx |
| - on-device |
| - android |
| - voice-assistant |
| - intent-classification |
| - tool-routing |
| --- |
| |
| # EmbeddingGemma-300M Phone Command Router (ONNX, 4-bit) |
|
|
| **What it does:** turns something a person says to their phone ("wake me at half six", "text mom I'm on my way", |
| "how long is the drive to Boston") into a ranking of **205 phone actions**: set an alarm, send a text, start |
| navigation, add to a list, and so on. It is the first stage of a fully on-device voice assistant for Android: this |
| model picks *which* action, and a small companion model, |
| [LFM2.5-350M Phone Command Extractor](https://huggingface.co/bpmatt/LFM2.5-350M-Phone-Command-Extractor-GGUF), |
| fills in *the details* (times, names, messages). |
|
|
| It is `google/embeddinggemma-300m`, fine-tuned for this task and exported to ONNX with 4-bit weights |
| (~195 MB). It runs on a phone CPU in about **30-40 ms** per request. |
|
|
| ## How it works |
|
|
| ``` |
| "remind me in 20 minutes to check the oven" |
| │ query text: "task: search result | query: remind me in 20 minutes to check the oven" |
| ▼ |
| EmbeddingGemma (this model) → 768-d vector → cosine vs. 205 tool embeddings (tool_embeddings.f32) |
| ▼ |
| set_timer 0.64 · add_reminder 0.47 · set_screen_timeout 0.43 … |
| ▼ the app's keyword rule ("remind me … to …" is a reminder) moves add_reminder to the top |
| ``` |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `model_q4.onnx` + `model_q4.onnx_data` | the model (inputs `input_ids`, `attention_mask`; output `sentence_embedding` [batch, 768]) | |
| | `tokenizer.json` | the Gemma tokenizer (for Python / Hugging Face `tokenizers`) | |
| | `gemma_tok.bin` | the same tokenizer in a compact format read by the Android app's Java tokenizer | |
| | `tools.json` | the 205 actions (name + description), in the row order of the embeddings | |
| | `tool_embeddings.f32` | 205 × 768 little-endian float32, unit-length tool embeddings | |
|
|
| ## Quick start (Python) |
|
|
| ```python |
| import json, numpy as np, onnxruntime as ort |
| from huggingface_hub import snapshot_download |
| from tokenizers import Tokenizer |
| |
| d = snapshot_download("bpmatt/EmbeddingGemma-300M-Phone-Command-Router-ONNX") |
| sess = ort.InferenceSession(f"{d}/model_q4.onnx") |
| tok = Tokenizer.from_file(f"{d}/tokenizer.json") |
| tools = json.load(open(f"{d}/tools.json")) |
| emb = np.fromfile(f"{d}/tool_embeddings.f32", dtype="<f4").reshape(tools["count"], tools["dim"]) |
| |
| def route(text, k=5): |
| ids = tok.encode(tools["query_prefix"] + text).ids |
| out = sess.run(["sentence_embedding"], {"input_ids": np.array([ids], dtype=np.int64), |
| "attention_mask": np.ones((1, len(ids)), dtype=np.int64)})[0][0] |
| q = out / (np.linalg.norm(out) + 1e-9) |
| s = emb @ q |
| return [(tools["tools"][i]["name"], round(float(s[i]), 3)) for i in np.argsort(-s)[:k]] |
| |
| print(route("wake me up at half six")) |
| ``` |
|
|
| Encode **one query per run**: the 4-bit model is sensitive to padding, so batching changes the scores slightly. |
|
|
| ## Results |
|
|
| Top-1 accuracy (the right action ranked first among 205), on held-out request sets never used for training or tuning: |
|
|
| | Set | Requests | Top-1 | |
| |---|---|---| |
| | Assistant dev set (everyday phone requests) | 294 | **96.7 %** | |
| | Public-dataset dev set (rephrased intents from public benchmarks) | 607 | **95.1 %** | |
|
|
| Inside the full assistant (this router + the extractor + safety rules + a confirmation step), 66 % of held-out requests |
| are completed correctly with no questions asked, 83 % are completed or settled with a single yes/no, and 4.6 % lead to a |
| wrong action. |
|
|
| ## Training |
|
|
| Fine-tuned (contrastive, query → tool description, with hard negatives between easily confused actions) on about |
| 12,000 examples: synthetic phone commands written for this project, covering every one of the 205 actions, plus samples |
| from public intent datasets (MASSIVE, TOPv2, MTOP, CLINC150, SNIPS, Home Assistant intents, and mobile-action datasets), |
| mapped onto the 205 actions. Held-out sets were checked for near-duplicates of the training data. |
|
|
| ## Use in the app |
|
|
| The Android app runs this model with ONNX Runtime, then adds a few deterministic rules (keyword anchors for families |
| such as alarms vs. timers, a chat filter, and a "Did you mean…?" question when the top two are close). Recognising the |
| action is all this model does: dates, numbers and names are filled in by the extractor and checked by app code. |
|
|
| ## Limitations |
|
|
| - English only; 205 fixed actions (a new action needs a new tool embedding, and ideally fine-tuning). |
| - It ranks actions; it does not decide whether a request is a command at all (the app uses a score threshold and rules). |
| - Requests that ask for two things at once ("set a timer and text Sam") are ranked as one; the app asks to split them. |
|
|
| ## Licence |
|
|
| Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. This model is a |
| Model Derivative of Gemma; see `NOTICE`, `GEMMA_TERMS_OF_USE.txt` and `GEMMA_PROHIBITED_USE_POLICY.txt`. |
|
|
