Ben Graville commited on
docs: descriptive model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gemma
|
| 3 |
+
base_model: google/embeddinggemma-300m
|
| 4 |
+
library_name: coreml
|
| 5 |
+
pipeline_tag: sentence-similarity
|
| 6 |
+
tags:
|
| 7 |
+
- coreml
|
| 8 |
+
- core-ml
|
| 9 |
+
- apple-silicon
|
| 10 |
+
- on-device
|
| 11 |
+
- ane
|
| 12 |
+
- quantized
|
| 13 |
+
- int8
|
| 14 |
+
- embeddings
|
| 15 |
+
- sentence-embedding
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# embeddinggemma-300m β Core ML (int8, seq 128, ANE)
|
| 19 |
+
|
| 20 |
+
`google/embeddinggemma-300m` as a compiled Core ML encoder for Apple silicon, mirrored and
|
| 21 |
+
independently rebuild-verified by the [Visible](https://visible.cx) project. **This is the
|
| 22 |
+
embedding model Visible actually serves.** Core ML is the production embedding path in the
|
| 23 |
+
app; the LiteRT artifact is retained as a benchmark instrument and never as a serving
|
| 24 |
+
fallback, and the Core AI embedding bundle only runs in a specific build configuration on
|
| 25 |
+
macOS 27+.
|
| 26 |
+
|
| 27 |
+
It is also the one artifact in this entire catalog whose weights are **provably
|
| 28 |
+
reproducible**: a rebuild from the recipe on Linux produced a `weight.bin` that is
|
| 29 |
+
byte-for-byte identical to the published one.
|
| 30 |
+
|
| 31 |
+
## Provenance
|
| 32 |
+
|
| 33 |
+
| | |
|
| 34 |
+
|---|---|
|
| 35 |
+
| Base checkpoint | `google/embeddinggemma-300m` |
|
| 36 |
+
| Upstream bundle | `mlboydaisuke/embeddinggemma-300m-coreml` @ `3fa12f0b97b8afe23264f76800afe14af4615ca5` |
|
| 37 |
+
| Recipe | `john-rocky/CoreML-LLM` β `conversion/build_embeddinggemma_bundle.py --max-seq-len 128 --quantize int8` |
|
| 38 |
+
| Format | Core ML `.mlmodelc` (compiled), int8 weights |
|
| 39 |
+
| Sequence length | **128** (static) |
|
| 40 |
+
| Output | 768-d embedding |
|
| 41 |
+
| ANE residency | **~99.80%**, 1950/1954 ops (published claim; not re-measured by Visible) |
|
| 42 |
+
| Pinned in the app as | `CoreMLEmbeddingGemmaBundle` |
|
| 43 |
+
|
| 44 |
+
The mirror was verified byte-exact against upstream on all six files of the published
|
| 45 |
+
artifact before this card was written.
|
| 46 |
+
|
| 47 |
+
## Contents
|
| 48 |
+
|
| 49 |
+
Two top-level directories. The first is the artifact; the second is the evidence.
|
| 50 |
+
|
| 51 |
+
**`3fa12f0b97b8afe23264f76800afe14af4615ca5/`** β the pinned upstream artifact, byte for
|
| 52 |
+
byte, under its own upstream revision as the directory name. 309,358,096 B total.
|
| 53 |
+
|
| 54 |
+
| File | Bytes |
|
| 55 |
+
|---|---:|
|
| 56 |
+
| `encoder.mlmodelc/weights/weight.bin` | 308,616,576 |
|
| 57 |
+
| `encoder.mlmodelc/model.mil` | 735,948 |
|
| 58 |
+
| `encoder.mlmodelc/metadata.json` | 2,570 |
|
| 59 |
+
| `encoder.mlmodelc/coremldata.bin` | 408 |
|
| 60 |
+
| `encoder.mlmodelc/analytics/coremldata.bin` | 243 |
|
| 61 |
+
| `model_config.json` | 2,351 |
|
| 62 |
+
|
| 63 |
+
**`rebuild-verification/2026-08-17/`** β an independent rebuild of the same recipe on a
|
| 64 |
+
different operating system and CPU architecture, published so the reproducibility claim
|
| 65 |
+
below can be checked rather than taken on trust. 309,346,242 B total.
|
| 66 |
+
|
| 67 |
+
| File | Bytes |
|
| 68 |
+
|---|---:|
|
| 69 |
+
| `encoder.mlpackage/Data/com.apple.CoreML/weights/weight.bin` | 308,616,576 |
|
| 70 |
+
| `encoder.mlpackage/Data/com.apple.CoreML/model.mlmodel` | 723,393 |
|
| 71 |
+
| `encoder.mlpackage/Manifest.json` | 617 |
|
| 72 |
+
| `model_config.json` | 2,351 |
|
| 73 |
+
| `SHA256SUMS` | 427 |
|
| 74 |
+
| `VERIFICATION.md` | 2,878 |
|
| 75 |
+
|
| 76 |
+
Note the shapes differ deliberately: the pinned artifact is a **compiled** `.mlmodelc`, the
|
| 77 |
+
rebuild is the **uncompiled** `.mlpackage` the recipe emits. Compilation is a macOS-only
|
| 78 |
+
step (`xcrun coremlcompiler`) and was not faked on Linux.
|
| 79 |
+
|
| 80 |
+
## Requirements
|
| 81 |
+
|
| 82 |
+
- **Apple silicon**, Core ML runtime. The encoder is shaped for the Neural Engine.
|
| 83 |
+
- **Static sequence length 128.** Inputs must be padded or truncated to 128 tokens; this is
|
| 84 |
+
a compile-time property of the artifact, not a runtime option.
|
| 85 |
+
- Weights β 0.31 GB resident. This is a small model by every measure in this catalog and
|
| 86 |
+
imposes no meaningful memory tier.
|
| 87 |
+
- `model_config.json` beside the encoder carries the pooling/dense/normalisation contract
|
| 88 |
+
the host must honour: mean pooling β dense stack β L2 normalise, 768-d out.
|
| 89 |
+
|
| 90 |
+
## Measured performance
|
| 91 |
+
|
| 92 |
+
**No throughput or latency figure is published here.** The rebuild verification was run on
|
| 93 |
+
a Linux conversion box, where Core ML cannot execute at all
|
| 94 |
+
(`coremltools.libcoremlpython` does not load), so no vector was computed and no timing was
|
| 95 |
+
taken during that work.
|
| 96 |
+
|
| 97 |
+
The ANE residency figure (99.80%, 1950/1954 ops) is the **upstream published claim**, not a
|
| 98 |
+
Visible measurement. A re-measure via `MLComputePlan` is an open follow-up.
|
| 99 |
+
|
| 100 |
+
The last cross-runtime quality datapoint on record is a cosine of **~0.966** between the
|
| 101 |
+
Core ML leg and the LiteRT leg on short text, measured 2026-08-11 against the previously
|
| 102 |
+
installed copies β not against the artifacts in this repo.
|
| 103 |
+
|
| 104 |
+
## Qualification status
|
| 105 |
+
|
| 106 |
+
| Artifact | Status |
|
| 107 |
+
|---|---|
|
| 108 |
+
| `3fa12f0bβ¦/encoder.mlmodelc` + `model_config.json` | **SHIP** β this is the production embedding path in the Visible app, pinned by `CoreMLEmbeddingGemmaBundle`, with the compiled-in `weight.bin` SHA-256 matching. |
|
| 109 |
+
| `rebuild-verification/2026-08-17/encoder.mlpackage` | **VERIFICATION EVIDENCE, not a runtime artifact.** It is uncompiled and has never been executed. Compiling it and diffing the result against the pinned `.mlmodelc` is a named open gate (below). |
|
| 110 |
+
|
| 111 |
+
## Verification β the bit-exact result, stated precisely
|
| 112 |
+
|
| 113 |
+
The recipe was re-run on **Debian 13 / x86_64 / torch 2.7.1+cpu / coremltools 9.0 /
|
| 114 |
+
transformers 5.15.0**, against a bundle originally built on **macOS**. What reproduced:
|
| 115 |
+
|
| 116 |
+
| File class | Verdict |
|
| 117 |
+
|---|---|
|
| 118 |
+
| `weights/weight.bin` (308,616,576 B) | **IDENTICAL** β same SHA-256 `f81f60ebβ¦`, `cmp -l` reports **0 differing bytes**. Also equal to the app's compiled-in `CoreMLEmbeddingGemmaBundle.weightSha256`. |
|
| 119 |
+
| `model_config.json` (2,351 B) | **IDENTICAL** β SHA-256 `0b949875β¦` |
|
| 120 |
+
| tokenizer / config JSON emitted by the recipe | **IDENTICAL** β all files |
|
| 121 |
+
| `encoder.mlmodelc/model.mil`, `coremldata.bin` Γ2, `metadata.json` | **not produced on Linux** β these are products of `xcrun coremlcompiler`, which is Xcode/macOS-only |
|
| 122 |
+
|
| 123 |
+
So the honest claim is: **everything the recipe produces reproduces exactly, across
|
| 124 |
+
operating systems and CPU architectures; the remaining four files are a macOS compile step
|
| 125 |
+
that could not be run and was not faked.**
|
| 126 |
+
|
| 127 |
+
This is worth stating against the rest of the catalog, because it is the exception. Core AI
|
| 128 |
+
`.aimodel` bundles are **not** byte-reproducible β the exporter is not deterministic even
|
| 129 |
+
against itself, with two runs of an identical command nine minutes apart on the same box
|
| 130 |
+
differing by 32 bytes in `main.mlirb`. For those artifacts, integrity has to rest on
|
| 131 |
+
per-file SHA-256 of the exact published bytes. For this one, "verify by rebuilding" is
|
| 132 |
+
genuinely available for the weights.
|
| 133 |
+
|
| 134 |
+
### The open gate
|
| 135 |
+
|
| 136 |
+
Compile `rebuild-verification/2026-08-17/encoder.mlpackage` with
|
| 137 |
+
`xcrun coremlcompiler compile` on a Mac and diff the resulting `encoder.mlmodelc` against
|
| 138 |
+
the pinned root artifact. That closes the four remaining files and would establish whether
|
| 139 |
+
the *whole* Core ML artifact is reproducible or only its weights. Until then the claim is
|
| 140 |
+
scoped to the weights and the config, as written above.
|
| 141 |
+
|
| 142 |
+
## License
|
| 143 |
+
|
| 144 |
+
EmbeddingGemma is Gemma-family. These files are a derivative of
|
| 145 |
+
`google/embeddinggemma-300m` and use is subject to the
|
| 146 |
+
[Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the
|
| 147 |
+
[Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). Those
|
| 148 |
+
terms travel with the artifact and with any redistribution of it. Visible's contribution is
|
| 149 |
+
the mirror and the rebuild verification, not the weights.
|