enconvo's picture
EmbeddingGemma 2 for MLX: bf16 with an 8-bit token table, audio removed
d0090d2 verified
|
Raw History Blame Contribute Delete
1.44 kB
metadata
base_model: google/embeddinggemma-2
library_name: mlx
license: apache-2.0
pipeline_tag: feature-extraction
tags:
  - mlx
  - embedding
  - feature-extraction
  - sentence-similarity
  - image-feature-extraction

enconvo/embeddinggemma-2-mlx

EmbeddingGemma 2 by Google for MLX, as Enconvo's Local Embeddings runs it on the Mac: text and image encoders, 768-dimensional normalized vectors.

Built from mlx-community/embeddinggemma-2-bf16 (revision 1a4ffddb, converted from Google's revision 914f7f89):

  • The audio encoder is removed (audio_config: null).
  • The token embedding table (language_model.embed_tokens) is stored in 8 bits (affine, group size 64): it is only looked up, so this costs no speed.
  • Every other weight is bf16, unchanged. Keep activations in bf16.

config.json records the quantization of that one module, in mlx-lm's layout.

Checked against the source on a fixed probe set (16 texts, 8 images at soft-token budgets 70, 280): minimum cosine 0.999761 for text and 0.999757 for images.

Use the task prefixes in config_sentence_transformers.json (task: search result | query: for queries, title: none | text: for documents). An image is embedded as <bos> <boi> image tokens <eoi> <eos>.

Licence: Apache 2.0, see LICENSE and NOTICE.