Anshulgada/embeddinggemma-2-740m

EmbeddingGemma 2 (740M - Base Backbone)

EmbeddingGemma 2 (740m) is an isolated, modularized foundation embedding checkpoint derived from Google DeepMind's google/embeddinggemma-2 (~740M parameters).

This repository contains the standalone base weights with full multimodal backbone (~740m parameters), optimized for lightweight downstream fine-tuning, retrieval-augmented generation (RAG), and zero-shot semantic similarity.

---

Model Architecture & Specifications

Parameter	Specification
Model ID	Anshulgada/embeddinggemma-2-740m
Base Architecture	Google Gemma 2 Transformer Backbone
Parameter Count	~740M parameters
Modality Specialization	Full Multimodal Backbone (~740M parameters)
Output Vector Dimension	768 dimensions (with native Matryoshka support down to 128d)
Max Context Window	8,192 tokens
Vocabulary Size	256,000 tokens (Gemma 2 SentencePiece tokenizer)
Supported Precision	bfloat16, float16, float32
Fine-Tuned Version	Anshulgada/tabman-embeddinggemma-2-740m
License	Apache 2.0

---

Matryoshka Representation Learning (MRL)

This checkpoint natively supports Matryoshka Representation Learning (MRL). You can truncate the 768-dimensional output vector to 512d, 256d, or 128d simply by slicing the first $N$ elements and re-normalizing:

embeddings_128d = embeddings[:, :128]
embeddings_128d = embeddings_128d / np.linalg.norm(embeddings_128d, axis=-1, keepdims=True)

---

Quickstart Usage

Using Sentence-Transformers (Recommended)

from sentence_transformers import SentenceTransformer

# Load the base model
model = SentenceTransformer("Anshulgada/embeddinggemma-2-740m")

queries = [
    "Efficient attention mechanisms in multimodal transformer architectures",
    "Scalable vector search and indexing using HNSW and ScaNN",
]

embeddings = model.encode(queries, normalize_embeddings=True)
print("Embedding Shape:", embeddings.shape)  # (2, 768)

# Truncate to 128d via Matryoshka slicing
embeddings_128d = embeddings[:, :128]
sim = model.similarity(embeddings_128d[0], embeddings_128d[1])
print(f"Cosine Similarity (128d): {sim.item():.4f}")

Using Hugging Face Transformers

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Anshulgada/embeddinggemma-2-740m")
model = AutoModel.from_pretrained("Anshulgada/embeddinggemma-2-740m", torch_dtype=torch.bfloat16)

inputs = tokenizer(["Semantic search query for embedding extraction"], return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
    # Mean pooling over token embeddings
    attention_mask = inputs["attention_mask"].unsqueeze(-1)
    embeddings = torch.sum(outputs.last_hidden_state * attention_mask, dim=1) / torch.clamp(
        attention_mask.sum(dim=1), min=1e-9
    )
    embeddings = F.normalize(embeddings, p=2, dim=1)

print("Embedding Vector (768d):", embeddings.shape)

---

Domain-Specific Fine-Tuned Version

For browser tab classification, URL clustering, web history semantic search, and WebGPU in-browser inference, see the companion fine-tuned model:

👉 Anshulgada/tabman-embeddinggemma-2-740m (Fine-tuned on 18,000 contrastive triplets, achieving 98.5%+ accuracy at 128 dimensions).

---

The Complete Tabman AI Model Matrix

Model Variant	Base Model (Raw Foundation)	Fine-Tuned Model (Tabman + WebGPU)	Parameters	Focus Area
270M	Anshulgada/embeddinggemma-2-270m	Anshulgada/tabman-embeddinggemma-2-270m	~270M	Text & Code / In-Browser Extension
440M	Anshulgada/embeddinggemma-2-440m	Anshulgada/tabman-embeddinggemma-2-440m	~440M	Text + Vision Multimodal
740M	Anshulgada/embeddinggemma-2-740m	Anshulgada/tabman-embeddinggemma-2-740m	~740M	Full Multimodal (Text + Vision + Audio)

---

License & Attribution

• Base Weights: Derived from Google DeepMind's google/embeddinggemma-2.
• License: Apache 2.0.