embeddinggemma-2-p150 on Tenstorrent p150

Derived from google/embeddinggemma-2 by Google DeepMind, weights unmodified. The TTNN port is models/autoports/google_embeddinggemma_2, on stock upstream tt-metal.

EmbeddingGemma 2, a multimodal embedding model. Text, code, images, video frames and 16 kHz audio each map to one L2-normalized 768-dimensional vector in a shared space, so a text query can find an image, a sound or a video. Matryoshka dimensions 512, 256 and 128 are the first values of the 768-dimensional vector, re-normalized on the chip.

The text tower is a bidirectional 24-layer Gemma encoder (hidden size 512, sliding and full attention, 8192-token context). Gemma 4 vision (16 layers) and audio (12 layers) towers feed soft tokens into it. Weights stay on the chip and every request shape runs as a captured TTNN trace. Two custom kernels (sliding-window SDPA and a GEGLU matmul) run through stock ttnn.generic_op; tt-metal itself is unmodified.

Runs on Tenstorrent p150 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150
License	apache-2.0

Intended use

Embeddings for retrieval, semantic search, clustering and classification over text, code, images, video and audio, served from one p150. Not for generating text.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150
• 1 GB hugepages mounted at /dev/hugepages-1G
• One chip of a p300 also works: the package sets the p150 mesh graph descriptor.

Quickstart

tt serve tt-hous/embeddinggemma-2

tt serve downloads the Docker image and the google/embeddinggemma-2 at 914f7f89142e33e77833254d9c9b90c3cef7303b weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/embeddinggemma-2.

Embedding text

curl -s localhost:20000/v1/embeddings -H 'content-type: application/json' \
  -d '{"input": ["What is the capital of France?"], "input_type": "query", "dimensions": 256}'

Demo

tt serve tt-hous/embeddinggemma-2 --profile demo

Open http://<host>:20000/demo/. A web page where every query is embedded live on the chip, over a bundled library of 163 openly licensed images, videos, sounds, texts in 10 languages and code snippets: media search, finding a moment in a long video, audio search, code search, multilingual search, and a side-by-side Matryoshka comparison.

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	description
default (default)	p150	P150	OpenAI-style /v1/embeddings server only.
demo	p150	P150	The /v1/embeddings server plus a Gradio demo at /demo (media, video moments, audio, code, multilingual and Matryoshka search over a bundled openly licensed library).

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /v1/embeddings follows the OpenAI embeddings request and response shapes, with media extensions.

field	meaning	default
input	a string, or a list of 1 to 2048 strings and media items	required
dimensions	768, 512, 256 or 128	768
input_type	a sentence-transformers prompt name, for example query, document, CodeRetrieval	no prompt
encoding_format	float, or base64 (little-endian float32)	float
model	optional; google/embeddinggemma-2 or embeddinggemma-2

A media item is an object with optional text and at most one media field:

field	value
image	base64 image bytes (any PIL format) or a data:image/...;base64, URL
audio	base64 16 kHz WAV; clips over 30 s are cut to 30 s
video	{"frames": [<base64 image>, ...], "fps": 1}; at most 32 frames are used

IMG=$(base64 -w0 cats.jpg)
curl -s localhost:20000/v1/embeddings -H 'content-type: application/json' \
  -d "{\"input\": [{\"image\": \"$IMG\"}, {\"text\": \"two cats on a couch\"}]}"

GET /health answers 200 once the traces are captured, with the precision policy and live counters, and 503 before that. GET /v1/models lists the model, its dimensions and modalities. Errors use the OpenAI error shape; an input over 8192 tokens gets 400 context_length_exceeded. Media URLs are refused: send media as base64.

Expected performance

benchmark (768d)	metric	this package	HF fp32	card
MTEB multilingual v2	Mean(Task)	59.55	59.51	61.36
MTEB eng v2	Mean(Task)	70.25	70.35	68.46
MTEB code v1	Mean(Task), NDCG@10	56.99	57.04	78.68
MIEB lite	Mean(TaskType)	60.93	60.50	64.64
MMEB v2 Image	Mean(Task), Hit@1	38.85	39.10	57.28
MMEB v2 VisDoc	Mean(Task), NDCG@5	70.26	69.39	67.84
MMEB v2 Video	Mean(Task), Hit@1	20.09	19.63	50.67
MSEB Retrieval	Mean(Task), MRR@10	80.20	80.18	69.54
MAEB	Mean(Task)	58.16	58.28	49.39

dimensions	storage	MTEB multilingual	MTEB eng	MTEB code	MIEB lite	MMEB overall	MSEB	MAEB
768	1x	59.55	70.25	56.99	60.93	44.19	80.20	58.16
512	1/1.5	59.20	70.24	56.62	60.77	44.70	78.99	57.93
256	1/3	58.61	69.54	55.72	59.50	42.37	76.26	57.84
128	1/6	55.76	67.90	52.30	55.57	37.12	60.25	55.25

load, one text of 480 to 512 tokens per request	requests/s	p50 (ms)	p99 (ms)
1 user	88.5	11.4	13.1
8 users	207.7	38.5	40.5
32 users	277.3	114.1	122.3

single request, warm	latency (ms)
text, 128 tokens	5.8
text, 512 tokens	7.7
text, 8192 tokens	80.3
image	47.8
video, 8 frames	138.1
audio, 30 s	32.2

Measured on one Blackhole chip of a p300c board opened as a 1x1 mesh (equal to a p150, compute grid 11x10), precision policy x02. Accuracy: a fixed subset of each benchmark, scored with the same inputs, prompts and code on this package and on the Hugging Face fp32 reference on CPU; "card" is the full-suite score from the google/embeddinggemma-2 card and is not comparable to subset means. No benchmark mean differs from HF fp32 by more than 1.0 point at any dimension. Load: a real server with closed-loop clients, 60 s per level, 0 errors. Single request: host staging, trace replay and readback, without tokenization or media decoding.

Limitations

• Bring-up build, not a Tenstorrent release.
• One chip per server; no multi-chip mode.
• At most 8192 tokens per input, counting the prompt and media tokens.
• A vector depends slightly on the device batch it ran in (cosine down to 0.98 on hard inputs). Send one input per request for repeatable vectors.
• Against HF fp32, 8 of the evaluation texts and 37 audio clips fall below 0.995 cosine (minimum 0.959); benchmark means still match within 1.0 point.
• Audio must be 16 kHz WAV and is cut to 30 s. Video arrives as decoded frames, at most 32. Images use the default 280-token budget.
• Never run in fp16: the base model card warns it gives NaN or degraded vectors.
• The first boot compiles kernels and captures 51 traces (about 6 minutes); later boots take about 30 s.
• Validated on one chip of a p300c. Performance on a p150 with a larger compute grid was not measured.

Licensing

Weights under the upstream Apache-2.0 licence (google/embeddinggemma-2). The TTNN port and the server in code/ are Apache-2.0 (tt-metal). Demo library items carry their own CC0, public domain, CC BY, CC BY-SA, Apache-2.0 or MIT licences, listed in code/models/autoports/google_embeddinggemma_2/demo/library/LICENSES.md.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/embeddinggemma-2/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	3be67656b28e3902f0366ea8bab03c41e7e5bd6c
code/ digest	852ada085de3df45 (sha256, first 16 hex digits)
built	2026-10-21T15:23:55+00:00 by tt-model 0.1.0