smdesai/Nemotron-3.5-Embed-1B-CoreAI

Nemotron-3.5-Embed-1B for Core AI

nvidia/Nemotron-3.5-Embed-1B-BF16 converted to Apple Core AI to run on an iPhone or a Mac GPU.

• The model: a 1B-parameter multimodal retrieval embedder: a Pixtral vision tower and projector feeding a 16-layer bidirectional Ministral-3 language model.
• Output: one 2048-d embedding (mean-pooled, L2-normalized, Matryoshka) per query (text) or passage (text, a page image, or both).
• Use: match questions against documents, charts, screenshots and photos by meaning.

Builds

Build	Download	Weights	Use
w8 (repo root)	1.85 GB	8-bit k-means palettized (one LUT per tensor), fp16 compute	iPhone; the default
float16/	3.13 GB	float16; the language model in two 8-layer chunks	Mac; exact precision

Both share host/. A download of either build is a runtime bundle: manifest.json, its .aimodel files and host/. Core AI specializes each model on its first load: about 4 s on an iPhone 17 Pro for w8, which is then cached.

Files

File	Size	What
NemotronEmbedText-w8.aimodel	873 MB	text / language model (16 layers), 8-bit k-means weights; functions embed_s128/512/2048
NemotronEmbedVision-w8.aimodel	415 MB	Pixtral vision encoder + projector, 8-bit k-means weights; functions encode_n1024/2304/4096/6400
float16/NemotronEmbedText-float16-c1of2.aimodel	873 MB	language model layers 0-7, float16
float16/NemotronEmbedText-float16-c2of2.aimodel	873 MB	language model layers 8-15 + norm + mean pooling, float16
float16/NemotronEmbedVision-float16.aimodel	828 MB	vision encoder + projector, float16
host/embed_tokens.f16	537 MB	token embedding table [131072, 2048], gathered on the host
host/text_rope.f32	2 MB	the language model's YaRN RoPE tables [2, 2048, 128]
host/tokenizer.json	22 MB	the upstream tokenizer

host/ also holds tokenizer_config.json, chat_template.jinja, processor_config.json and config.json from the upstream checkpoint.

How it runs

The host does the work around the two graphs:

• Prompt: the chat template with the query: / passage: prefix, then tokenization.
• Image preprocessing: Pixtral's, longest edge 1120 px.
• Inputs: patchify and the vision RoPE tables. The token embeddings are gathered from host/embed_tokens.f16, and the image features are spliced into the [IMG] slots.
• Buckets: every input is padded to the smallest bucket that holds it: 128, 512 or 2048 tokens; 1024, 2304, 4096 or 6400 patches.
• Output: the host L2-normalizes the embedding. For a Matryoshka size k, take the first k values and renormalize.

Accuracy

Checked against the upstream transformers model in fp32 on 19 cases:

• queries and passages in five languages, up to 1817 tokens;
• seven images: charts, pages, a 2000-px strip, a 50×30 image and RGBA;
• the Swift runtime's token ids and pixels are exact.

Build	Device	Embedding cosine to fp32 (worst case)
float16	MacBook Pro M3 Max	0.99999620
float16	iPhone 17 Pro	0.99999630
w8	MacBook Pro M3 Max	0.99967110
w8	iPhone 17 Pro	0.99968213

Retrieval quality, the shipped assets run through Core AI (nDCG@10, paired bootstrap 95% CI):

Benchmark	float16	w8	Δ (95% CI)
MIRACL-ja (250 queries, 8000-document pool with hard negatives)	0.8732	0.8731	−0.0001 [−0.0007, +0.0004]
ViDoRe V3 computer science (215 queries, 1360 page images)	0.7678	0.7689	+0.0011 [−0.0001, +0.0026]

4-bit palettization (16-channel LUT groups) loses 0.013 nDCG on MIRACL-ja with an interval that excludes 0, so there is no 4-bit build.

Performance (median of one call, after warm-up)

Build	Device	Load (cached)	128-token text	1024-patch image	6400-patch page
w8	iPhone 17 Pro	3.4 s (first launch the same)	93 ms	197 ms	1.9 s
float16	iPhone 17 Pro	11.8 s (first launch 290 s)	52 ms	306 ms	3.0 s
float16	MacBook Pro M3 Max	7.5 s	25 ms	98 ms	1.2 s

On an iPhone the w8 build runs within the default memory limit (about 850 MB footprint while indexing images).

License

The model weights are NVIDIA's, under the OpenMDW-1.1 License, as upstream. The Ministral-3 base models are governed by the Apache License, Version 2.0.