Model card (TheStage Apple SDK)

2616e2f verified 5 days ago

8.02 kB

---
base_model:
- google/embeddinggemma-2
base_model_relation: quantized
license: apache-2.0
pipeline_tag: feature-extraction
library_name: coreml
tags:
- embeddings
- multimodal
- sentence-similarity
- retrieval
- matryoshka
- apple-silicon
- on-device
- coreml
- ios
- macos
- thestage-apple-sdk
---
# EmbeddingGemma 2 (Apple Silicon)

![On-Device Apple SDK](assets/on-device-apple-sdk-banner.png)


[TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)

**Original model:** [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) (Google)

On-device **multimodal embeddings** compressed and packaged by **TheStage AI** for Apple Silicon (CoreML / Neural Engine). Text, images, audio, video, and mixed inputs (an image with its caption) map into **one 768-d, L2-normalized space**, so a typed query can find a paragraph, a photo, a sound or a clip. Matryoshka-trained: the first 512, 256 or 128 values are themselves embeddings.

| | |
| --- | --- |
| Runtime | TheStage Apple SDK (`EmbeddingPipeline`) |
| Embedding dim | **768** (L2-normalized); Matryoshka **512 / 256 / 128** |
| Inputs | text, image, audio, video, mixed |
| Default device | **NPU** (Neural Engine) |
| SDK | **1.7.0+** |
| HF engines | `TheStageAI/embeddinggemma-2` (tag `v1.7`) |


## Overview

---

This repo hosts **CoreML engine bundles** for the TheStage Apple SDK. Point `engines_path` at the repo id; the SDK downloads, caches, and loads the engines automatically. Task prompts, token limits and Matryoshka sizes all come from the bundle. Open only the inputs you use (`modalities:`) to shorten the first load.

## System Requirements

---

| **Property** | **Value** |
| --- | --- |
| **Hardware** | Apple Silicon Mac, or physical iPhone / iPad |
| **macOS** | 15.0+ |
| **iOS** | 18.0+ |
| **Xcode** | 16.0+ |
| **Swift** | 6.0+ |
| **Flutter** (optional) | 3.24+ |

> Simulator is **not** supported — run on real Apple Silicon hardware.


## TheStage AI Access Token

---

Create a token at [app.thestage.ai](https://app.thestage.ai) and pass it to the SDK:

```swift
try await TheStageAI.shared.initialize(api_token: "th_…")
```

```dart
await TheStageFlutterSDK.initialize(api_token: 'th_…');
```

Token is checked online in `initialize` (once per app process when reachable). Inference runs fully on-device. Offline initialize fails — reconnect and call `initialize` again.


## TheStage Apple SDK

---

Docs: [TheStage Apple SDK · Embeddings](https://docs.thestage.ai) · Repo: [TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)


### Installation (SwiftPM)

In Xcode: **File → Add Package Dependencies…**, paste `https://github.com/TheStageAI/AppleSDK.git`, and add the `TheStageSDK` product. Or in `Package.swift`:

```swift
.package(
    url: "https://github.com/TheStageAI/AppleSDK.git",
    exact: Version(1, 7, 0)
)
```

### Installation (Flutter / iOS)

```yaml
# pubspec.yaml
dependencies:
  thestage_apple_sdk:
    git:
      url: https://github.com/TheStageAI/AppleSDK.git
      path: plugin/thestage_apple_sdk
      ref: 1.7.0
```


### Swift

```swift
import TheStageSDK
try await TheStageAI.shared.initialize(api_token: "th_…")
let embedder = try await EmbeddingPipeline(
    engines_path: "TheStageAI/embeddinggemma-2",
    modalities: [.TEXT, .IMAGE]          // nil = every input kind
)
// Search: queries and documents use different task prompts
let query: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY)
let doc: EmbeddingResult = try await embedder.infer(text: doc_text, task: .DOCUMENT)
let score: Float = try EmbeddingPipeline.similarity(query, doc)
// Other inputs — same space
let photo: EmbeddingResult = try await embedder.infer(image: cg_image)
let product: EmbeddingResult = try await embedder.infer(inputs: [.image(cg_image), .text("red sneakers")])
// with modalities [.AUDIO] / [.VIDEO]:
//   try await embedder.infer(audio: samples, sample_rate: 44_100)
//   try await embedder.infer(video: file_url, max_frames: 8)
// Smaller vectors
let small: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY, dimension: 256)
```

### Flutter (iOS)

```dart
await TheStageFlutterSDK.start_model(
  model_name: 'embed',
  engines_path: 'TheStageAI/embeddinggemma-2',
);
final rows = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'text': 'how to reset the router', 'task': 'query'},
);
final embedding = (rows[0]['embedding'] as List).cast<double>();
// Images, audio, video: pass a file path
final photo = await TheStageFlutterSDK.infer(
  model_name: 'embed',
  input_json: {'image_path': '/path/to/photo.jpg', 'dimension': 256},
);
```

### Inputs / outputs

| Direction | Swift | Flutter key | Notes |
| --- | --- | --- | --- |
| in | `infer(text:)` | `text` | Up to 2048 tokens; longer text is cut (`truncated`) |
| in | `infer(image:)` | `image_path` | Any size; 256–280 tokens per image |
| in | `infer(audio:sample_rate:)` | `audio_path` | Mono, any rate (resampled); 25 tokens/s, first 30 s (`truncated` past that) |
| in | `infer(video:max_frames:)` | `video_path`, `max_frames` | 1 frame/s, up to 14 frames (default), up to 140 tokens per frame |
| in | `infer(inputs:)` | several keys | One embedding for mixed parts (Flutter order: text, image, audio, video) |
| in | `task:` | `task` | `query`, `document`, `STS`, `Classification`, `Clustering`, `CodeRetrieval`, `QuestionAnswering`, `FactChecking` |
| in | `dimension:` | `dimension` | 768 (default), 512, 256, 128 |
| out | `embedding` | `embedding` | Unit-length vector |
| out | `dimension`, `embedding_space_id` | same | Compare only results with equal `embedding_space_id` |
| out | `token_count`, `truncated` | same | Input length read; whether anything was cut |

### Matryoshka sizes

Reference quality of the base model (Google model card, bf16):

| Size | MTEB eng v2 | MTEB Code | MIEB lite | MAEB | MMEB v2 |
| --- | --- | --- | --- | --- | --- |
| 768 | 68.46 | 78.68 | 65.67 | 49.87 | 59.01 |
| 512 | 68.41 | 77.24 | 65.34 | 49.76 | 58.38 |
| 256 | 67.78 | 76.18 | 64.16 | 49.13 | 56.24 |
| 128 | 65.68 | 71.41 | 59.97 | 47.28 | 45.65 |

256 is near-lossless for text; 128 costs noticeably more on images, audio and video. For large indexes, store 768-d vectors, shortlist on the first 256 values renormalized, and re-rank the shortlist on 768.

### On-device latency

Release build, Neural Engine, warmed, median of 5 (guidance, not an SLA):

| Input | Tokens | Apple M-series Mac | iPhone 13 |
| --- | --- | --- | --- |
| Text query | 16 | **8.1 ms** | **12.6 ms** |
| Text document | 368 | **47.6 ms** | **69.7 ms** |
| Image 768x768 | 260 | **439.5 ms** | **538.0 ms** |
| Audio 3 s | 79 | **137.7 ms** | **227.7 ms** |
| Audio 30 s | 754 | **236.6 ms** | **382.7 ms** |
| Video, 4 frames | 494 | **607.1 ms** | **757.8 ms** |
| Image + caption | 271 | **439.8 ms** | **538.4 ms** |

The first load on a device prepares every engine (about two minutes on a Mac, under four on an iPhone 13, with all four inputs); later loads take seconds (7.9 s on an iPhone 13). Memory stays under 200 MB once loaded on an iPhone 13. Always measure release builds on device.

## Acknowledgments

---

This work builds on **EmbeddingGemma 2** by Google: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2).

The base model [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) is released under Apache-2.0. This Apple Silicon package is produced by TheStage AI; it is not an official Google release.

## Links

---


* __Original model__: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2)
* __Platform__: [app.thestage.ai](https://app.thestage.ai)
* __TheStage Apple SDK__: [github.com/TheStageAI/AppleSDK](https://github.com/TheStageAI/AppleSDK)
* __Docs__: [docs.thestage.ai](https://docs.thestage.ai)
* __Subscribe for updates__: [TheStageAI X](https://x.com/TheStageAI)
* __Contact email__: contact@thestage.ai

	---
	base_model:
	- google/embeddinggemma-2
	base_model_relation: quantized
	license: apache-2.0
	pipeline_tag: feature-extraction
	library_name: coreml
	tags:
	- embeddings
	- multimodal
	- sentence-similarity
	- retrieval
	- matryoshka
	- apple-silicon
	- on-device
	- coreml
	- ios
	- macos
	- thestage-apple-sdk
	---

	# EmbeddingGemma 2 (Apple Silicon)

	![On-Device Apple SDK](assets/on-device-apple-sdk-banner.png)


	[TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)

	Original model: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) (Google)

	On-device multimodal embeddings compressed and packaged by TheStage AI for Apple Silicon (CoreML / Neural Engine). Text, images, audio, video, and mixed inputs (an image with its caption) map into one 768-d, L2-normalized space, so a typed query can find a paragraph, a photo, a sound or a clip. Matryoshka-trained: the first 512, 256 or 128 values are themselves embeddings.

	\| \| \|
	\| --- \| --- \|
	\| Runtime \| TheStage Apple SDK (`EmbeddingPipeline`) \|
	\| Embedding dim \| 768 (L2-normalized); Matryoshka 512 / 256 / 128 \|
	\| Inputs \| text, image, audio, video, mixed \|
	\| Default device \| NPU (Neural Engine) \|
	\| SDK \| 1.7.0+ \|
	\| HF engines \| `TheStageAI/embeddinggemma-2` (tag `v1.7`) \|


	## Overview

	---

	This repo hosts CoreML engine bundles for the TheStage Apple SDK. Point `engines_path` at the repo id; the SDK downloads, caches, and loads the engines automatically. Task prompts, token limits and Matryoshka sizes all come from the bundle. Open only the inputs you use (`modalities:`) to shorten the first load.

	## System Requirements

	---

	\| Property \| Value \|
	\| --- \| --- \|
	\| Hardware \| Apple Silicon Mac, or physical iPhone / iPad \|
	\| macOS \| 15.0+ \|
	\| iOS \| 18.0+ \|
	\| Xcode \| 16.0+ \|
	\| Swift \| 6.0+ \|
	\| Flutter (optional) \| 3.24+ \|

	> Simulator is not supported — run on real Apple Silicon hardware.


	## TheStage AI Access Token

	---

	Create a token at [app.thestage.ai](https://app.thestage.ai) and pass it to the SDK:

	```swift
	try await TheStageAI.shared.initialize(api_token: "th_…")
	```

	```dart
	await TheStageFlutterSDK.initialize(api_token: 'th_…');
	```

	Token is checked online in `initialize` (once per app process when reachable). Inference runs fully on-device. Offline initialize fails — reconnect and call `initialize` again.


	## TheStage Apple SDK

	---

	Docs: [TheStage Apple SDK · Embeddings](https://docs.thestage.ai) · Repo: [TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK)


	### Installation (SwiftPM)

	In Xcode: File → Add Package Dependencies…, paste `https://github.com/TheStageAI/AppleSDK.git`, and add the `TheStageSDK` product. Or in `Package.swift`:

	```swift
	.package(
	url: "https://github.com/TheStageAI/AppleSDK.git",
	exact: Version(1, 7, 0)
	)
	```

	### Installation (Flutter / iOS)

	```yaml
	# pubspec.yaml
	dependencies:
	thestage_apple_sdk:
	git:
	url: https://github.com/TheStageAI/AppleSDK.git
	path: plugin/thestage_apple_sdk
	ref: 1.7.0
	```


	### Swift

	```swift
	import TheStageSDK

	try await TheStageAI.shared.initialize(api_token: "th_…")

	let embedder = try await EmbeddingPipeline(
	engines_path: "TheStageAI/embeddinggemma-2",
	modalities: [.TEXT, .IMAGE] // nil = every input kind
	)

	// Search: queries and documents use different task prompts
	let query: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY)
	let doc: EmbeddingResult = try await embedder.infer(text: doc_text, task: .DOCUMENT)
	let score: Float = try EmbeddingPipeline.similarity(query, doc)

	// Other inputs — same space
	let photo: EmbeddingResult = try await embedder.infer(image: cg_image)
	let product: EmbeddingResult = try await embedder.infer(inputs: [.image(cg_image), .text("red sneakers")])
	// with modalities [.AUDIO] / [.VIDEO]:
	// try await embedder.infer(audio: samples, sample_rate: 44_100)
	// try await embedder.infer(video: file_url, max_frames: 8)

	// Smaller vectors
	let small: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY, dimension: 256)
	```

	### Flutter (iOS)

	```dart
	await TheStageFlutterSDK.start_model(
	model_name: 'embed',
	engines_path: 'TheStageAI/embeddinggemma-2',
	);

	final rows = await TheStageFlutterSDK.infer(
	model_name: 'embed',
	input_json: {'text': 'how to reset the router', 'task': 'query'},
	);
	final embedding = (rows[0]['embedding'] as List).cast<double>();

	// Images, audio, video: pass a file path
	final photo = await TheStageFlutterSDK.infer(
	model_name: 'embed',
	input_json: {'image_path': '/path/to/photo.jpg', 'dimension': 256},
	);
	```

	### Inputs / outputs

	\| Direction \| Swift \| Flutter key \| Notes \|
	\| --- \| --- \| --- \| --- \|
	\| in \| `infer(text:)` \| `text` \| Up to 2048 tokens; longer text is cut (`truncated`) \|
	\| in \| `infer(image:)` \| `image_path` \| Any size; 256–280 tokens per image \|
	\| in \| `infer(audio:sample_rate:)` \| `audio_path` \| Mono, any rate (resampled); 25 tokens/s, first 30 s (`truncated` past that) \|
	\| in \| `infer(video:max_frames:)` \| `video_path`, `max_frames` \| 1 frame/s, up to 14 frames (default), up to 140 tokens per frame \|
	\| in \| `infer(inputs:)` \| several keys \| One embedding for mixed parts (Flutter order: text, image, audio, video) \|
	\| in \| `task:` \| `task` \| `query`, `document`, `STS`, `Classification`, `Clustering`, `CodeRetrieval`, `QuestionAnswering`, `FactChecking` \|
	\| in \| `dimension:` \| `dimension` \| 768 (default), 512, 256, 128 \|
	\| out \| `embedding` \| `embedding` \| Unit-length vector \|
	\| out \| `dimension`, `embedding_space_id` \| same \| Compare only results with equal `embedding_space_id` \|
	\| out \| `token_count`, `truncated` \| same \| Input length read; whether anything was cut \|

	### Matryoshka sizes

	Reference quality of the base model (Google model card, bf16):

	\| Size \| MTEB eng v2 \| MTEB Code \| MIEB lite \| MAEB \| MMEB v2 \|
	\| --- \| --- \| --- \| --- \| --- \| --- \|
	\| 768 \| 68.46 \| 78.68 \| 65.67 \| 49.87 \| 59.01 \|
	\| 512 \| 68.41 \| 77.24 \| 65.34 \| 49.76 \| 58.38 \|
	\| 256 \| 67.78 \| 76.18 \| 64.16 \| 49.13 \| 56.24 \|
	\| 128 \| 65.68 \| 71.41 \| 59.97 \| 47.28 \| 45.65 \|

	256 is near-lossless for text; 128 costs noticeably more on images, audio and video. For large indexes, store 768-d vectors, shortlist on the first 256 values renormalized, and re-rank the shortlist on 768.

	### On-device latency

	Release build, Neural Engine, warmed, median of 5 (guidance, not an SLA):

	\| Input \| Tokens \| Apple M-series Mac \| iPhone 13 \|
	\| --- \| --- \| --- \| --- \|
	\| Text query \| 16 \| 8.1 ms \| 12.6 ms \|
	\| Text document \| 368 \| 47.6 ms \| 69.7 ms \|
	\| Image 768x768 \| 260 \| 439.5 ms \| 538.0 ms \|
	\| Audio 3 s \| 79 \| 137.7 ms \| 227.7 ms \|
	\| Audio 30 s \| 754 \| 236.6 ms \| 382.7 ms \|
	\| Video, 4 frames \| 494 \| 607.1 ms \| 757.8 ms \|
	\| Image + caption \| 271 \| 439.8 ms \| 538.4 ms \|

	The first load on a device prepares every engine (about two minutes on a Mac, under four on an iPhone 13, with all four inputs); later loads take seconds (7.9 s on an iPhone 13). Memory stays under 200 MB once loaded on an iPhone 13. Always measure release builds on device.

	## Acknowledgments

	---

	This work builds on EmbeddingGemma 2 by Google: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2).

	The base model [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) is released under Apache-2.0. This Apple Silicon package is produced by TheStage AI; it is not an official Google release.

	## Links

	---


	* __Original model__: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2)
	* __Platform__: [app.thestage.ai](https://app.thestage.ai)
	* __TheStage Apple SDK__: [github.com/TheStageAI/AppleSDK](https://github.com/TheStageAI/AppleSDK)
	* __Docs__: [docs.thestage.ai](https://docs.thestage.ai)
	* __Subscribe for updates__: [TheStageAI X](https://x.com/TheStageAI)
	* __Contact email__: contact@thestage.ai