--- base_model: - google/embeddinggemma-2 base_model_relation: quantized license: apache-2.0 pipeline_tag: feature-extraction library_name: coreml tags: - embeddings - multimodal - sentence-similarity - retrieval - matryoshka - apple-silicon - on-device - coreml - ios - macos - thestage-apple-sdk --- # EmbeddingGemma 2 (Apple Silicon) ![On-Device Apple SDK](assets/on-device-apple-sdk-banner.png) [TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK) **Original model:** [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) (Google) On-device **multimodal embeddings** compressed and packaged by **TheStage AI** for Apple Silicon (CoreML / Neural Engine). Text, images, audio, video, and mixed inputs (an image with its caption) map into **one 768-d, L2-normalized space**, so a typed query can find a paragraph, a photo, a sound or a clip. Matryoshka-trained: the first 512, 256 or 128 values are themselves embeddings. | | | | --- | --- | | Runtime | TheStage Apple SDK (`EmbeddingPipeline`) | | Embedding dim | **768** (L2-normalized); Matryoshka **512 / 256 / 128** | | Inputs | text, image, audio, video, mixed | | Default device | **NPU** (Neural Engine) | | SDK | **1.7.0+** | | HF engines | `TheStageAI/embeddinggemma-2` (tag `v1.7`) | ## Overview --- This repo hosts **CoreML engine bundles** for the TheStage Apple SDK. Point `engines_path` at the repo id; the SDK downloads, caches, and loads the engines automatically. Task prompts, token limits and Matryoshka sizes all come from the bundle. Open only the inputs you use (`modalities:`) to shorten the first load. ## System Requirements --- | **Property** | **Value** | | --- | --- | | **Hardware** | Apple Silicon Mac, or physical iPhone / iPad | | **macOS** | 15.0+ | | **iOS** | 18.0+ | | **Xcode** | 16.0+ | | **Swift** | 6.0+ | | **Flutter** (optional) | 3.24+ | > Simulator is **not** supported — run on real Apple Silicon hardware. ## TheStage AI Access Token --- Create a token at [app.thestage.ai](https://app.thestage.ai) and pass it to the SDK: ```swift try await TheStageAI.shared.initialize(api_token: "th_…") ``` ```dart await TheStageFlutterSDK.initialize(api_token: 'th_…'); ``` Token is checked online in `initialize` (once per app process when reachable). Inference runs fully on-device. Offline initialize fails — reconnect and call `initialize` again. ## TheStage Apple SDK --- Docs: [TheStage Apple SDK · Embeddings](https://docs.thestage.ai) · Repo: [TheStage Apple SDK](https://github.com/TheStageAI/AppleSDK) ### Installation (SwiftPM) In Xcode: **File → Add Package Dependencies…**, paste `https://github.com/TheStageAI/AppleSDK.git`, and add the `TheStageSDK` product. Or in `Package.swift`: ```swift .package( url: "https://github.com/TheStageAI/AppleSDK.git", exact: Version(1, 7, 0) ) ``` ### Installation (Flutter / iOS) ```yaml # pubspec.yaml dependencies: thestage_apple_sdk: git: url: https://github.com/TheStageAI/AppleSDK.git path: plugin/thestage_apple_sdk ref: 1.7.0 ``` ### Swift ```swift import TheStageSDK try await TheStageAI.shared.initialize(api_token: "th_…") let embedder = try await EmbeddingPipeline( engines_path: "TheStageAI/embeddinggemma-2", modalities: [.TEXT, .IMAGE] // nil = every input kind ) // Search: queries and documents use different task prompts let query: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY) let doc: EmbeddingResult = try await embedder.infer(text: doc_text, task: .DOCUMENT) let score: Float = try EmbeddingPipeline.similarity(query, doc) // Other inputs — same space let photo: EmbeddingResult = try await embedder.infer(image: cg_image) let product: EmbeddingResult = try await embedder.infer(inputs: [.image(cg_image), .text("red sneakers")]) // with modalities [.AUDIO] / [.VIDEO]: // try await embedder.infer(audio: samples, sample_rate: 44_100) // try await embedder.infer(video: file_url, max_frames: 8) // Smaller vectors let small: EmbeddingResult = try await embedder.infer(text: "how to reset the router", task: .QUERY, dimension: 256) ``` ### Flutter (iOS) ```dart await TheStageFlutterSDK.start_model( model_name: 'embed', engines_path: 'TheStageAI/embeddinggemma-2', ); final rows = await TheStageFlutterSDK.infer( model_name: 'embed', input_json: {'text': 'how to reset the router', 'task': 'query'}, ); final embedding = (rows[0]['embedding'] as List).cast(); // Images, audio, video: pass a file path final photo = await TheStageFlutterSDK.infer( model_name: 'embed', input_json: {'image_path': '/path/to/photo.jpg', 'dimension': 256}, ); ``` ### Inputs / outputs | Direction | Swift | Flutter key | Notes | | --- | --- | --- | --- | | in | `infer(text:)` | `text` | Up to 2048 tokens; longer text is cut (`truncated`) | | in | `infer(image:)` | `image_path` | Any size; 256–280 tokens per image | | in | `infer(audio:sample_rate:)` | `audio_path` | Mono, any rate (resampled); 25 tokens/s, first 30 s (`truncated` past that) | | in | `infer(video:max_frames:)` | `video_path`, `max_frames` | 1 frame/s, up to 14 frames (default), up to 140 tokens per frame | | in | `infer(inputs:)` | several keys | One embedding for mixed parts (Flutter order: text, image, audio, video) | | in | `task:` | `task` | `query`, `document`, `STS`, `Classification`, `Clustering`, `CodeRetrieval`, `QuestionAnswering`, `FactChecking` | | in | `dimension:` | `dimension` | 768 (default), 512, 256, 128 | | out | `embedding` | `embedding` | Unit-length vector | | out | `dimension`, `embedding_space_id` | same | Compare only results with equal `embedding_space_id` | | out | `token_count`, `truncated` | same | Input length read; whether anything was cut | ### Matryoshka sizes Reference quality of the base model (Google model card, bf16): | Size | MTEB eng v2 | MTEB Code | MIEB lite | MAEB | MMEB v2 | | --- | --- | --- | --- | --- | --- | | 768 | 68.46 | 78.68 | 65.67 | 49.87 | 59.01 | | 512 | 68.41 | 77.24 | 65.34 | 49.76 | 58.38 | | 256 | 67.78 | 76.18 | 64.16 | 49.13 | 56.24 | | 128 | 65.68 | 71.41 | 59.97 | 47.28 | 45.65 | 256 is near-lossless for text; 128 costs noticeably more on images, audio and video. For large indexes, store 768-d vectors, shortlist on the first 256 values renormalized, and re-rank the shortlist on 768. ### On-device latency Release build, Neural Engine, warmed, median of 5 (guidance, not an SLA): | Input | Tokens | Apple M-series Mac | iPhone 13 | | --- | --- | --- | --- | | Text query | 16 | **8.1 ms** | **12.6 ms** | | Text document | 368 | **47.6 ms** | **69.7 ms** | | Image 768x768 | 260 | **439.5 ms** | **538.0 ms** | | Audio 3 s | 79 | **137.7 ms** | **227.7 ms** | | Audio 30 s | 754 | **236.6 ms** | **382.7 ms** | | Video, 4 frames | 494 | **607.1 ms** | **757.8 ms** | | Image + caption | 271 | **439.8 ms** | **538.4 ms** | The first load on a device prepares every engine (about two minutes on a Mac, under four on an iPhone 13, with all four inputs); later loads take seconds (7.9 s on an iPhone 13). Memory stays under 200 MB once loaded on an iPhone 13. Always measure release builds on device. ## Acknowledgments --- This work builds on **EmbeddingGemma 2** by Google: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2). The base model [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) is released under Apache-2.0. This Apple Silicon package is produced by TheStage AI; it is not an official Google release. ## Links --- * __Original model__: [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2) * __Platform__: [app.thestage.ai](https://app.thestage.ai) * __TheStage Apple SDK__: [github.com/TheStageAI/AppleSDK](https://github.com/TheStageAI/AppleSDK) * __Docs__: [docs.thestage.ai](https://docs.thestage.ai) * __Subscribe for updates__: [TheStageAI X](https://x.com/TheStageAI) * __Contact email__: contact@thestage.ai