keras/harrier_embedding_oss_270m

Model Overview

Model Summary

Harrier Embedding 0.6B is a highly efficient multilingual text embedding model developed by Microsoft, built on top of the Qwen3 architecture. It is designed to generate fixed-size sentence embeddings suitable for semantic similarity, dense retrieval, and clustering tasks across 94+ languages.

Despite its compact size of just few million parameters, it achieves strong performance on embedding benchmarks, scoring 69.0 on Multilingual MTEB v2, making it an excellent choice for resource-constrained environments requiring high-quality multilingual retrieval.

Architecture Highlights:

• Base Architecture: Qwen3 (Transformer Decoder)
• Pooling Strategy: Last non-padding token pooling (default)
• Normalization: L2 normalized embeddings (default)
• Languages Supported: 94+

Intended Use

• Semantic Search / RAG: Generating embeddings for documents and queries in Retrieval-Augmented Generation systems.
• Clustering: Grouping similar text documents together based on spatial distance.
• Classification: Using the resulting embeddings as features for downstream classification tasks.

Training

All models are trained with contrastive learning objectives on a large-scale mixture of multilingual datasets covering diverse tasks. The 270m and 0.6b variants are additionally trained with knowledge distillation from larger embedding models.

Model Summary

A 270M parameter multilingual text embedding model based on the Gemma 3 architecture by Microsoft. Fine-tuned for dense retrieval and semantic similarity across 94+ languages, achieving 66.5 on Multilingual MTEB v2.

Overview

Harrier-OSS 270M is a highly efficient, multilingual text embedding model developed by Microsoft. It is built upon the decoder-only Gemma 3 270M architecture and fine-tuned specifically for dense vector representations.

The model maps variable-length text inputs into fixed-size 640-dimensional vectors, making it ideal for downstream NLP tasks such as:

• Semantic Search and Dense Retrieval
• Clustering and Topic Modeling
• Retrieval-Augmented Generation (RAG) pipelines
• Zero-shot cross-lingual information retrieval

Despite its small size (268M parameters), Harrier-OSS 270M achieves highly competitive performance across 94+ languages, scoring a 66.5 on the Multilingual MTEB v2 benchmark.

Architecture

• Backbone: Gemma 3 (Decoder-only causal language model)
• Parameters: 268 Million
• Hidden Dimension / Output Size: 640
• Pooling Strategy: Last-token pooling (standard for causal architectures)
• Normalization: L2-normalized outputs (ready for fast dot-product similarity)

Model Details

• [Harrier Quickstart Notebook](coming soon..)
• Harrier API Documentation
• Harrier Model Card
• KerasHub Beginner Guide
• KerasHub Model Publishing Guide

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the Keras Getting Started page.

Presets

Preset Name	Parameters	Description
harrier_embedding_oss_0.6B	0.6B	Multilingual text embedding model based on the Qwen3-0.6B architecture, fine-tuned for dense retrieval and semantic similarity across 94+ languages.
harrier_embedding_oss_270m	270M	Multilingual text embedding model based on the Gemma3-270M architecture, fine-tuned for dense retrieval and semantic similarity across 94+ languages.

Example Usage

Basic Semantic Search

import keras_hub

# Load the embedder
embedder = keras_hub.models.Qwen3TextEmbedder.from_preset(
   "harrier_embedding_oss_270m",
)


# Define query and documents
query = "Which planet is known as the Red Planet?"
documents = [
    "Mars is often referred to as the Red Planet.",
    "Venus is often called Earth's twin.",
]

# For best results with Harrier, prepend the instruction to queries
q_emb = embedder.encode_text(
    "Instruct: Given a web search query, retrieve relevant passages\n"
    f"Query: {query}"
)
d_embs = embedder.encode_text(documents)

# Compute cosine similarity
sims = embedder.similarity(q_emb, d_embs)
best_match_idx = sims.numpy().argmax()

print(f"Best match: {documents[best_match_idx]}")

Extracting Raw Embeddings

import keras_hub

embedder = keras_hub.models.Qwen3TextEmbedder.from_preset(
    "harrier_embedding_oss_270m",
)

# Encode a batch of texts
embeddings = embedder.encode_text([
    "The quick brown fox jumps over the lazy dog.",
    "Keras makes machine learning accessible."
])

print(embeddings.shape) # (2, 1024)

Example Usage with Hugging Face URI

Basic Semantic Search

import keras_hub

# Load the embedder
embedder = keras_hub.models.Qwen3TextEmbedder.from_preset(
   "hf://keras/harrier_embedding_oss_270m",
)


# Define query and documents
query = "Which planet is known as the Red Planet?"
documents = [
    "Mars is often referred to as the Red Planet.",
    "Venus is often called Earth's twin.",
]

# For best results with Harrier, prepend the instruction to queries
q_emb = embedder.encode_text(
    "Instruct: Given a web search query, retrieve relevant passages\n"
    f"Query: {query}"
)
d_embs = embedder.encode_text(documents)

# Compute cosine similarity
sims = embedder.similarity(q_emb, d_embs)
best_match_idx = sims.numpy().argmax()

print(f"Best match: {documents[best_match_idx]}")

Extracting Raw Embeddings

import keras_hub

embedder = keras_hub.models.Qwen3TextEmbedder.from_preset(
    "hf://keras/harrier_embedding_oss_270m",
)

# Encode a batch of texts
embeddings = embedder.encode_text([
    "The quick brown fox jumps over the lazy dog.",
    "Keras makes machine learning accessible."
])

print(embeddings.shape) # (2, 1024)