File size: 6,260 Bytes

---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- rag
- universal-embedding
- matryoshka
- embeddings
- information-retrieval
base_model: nomic-ai/modernbert-embed-base
pipeline_tag: sentence-similarity
library_name: sentence-transformers
model-index:
- name: TensorFluxEmbedder
  results:
  - task:
      type: information-retrieval
    dataset:
      name: SciFact
      type: scifact
    metrics:
    - type: ndcg_at_10
      value: 0.718
  - task:
      type: information-retrieval
    dataset:
      name: Legal RAG
      type: custom
    metrics:
    - type: ndcg_at_10
      value: 0.422
---

# TensorFluxEmbedder

TensorFluxEmbedder is a fine-tuned text embedding model covering four domains — general language, web search, scientific literature, and legal documents. Embeddings can be truncated from 768 → 512 → 256 → 128 → 64 dimensions at inference time with no retraining, letting you directly trade retrieval quality for speed and memory.

## Model Specifications

| Property | Value |
|---|---|
| **Architecture** | ModernBERT |
| **Embedding dimensions** | 768, 512, 256, 128, 64 |
| **Max input length** | 8,192 tokens |
| **Similarity metric** | Cosine |
| **Language** | English |
| **License** | Apache 2.0 |

## Performance

<div align="center">

![TensorFluxEmbedder Performance Chart](assets/TensorFluxEmbedderPerformance.png)

</div>


NDCG@10 on held-out test sets across all five supported dimensionalities:

| Dims | SciFact NDCG@10 | Legal NDCG@10 | Vector size vs 768d |
|-----:|:---:|:---:|:---:|
| **768** | **0.718** | **0.422** | 100% |
| 512 | 0.707 | 0.417 | 67% |
| 256 | 0.697 | 0.400 | 33% |
| 128 | 0.665 | 0.345 | 17% |
| 64  | 0.587 | 0.278 | 8%  |

**256 dimensions** is the recommended default for latency-sensitive RAG: it retains 97% of peak SciFact quality at one-third the storage cost.

## Full Evaluation Results

### SciFact (held-out test set)

| Metric | 768d | 512d | 256d | 128d | 64d |
|---|:---:|:---:|:---:|:---:|:---:|
| Accuracy@1  | 0.603 | 0.603 | 0.580 | 0.543 | 0.447 |
| Accuracy@10 | 0.860 | 0.843 | 0.840 | 0.810 | 0.743 |
| Recall@1    | 0.576 | 0.573 | 0.553 | 0.524 | 0.434 |
| Recall@10   | 0.851 | 0.831 | 0.829 | 0.795 | 0.727 |
| **NDCG@10** | **0.718** | **0.707** | **0.697** | **0.665** | **0.587** |
| MRR@10      | 0.683 | 0.675 | 0.663 | 0.631 | 0.549 |
| MAP@100     | 0.678 | 0.670 | 0.656 | 0.625 | 0.544 |

### Legal RAG (held-out 10% split)

| Metric | 768d | 512d | 256d | 128d | 64d |
|---|:---:|:---:|:---:|:---:|:---:|
| Accuracy@1  | 0.158 | 0.161 | 0.145 | 0.107 | 0.093 |
| Accuracy@10 | 0.702 | 0.692 | 0.675 | 0.598 | 0.491 |
| Recall@1    | 0.158 | 0.161 | 0.145 | 0.107 | 0.093 |
| Recall@10   | 0.702 | 0.692 | 0.675 | 0.598 | 0.491 |
| **NDCG@10** | **0.422** | **0.417** | **0.400** | **0.345** | **0.278** |
| MRR@10      | 0.332 | 0.329 | 0.312 | 0.265 | 0.212 |
| MAP@100     | 0.342 | 0.338 | 0.321 | 0.276 | 0.224 |

The legal corpus uses a 1:1 query-to-passage evaluation scheme where each query has exactly one relevant passage in the full corpus, making the task harder than typical multi-relevant benchmarks.

## Usage

```bash
pip install sentence-transformers
```

Always prefix your inputs to distinguish query intent from document content:

| Input type | Prefix |
|---|---|
| Search query | `search_query: ` |
| Document / passage | `search_document: ` |

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("IstishadAlamTishad/TensorFluxEmbedder")

queries = [
    "search_query: What are the procurement rules for small business government contracts?",
]
documents = [
    "search_document: Agency procurement regulations require that small business offerors "
    "receive fair opportunities pursuant to FAR Part 19.",
    "search_document: Antibody neutralization of SARS-CoV-2 spike protein prevents viral "
    "entry into host cells via the ACE2 receptor pathway.",
]

q_emb = model.encode(queries,   normalize_embeddings=True)
d_emb = model.encode(documents, normalize_embeddings=True)

scores = model.similarity(q_emb, d_emb)
print(scores)
# tensor([[0.79, 0.51]])
```

### Choosing an embedding dimension

Set `truncate_dim` at load time — no extra downloads, no retraining required:

```python
# Full quality (768d)
model = SentenceTransformer("IstishadAlamTishad/TensorFluxEmbedder")

# Recommended for RAG: 97% quality at 1/3 storage (256d)
model = SentenceTransformer("IstishadAlamTishad/TensorFluxEmbedder", truncate_dim=256)

# Fastest retrieval, ~82% of peak quality (64d)
model = SentenceTransformer("IstishadAlamTishad/TensorFluxEmbedder", truncate_dim=64)
```

## Limitations

- **English only** — trained exclusively on English-language corpora; not expected to perform well on non-English text.
- **Max 8192 tokens** — inputs longer than 8192 tokens will be silently truncated. Split long documents into chunks before encoding.
- **Prefix required** — always apply the correct prefix (`search_query:` / `search_document:`). Omitting it will noticeably degrade retrieval quality.
- **Legal domain scope** — legal evaluation was performed on a single corpus with a 1:1 query-to-passage scheme. Performance on other legal datasets may differ.
- **Retrieval only** — optimized for dense retrieval; not suitable for use as a cross-encoder, classifier, or NLI model.

## Citation

If you use TensorFluxEmbedder in your work, please cite:

```bibtex
@misc{tishad2026tensorfluxembedder,
  author    = {Istishad Alam Tishad},
  title     = {TensorFluxEmbedder: A Multi-Domain Embedding Model},
  year      = {2026},
  url       = {https://huggingface.co/IstishadAlamTishad/TensorFluxEmbedder},
  note      = {HuggingFace model repository}
}
```

## Acknowledgements

**Sentence Transformers**

```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title     = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author    = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month     = "11",
    year      = "2019",
    publisher = "Association for Computational Linguistics",
    url       = "https://arxiv.org/abs/1908.10084",
}
```