---
license: apache-2.0
pipeline_tag: text-ranking
tags:
  - cross-encoder
  - reranker
  - onnx
  - teradata
base_model: jinaai/jina-reranker-v1-tiny-en
---

> ⚠️ **See [Disclaimer](#disclaimer) below before using.**

# jina-reranker-v1-tiny-en

## A Teradata Vantage compatible Reranker Model

An ONNX-converted, BYOM-ready copy of
[jinaai/jina-reranker-v1-tiny-en](https://huggingface.co/jinaai/jina-reranker-v1-tiny-en): given a
`(query, passage)` pair, it outputs a single relevance score for that pair.

## Reranking in two-stage retrieval

Retrieval pipelines typically run in two stages. The first stage -- a vector search over an
index such as HNSW, or classical keyword search -- scans the whole corpus and returns a
shortlist (say, the top 20-100 candidates). It's fast because it embeds the query and each
document independently and only ever compares the resulting vectors; the two texts never
interact. A reranker is a cross-encoder: it reads the query and one candidate together, token by
token, and produces a single relevance score for that pair. That joint attention makes it far
more accurate than the first-stage vector search, but also too expensive to run over an entire
corpus. The standard pattern is therefore to let cheap first-stage retrieval narrow things down,
then use a reranker like this one to re-score and re-order just that shortlist.

## Overview of this Model

- 33.0M params (ONNX fp32: 128MB)
- JinaBERT architecture (BERT with ALiBi position encoding, 8192-token native context; this
  repo's scripts truncate at 512 tokens, matching the rest of this collection)
- Output: a single logit, monotonically related to relevance (higher = more relevant)
- License: apache-2.0. The released model can be used for commercial purposes free of charge.
- Reference to Original Model: https://huggingface.co/jinaai/jina-reranker-v1-tiny-en

## Quickstart: Deploying this Model in Teradata Vantage

```python
import getpass
import teradataml as tdml
from huggingface_hub import hf_hub_download

repo_id  = "martinhillebrandtd/jina-reranker-v1-tiny-en"
model_id = "jina-reranker-v1-tiny-en"

# 1. Download artifacts from this repo
hf_hub_download(repo_id=repo_id, filename="onnx/model.onnx", local_dir="./")
hf_hub_download(repo_id=repo_id, filename="tokenizer.json",  local_dir="./")

# 2. Connect to Teradata
tdml.create_context(host=input("host: "), username=input("user: "), password=getpass.getpass("password: "))

# 3. Load model + tokenizer into BYOM tables
tdml.save_byom(model_id=model_id, model_file="onnx/model.onnx", table_name="reranker_models")
tdml.save_byom(model_id=model_id, model_file="tokenizer.json",  table_name="reranker_tokenizers")

# 4. Score an existing chunks table against one fixed query.
#    This model's own pair-separator is </s></s> (see "How it works" below for why concatenation
#    is a valid substitute for real pair-encoding, for this specific model).
chunks_table = "your_chunks_table"  # columns: chunk_id, chunk_txt
QUERY = "What is the best-known landmark in Paris?"

query = f"""
SELECT chunk_id, chunk_txt, emb_0 AS relevance_score
FROM mldb.ONNXEmbeddings(
    ON (SELECT chunk_id, chunk_txt, '{QUERY}' || '</s></s>' || chunk_txt AS txt
         FROM {chunks_table}) AS InputTable
    ON (SELECT * FROM reranker_models WHERE model_id = '{model_id}') AS ModelTable DIMENSION
    ON (SELECT model AS tokenizer FROM reranker_tokenizers WHERE model_id = '{model_id}') AS TokenizerTable DIMENSION
    USING
        Accumulate('chunk_id', 'chunk_txt')
        ModelOutputTensor('sentence_embedding')
        OutputFormat('FLOAT32(1)')
        OverwriteCachedModel('true')
) a
"""
result = tdml.DataFrame.from_query(query).to_pandas().sort_values("relevance_score", ascending=False)
```

`relevance_score` is the model's raw logit -- sorting by it directly already gives the correct
ranking, since `sigmoid` is monotonic. If you need an actual `[0, 1]` probability rather than
just a ranking, compute it in SQL instead of Python: `1 / (1 + EXP(-emb_0))`.

## Combining with vector search for two-stage retrieval

A typical pipeline: embed your corpus once with an embedding model, then at query time use
`VectorDistance` to get a shortlist of candidates by cosine distance, and rerank only that
shortlist with this model.

```sql
-- Stage 1: shortlist the 20 nearest chunks to the query embedding
CREATE VOLATILE TABLE shortlist AS (
    SELECT ref_id AS chunk_id, distance
    FROM TD_VECTORDISTANCE(
        ON query_embedding AS TargetTable
        ON chunk_embeddings AS ReferenceTable DIMENSION
        USING
            TargetIDColumn('query_id')
            TargetFeatureColumns('[emb_0:emb_383]')
            RefIDColumn('chunk_id')
            RefFeatureColumns('[emb_0:emb_383]')
            DistanceMeasure('cosine')
            TopK(20)
    ) AS dt
) WITH DATA ON COMMIT PRESERVE ROWS;

-- Stage 2: rerank only those 20 shortlisted chunks (see Quickstart above, with
-- your_chunks_table replaced by a join against `shortlist`)
```

## How it works

`ONNXEmbeddings` is built to map one text column to a fixed-size float vector -- it has no
native concept of a `(query, passage)` pair, and doesn't care that its output is semantically an
"embedding": any graph with the right tensor names/shapes runs. This repo reuses it as a generic
text-to-number function instead: query and passage are concatenated into a single string before
tokenization, and the model's one classification logit comes back through the (1-value)
`sentence_embedding` output.

This is only safe because this model's exported graph never uses `token_type_ids` to mark the
query/passage boundary -- confirmed by testing the concatenated score against the model's true
pair-encoded score (`AutoModelForSequenceClassification`, real two-argument tokenizer call), both
locally and against a live Vantage instance:

| | mean \|delta\| | max \|delta\| |
|---|---|---|
| Local ONNX (concat) vs. true PyTorch (pair) | 3.5e-7 | 6.6e-7 |
| Live Teradata `ONNXEmbeddings` vs. true PyTorch (pair) | 4.0e-7 | 7.9e-7 |

If you plan to convert a *different* reranker with this same recipe, check first whether its
exported graph actually declares/uses `token_type_ids` as an input -- if it does, this concat
trick is not valid for that model.

## How this model was converted

`jinaai/jina-reranker-v1-tiny-en` already ships a ready-made ONNX export
(`onnx/model.onnx`, fp32, opset 11) -- no re-export via `optimum` was needed. The only change
required was renaming the graph's native output tensor (`logits`) to `sentence_embedding`, since
`ONNXEmbeddings`'s `model_output_tensor` argument only accepts the literal values
`sentence_embedding` or `token_embeddings`. See [convert.py](./convert.py) for the conversion
steps, and [test_local.py](./test_local.py) for the ONNX-vs-PyTorch parity check above.

-----

## Disclaimer

DISCLAIMER: The content herein ("Content") is provided "AS IS" and is not covered by any Teradata Operations, Inc. and its affiliates ("Teradata") agreements. Its listing here does not constitute certification or endorsement by Teradata.

To the extent any of the Content contains or is related to any artificial intelligence ("AI") or other language learning models ("Models") that interoperate with the products and services of Teradata, by accessing, bringing, deploying or using such Models, you acknowledge and agree that you are solely responsible for ensuring compliance with all applicable laws, regulations, and restrictions governing the use, deployment, and distribution of AI technologies. This includes, but is not limited to, AI Diffusion Rules, European Union AI Act, AI-related laws and regulations, privacy laws, export controls, and financial or sector-specific regulations.

While Teradata may provide support, guidance, or assistance in the deployment or implementation of Models to interoperate with Teradata's products and/or services, you remain fully responsible for ensuring that your Models, data, and applications comply with all relevant legal and regulatory obligations. Our assistance does not constitute legal or regulatory approval, and Teradata disclaims any liability arising from non-compliance with applicable laws.

You must determine the suitability of the Models for any purpose. Given the probabilistic nature of machine learning and modeling, the use of the Models may in some situations result in incorrect output that does not accurately reflect the action generated. You should evaluate the accuracy of any output as appropriate for your use case, including by using human review of the output.