Initial publish via td-embeddings

README.md

@@ -0,0 +1,292 @@
+---
+license: apache-2.0
+language:
+  - ja
+library_name: sentence-transformers
+pipeline_tag: feature-extraction
+base_model: cl-nagoya/ruri-v3-70m
+tags:
+  - onnx
+  - teradata
+  - byom
+  - embeddings
+  - feature-extraction
+  - modernbert
+  - sentence-transformers
+---
+
+
+
+> Read the disclaimer below before using this model.
+
+----
+
+# ruri-v3-70m -- ONNX for Teradata BYOM
+
+This repository hosts an **ONNX-converted** version of the upstream
+model [`cl-nagoya/ruri-v3-70m`](https://huggingface.co/cl-nagoya/ruri-v3-70m),
+packaged for the Teradata Vantage `mldb.ONNXEmbeddings` BYOM
+function. It is **not** the original PyTorch model -- only the
+inference graph and tokenizer needed for in-database embedding
+generation.
+
+What's different from upstream:
+
+- **Format**: ONNX (opset 14, IR version 8 -- BYOM 6+ compatible),
+  produced from the upstream weights with architecture-aware
+  post-processing baked in.
+- **Precision**: dynamic int8 quantization. See the variants table
+  below for what is shipped for this model.
+- **Pooling and post-processing**: this graph emits the raw
+  `sentence_embedding` tensor. Pooling rule is
+  **mean** and the model expects
+  a query-time instruction prefix (see "Instruction prefix" below).
+- **Verification**: every variant's cosine fidelity vs. the
+  upstream PyTorch reference is recorded on a fixed
+  JMTEB retrieval (2 of 8 subsets: jagovfaqs_22k, nlp_journal_title_abs) sample. Numbers may not generalize
+  to your data.
+
+## Model details
+
+| | |
+|---|---|
+| Upstream repo | [`cl-nagoya/ruri-v3-70m`](https://huggingface.co/cl-nagoya/ruri-v3-70m) |
+| Architecture | `ModernBertModel` (encoder) |
+| Parameters | 70,002,816 |
+| Output dimensions | 384 |
+| Pooling | `mean` |
+| Instruction prefix | yes |
+| Max input tokens (advertised) | 8192 |
+| Languages | 1 |
+| License | apache-2.0 |
+| ONNX opset | 14 |
+| ONNX IR version | 8 (BYOM 6+ compatible) |
+
+<details>
+<summary>Full language list (1)</summary>
+
+- `ja`
+
+</details>
+
+### Instruction prefix
+
+This model was trained with a **role-specific prefix scheme**: the text
+you encode must be prefixed according to what you are using the
+embedding *for*. The prefix wording is not customisable -- the model
+only understands these specific strings, and the trailing `: ` (ASCII
+colon + ASCII space) is part of the prefix.
+
+| Prefix | Use it for |
+|---|---|
+| (empty string) | semantic similarity and general-purpose encoding |
+| `トピック: ` | classification, clustering, and topical encoding |
+| `検索クエリ: ` | retrieval -- the **query** side |
+| `検索文書: ` | retrieval -- the **document** side |
+
+**The two sides of a retrieval pair take *different* prefixes.** This
+is the part most easily got wrong: prefixing both sides with the same
+string, or prefixing only one side, degrades retrieval quality without
+producing any error. Prefix your queries with
+`検索クエリ: ` and the passages you index
+with `検索文書: `.
+
+The ONNX graph itself is prefix-agnostic -- the prefix is plain text
+that flows through the tokenizer. Downstream BYOM SQL is responsible
+for prepending it, typically with a CTE that concatenates the prefix
+onto each input row.
+
+Worked example -- the same subject encoded on each side of a retrieval
+pair:
+
+```
+query:    検索クエリ: 瑠璃色はどんな色？
+document: 検索文書: 瑠璃色（るりいろ）は、紫みを帯びた濃い青。
+```
+
+Example SQL (queries and documents prefixed separately, then embedded
+through the same model):
+
+```sql
+WITH prefixed_queries AS (
+  SELECT id,
+         '検索クエリ: ' || query_text AS txt
+  FROM   my_query_table
+),
+prefixed_documents AS (
+  SELECT id,
+         '検索文書: ' || body_text AS txt
+  FROM   my_document_table
+)
+SELECT *
+FROM   mldb.ONNXEmbeddings(
+         ON prefixed_queries
+         ON embeddings_models     AS ModelTable DIMENSION
+         ON embeddings_tokenizers AS TokenizerTable DIMENSION
+         USING
+           Accumulate('id')
+           ModelOutputTensor('sentence_embedding')
+       ) AS s;
+```
+
+See
+[`cl-nagoya/ruri-v3-70m`](https://huggingface.co/cl-nagoya/ruri-v3-70m) for the
+canonical guidance.
+
+## Quantization variants
+
+This repository ships the following variants. The **Size** column is the
+on-disk size of the ONNX weight file in megabytes (MB, 10^6 bytes).
+Quality numbers may not generalize to your data; they come from a fixed
+sample of JMTEB retrieval (2 of 8 subsets: jagovfaqs_22k, nlp_journal_title_abs).
+
+Those numbers were measured with inputs tokenized at a maximum sequence
+length of **512 tokens**.
+
+| Variant | Size (MB) | p50 cosine | R@1 | Δ R@1 vs fp32 |
+|---|---|---|---|---|
+| `fp32` | 280.3 | 1.000000 | 0.927 | — |
+| `per_channel` | 72.7 | 0.952567 | 0.891 | -0.036 |
+| `ffn_skip` | 139.6 | 0.999625 | 0.929 | +0.002 |
+
+
+How to read the quality columns:
+
+- **p50 cosine** is the median cosine similarity between this
+  variant's embeddings and the fp32 ONNX reference, computed over
+  a fixed evaluation set. Higher means closer to the unquantized
+  model; **1.0** is identical. On the `fp32` row the comparison is
+  against the upstream PyTorch model instead, so that row measures
+  export drift rather than quantization drift.
+- **R@1** is **absolute top-1 retrieval accuracy** on that evaluation
+  sample: each query is ranked against the whole document pool, and R@1
+  is the fraction of queries whose own canonical document comes back
+  first. Higher is better. It is *not* a comparison against fp32 --
+  every row is measured exactly the same way.
+- **Δ R@1 vs fp32** is that row's R@1 minus the `fp32` row's, on the same
+  corpus at the same sequence length. Because every row is measured the
+  same way, the `fp32` row is this model's unquantized ceiling, and this
+  column is the retrieval quality actually given up to quantization.
+  Read it when choosing between variants: two artifacts can post similar
+  absolute R@1 while one has given up several times as much against its
+  own ceiling, and only the delta shows that.
+
+Notes:
+- **fp32**: full-precision reference. Useful for an accuracy ceiling,
+  but BYOM users almost always want one of the int8 variants for
+  in-database scoring -- they are 3-4x smaller and load much faster.
+- **per_channel**: dynamic int8 with weights quantized **per
+  output channel**. Each output channel keeps its own scale, so
+  layer-wide outliers don't blow up the quantization range. The
+  artifact is roughly **4x smaller than fp32** and is the right
+  default when storage, memory, or load time matters more than the
+  last percent of retrieval quality.
+- **ffn_skip**: dynamic int8 with the feed-forward (FFN) MatMul
+  layers kept in **fp32**, while attention and projection MatMuls
+  stay quantized. The FFN layers are where most of the quantization
+  error in transformer blocks concentrates; leaving them in fp32
+  recovers most of the quality loss for a modest size increase.
+  The artifact is roughly **3x smaller than fp32** (larger than the
+  per_channel int8 sibling).
+
+## Quickstart: using this model with Teradata BYOM
+
+Requires Teradata Vantage with **BYOM 6+** (`mldb.ONNXEmbeddings`).
+
+```python
+import getpass
+import teradataml as tdml
+from huggingface_hub import hf_hub_download
+
+repo_id   = "Teradata/ruri-v3-70m"
+model_id  = "ruri-v3-70m"        # arbitrary, used as the BYOM model_id
+onnx_file = "onnx/model-per_channel.onnx"
+
+# 1. Download the ONNX + tokenizer for the chosen variant.
+hf_hub_download(repo_id=repo_id, filename=onnx_file,       local_dir="./")
+hf_hub_download(repo_id=repo_id, filename="tokenizer.json", local_dir="./")
+
+# 2. Connect to Vantage.
+tdml.create_context(
+    host=input("host: "),
+    username=input("user: "),
+    password=getpass.getpass("password: "),
+)
+
+# 3. Load model + tokenizer into BYOM tables (one-time per model_id).
+tdml.save_byom(model_id=model_id, model_file=onnx_file,
+               table_name="embeddings_models")
+tdml.save_byom(model_id=model_id, model_file="tokenizer.json",
+               table_name="embeddings_tokenizers")
+```
+
+Then call `mldb.ONNXEmbeddings` against an input table whose
+`txt` column carries the strings to embed:
+
+```sql
+SELECT *
+FROM mldb.ONNXEmbeddings(
+    ON (SELECT id, txt FROM your_input_table) AS InputTable
+    ON (SELECT model_id, model FROM embeddings_models
+         WHERE model_id = 'ruri-v3-70m') AS ModelTable DIMENSION
+    ON (SELECT model_id, tokenizer FROM embeddings_tokenizers
+         WHERE model_id = 'ruri-v3-70m') AS TokenizerTable DIMENSION
+    USING
+        Accumulate('id')
+        ModelOutputTensor('sentence_embedding')
+        OutputFormat('FLOAT32(384)')
+        OverwriteCachedModel('*')
+) AS t
+ORDER BY id;
+```
+
+Pooling rule **`mean`** is applied **inside** the converted
+ONNX graph -- the output tensor named above already contains the
+pooled, post-processed embedding vector. For instruction-prefix models, prepend
+the recommended instruction text to each input `txt` before calling
+`ONNXEmbeddings`; the prefix is plain text that the tokenizer handles
+unchanged.
+
+## Original model attribution
+
+The original weights and training methodology belong to
+**the Ruri authors at Nagoya University**. Please cite their work, not this
+repository, in academic contexts. The canonical upstream model card
+is at
+[`cl-nagoya/ruri-v3-70m`](https://huggingface.co/cl-nagoya/ruri-v3-70m);
+refer to it for benchmarks, training details, intended use, and
+citation information.
+
+`cl-nagoya/ruri-v3-70m` is itself derived from earlier models, and **not every
+model in that chain is under the same license as this one**. The
+`license` field in this repository's metadata can only carry a single
+value (apache-2.0 -- the license of `cl-nagoya/ruri-v3-70m`), so the
+full chain is stated here:
+
+| Upstream model | License | Relationship |
+|---|---|---|
+| [`cl-nagoya/ruri-v3-70m`](https://huggingface.co/cl-nagoya/ruri-v3-70m) | apache-2.0 | the model converted here |
+| [`cl-nagoya/ruri-v3-pt-70m`](https://huggingface.co/cl-nagoya/ruri-v3-pt-70m) | apache-2.0 | direct base (Ruri v3 pretrained backbone) |
+| [`sbintuitions/modernbert-ja-70m`](https://huggingface.co/sbintuitions/modernbert-ja-70m) | mit | ModernBERT-Ja base of the pretrained backbone |
+
+Your use of this artifact is subject to the terms of **every** license
+in that chain. Check each upstream model card for its authoritative
+license text.
+
+## Reporting issues
+
+For ONNX-conversion or BYOM-compatibility issues specific to this
+Teradata-converted artifact, please open a **Discussion** on this
+model's Hugging Face page. Questions about the underlying model
+quality, training, or intended use should go to the upstream
+maintainer's model card.
+
+----
+
+DISCLAIMER: The content herein ("Content") is provided "AS IS" and is not covered by any Teradata Operations, Inc. and its affiliates ("Teradata") agreements. Its listing here does not constitute certification or endorsement by Teradata.
+
+To the extent any of the Content contains or is related to any artificial intelligence ("AI") or other language learning models ("Models") that interoperate with the products and services of Teradata, by accessing, bringing, deploying or using such Models, you acknowledge and agree that you are solely responsible for ensuring compliance with all applicable laws, regulations, and restrictions governing the use, deployment, and distribution of AI technologies. This includes, but is not limited to, AI Diffusion Rules, European Union AI Act, AI-related laws and regulations, privacy laws, export controls, and financial or sector-specific regulations.
+
+While Teradata may provide support, guidance, or assistance in the deployment or implementation of Models to interoperate with Teradata's products and/or services, you remain fully responsible for ensuring that your Models, data, and applications comply with all relevant legal and regulatory obligations. Our assistance does not constitute legal or regulatory approval, and Teradata disclaims any liability arising from non-compliance with applicable laws.
+
+You must determine the suitability of the Models for any purpose. Given the probabilistic nature of machine learning and modeling, the use of the Models may in some situations result in incorrect output that does not accurately reflect the action generated. You should evaluate the accuracy of any output as appropriate for your use case, including by using human review of the output.

config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "outputs/ft/vanilla/cl-nagoya__ruri-v3-pt-70m/B512E1LR1e-05L8192Temp0.01/default14",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "cls",
+  "cls_token_id": 6,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 2,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 384,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 6,
+  "num_hidden_layers": 13,
+  "pad_token_id": 3,
+  "position_embedding_type": "rope",
+  "reference_compile": false,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 4,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.3",
+  "vocab_size": 102400
+}

onnx/model-ffn_skip.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdd04431dd716f720df04d0b31aed4c39a6558d610fb4f6eaabe4706f9217bf4
+size 139644978

onnx/model-fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4219242d44c62ec855056496b7806a570bede1512faf6c4779861e2e3baba91c
+size 280334146

onnx/model-per_channel.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bad7aa60569b87adb359be39786606e891d81205b26ef59c3649c125acb447d3
+size 72656615

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<cls>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "<sep>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,171 @@
+{
+  "add_bos_token": true,
+  "add_dummy_prefix_space": false,
+  "add_eos_token": true,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<cls>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<|available_tools|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<|tool_calls|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<|tool_results|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<|code|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<|file|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102397": {
+      "content": "<|prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102398": {
+      "content": "<|suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102399": {
+      "content": "<|middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<cls>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "extra_ids": 0,
+  "extra_special_tokens": {},
+  "keep_accents": true,
+  "legacy": false,
+  "mask_token": "<mask>",
+  "model_max_length": 8192,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sep_token": "<sep>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}