Hierarchical E5-Math for exact chunk retrieval - 25/06/2025

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,7 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false
+}

README.md

@@ -0,0 +1,70 @@
+---
+language: 
+- vi
+- en
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- mathematics
+- vietnamese
+- exact-chunk-retrieval
+- hierarchical-learning
+- e5-base
+base_model: intfloat/multilingual-e5-base
+---
+
+# E5-Math-Vietnamese: Hierarchical Exact Chunk Retrieval
+
+## Model Overview
+Fine-tuned E5-base model for **exact chunk retrieval** in Vietnamese mathematics with **hierarchical scoring**:
+- **Correct chunks**: Score ~1.0 (exact answers)
+- **Related chunks**: Score ~0.3 (supplementary info)  
+- **Irrelevant chunks**: Score ~0.0 (unrelated content)
+
+## Training Results
+- **Best Score**: 0.75
+- **Epochs**: 5
+- **Hierarchy Maintained**: True
+- **Task**: Find 1 correct chunk in top 5 results
+
+## Usage
+```python
+from sentence_transformers import SentenceTransformer
+from sklearn.metrics.pairwise import cosine_similarity
+
+# Load model
+model = SentenceTransformer('ThanhLe0125/e5-math')
+
+# ⚠️ CRITICAL: Must use E5 prefixes
+query = "query: Định nghĩa hàm số đồng biến là gì?"
+chunks = [
+    "passage: Hàm số đồng biến trên khoảng (a;b) là...",  # CORRECT
+    "passage: Ví dụ bài tập về hàm đồng biến...",        # RELATED
+    "passage: Phương trình bậc hai có dạng..."           # IRRELEVANT
+]
+
+# Find exact chunk in top 5
+query_emb = model.encode([query])
+chunk_embs = model.encode(chunks)
+similarities = cosine_similarity(query_emb, chunk_embs)[0]
+top5_indices = similarities.argsort()[::-1][:5]
+
+# Get exact answer (should be rank 1)
+exact_answer = chunks[top5_indices[0]]
+print(f"Exact answer: {exact_answer}")
+```
+
+## Key Features
+- ✅ Finds exact answers in top 5 results
+- ✅ Maintains hierarchy: Exact > Related > Irrelevant  
+- ✅ Optimized for Vietnamese mathematics
+- ⚠️ Requires E5 prefixes for optimal performance
+
+## Perfect For
+- Educational Q&A systems
+- Exact answer retrieval
+- Vietnamese math tutoring
+- Content recommendation with hierarchy
+
+*Trained on 25/06/2025 with hierarchical contrastive learning.*

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/intfloat_multilingual-e5-base/",
+  "architectures": [
+    "XLMRobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.32.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 250002
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.32.0",
+    "pytorch": "2.6.0+cu124"
+  }
+}

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f574db303fec96d1c21659f6acde3937d02de3744848f3c0a49bd3cd3023b18
+size 1112241766

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 256,
+  "do_lower_case": false
+}

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
+size 5069051

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:065854c334ebd3edacd11072e41985f97df74967657f2d6e4fb0be122e8d1613
+size 17082913

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "__type": "AddedToken",
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "XLMRobertaTokenizer",
+  "unk_token": "<unk>"
+}

usage_example.py

@@ -0,0 +1,28 @@
+
+from sentence_transformers import SentenceTransformer
+from sklearn.metrics.pairwise import cosine_similarity
+
+# Load hierarchical model
+model = SentenceTransformer('ThanhLe0125/e5-math')
+
+# Example: Find exact chunk among similar options
+query = "query: Định nghĩa hàm số đồng biến"
+chunks = [
+    "passage: Hàm số đồng biến trên khoảng (a;b) là hàm số mà với mọi x1 < x2 thì f(x1) < f(x2)",
+    "passage: Ví dụ: Tìm khoảng đồng biến của hàm số y = x^2 - 2x + 1", 
+    "passage: Phương trình bậc hai ax^2 + bx + c = 0"
+]
+
+# Encode and rank
+query_emb = model.encode([query])
+chunk_embs = model.encode(chunks)
+similarities = cosine_similarity(query_emb, chunk_embs)[0]
+
+# Get top 5 results
+top5_indices = similarities.argsort()[::-1][:5]
+for rank, idx in enumerate(top5_indices, 1):
+    print(f"{rank}. Score: {similarities[idx]:.4f} - {chunks[idx][:50]}...")
+
+# Expected: Exact definition gets highest score (~1.0)
+# Related example gets medium score (~0.3)
+# Irrelevant content gets low score (~0.0)