Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text
+colbert_linear.pt filter=lfs diff=lfs merge=lfs -text
+sentencepiece.bpe.model filter=lfs diff=lfs merge=lfs -text
+training_args.bin filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,220 @@
+
+---
+datasets:
+- GreenNode/GreenNode-Table-Markdown-Retrieval
+language:
+- vi
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+widget: []
+metrics:
+- InfoNCE
+license: cc-by-4.0
+---
+
+# SentenceTransformer
+
+This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 1024 tokens
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:** - GreenNode/GreenNode-Table-Markdown-Retrieval 
+- **Language:** Vietnamese
+- **License:** cc-by-4.0
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'The weather is lovely today.',
+    "It's so sunny outside!",
+    'He drove to the stadium.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 1024]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+## Evaluation
+### Table: Performance comparison of various models on GreenNodeTableRetrieval
+Dataset:  [GreenNode/GreenNode-Table-Markdown-Retrieval](https://huggingface.co/datasets/GreenNode/GreenNode-Table-Markdown-Retrieval-VN)
+
+| Model Name                                  | MAP@5 ↑ | MRR@5 ↑ | NDCG@5 ↑ | Recall@5 ↑ | Mean ↑ |
+|--------------------------------------------|--------:|--------:|---------:|-----------:|-------:|
+| **Multilingual Embedding models**          |         |         |          |            |        |
+| me5_small                                   | 33.75   | 33.75   | 35.68    | 41.49      | 36.17  |
+| me5_large                                   | 38.16   | 38.16   | 40.27    | 46.62      | 40.80  |
+| M3-Embedding                                | 36.52   | 36.52   | 38.60    | 44.84      | 39.12  |
+| OpenAI-embedding-v3                         | 30.61   | 30.61   | 32.57    | 38.46      | 33.06  |
+| **Vietnamese Embedding models (Prior Work)**|         |         |          |            |        |
+| halong-embedding                            | 32.15   | 32.15   | 34.13    | 40.09      | 34.63  |
+| sup-SimCSE-VietNamese-phobert_base          | 10.90   | 10.90   | 12.03    | 15.41      | 12.31  |
+| vietnamese-bi-encoder                       | 13.61   | 13.61   | 14.63    | 17.68      | 14.89  |
+| **GreenNode-Embedding (Our Work)**          |         |         |          |            |        |
+| *M3-GN-VN*                                   | _41.85_ | _41.85_ | _44.15_  | _57.05_    | _46.23_ |
+| **M3-GN-VN-Mixed**                           | **42.08** | **42.08** | **44.33** | **51.06** | **44.89** |
+### Table: Performance comparison of various models on ZacLegalTextRetrieval
+Dataset:  [GreenNode/zalo-ai-legal-text-retrieval-vn](https://huggingface.co/datasets/GreenNode/zalo-ai-legal-text-retrieval-vn)
+
+| Model Name                                  | MAP@5 ↑ | MRR@5 ↑ | NDCG@5 ↑ | Recall@5 ↑ | Mean ↑ |
+|--------------------------------------------|--------:|--------:|---------:|-----------:|-------:|
+| **Multilingual Embedding models**          |         |         |          |            |        |
+| me5_small                                   | 54.68   | 54.37   | 58.32    | 69.16      | 59.13  |
+| me5_large                                   | 60.14   | 59.62   | 64.17    | 76.02      | 64.99  |
+| *M3-Embedding*                              | _69.34_ | _68.96_ | _73.70_  | _86.68_    | _74.67_ |
+| OpenAI-embedding-v3                         | 38.68   | 38.80   | 41.53    | 49.94      | 41.74  |
+| **Vietnamese Embedding models (Prior Work)**|         |         |          |            |        |
+| halong-embedding                            | 52.57   | 52.28   | 56.64    | 68.72      | 57.55  |
+| sup-SimCSE-VietNamese-phobert_base          | 25.15   | 25.07   | 27.81    | 35.79      | 28.46  |
+| vietnamese-bi-encoder                       | 54.88   | 54.47   | 59.10    | 79.51      | 61.99  |
+| **GreenNode-Embedding (Our Work)**          |         |         |          |            |        |
+| M3-GN-VN                                     | 65.03   | 64.80   | 69.19    | 81.66      | 70.17  |
+| **M3-GN-VN-Mixed**                           | **69.75** | **69.28** | **74.01** | **86.74** | **74.95** |
+### Table: Performance comparison of various models on VieQuADRetrieval
+Dataset: [taidng/UIT-ViQuAD2.0](https://huggingface.co/datasets/taidng/UIT-ViQuAD2.0)
+
+| Model Name                                  | MAP@5 ↑ | MRR@5 ↑ | NDCG@5 ↑ | Recall@5 ↑ | Mean ↑ |
+|--------------------------------------------|--------:|--------:|---------:|-----------:|-------:|
+| **Multilingual Embedding models**          |         |         |          |            |        |
+| me5_small                                   | 40.42   | 69.21   | 50.05    | 50.71      | 52.60  |
+| me5_large                                   | 44.18   | 67.81   | 53.04    | 55.86      | 55.22  |
+| *M3-Embedding*                              | _44.08_ | _72.28_ | _54.07_  | _56.01_    | _56.61_ |
+| OpenAI-embedding-v3                         | 32.39   | 53.97   | 40.48    | 43.02      | 42.47  |
+| **Vietnamese Embedding models (Prior Work)**|         |         |          |            |        |
+| halong-embedding                            | 39.42   | 62.31   | 48.63    | 52.73      | 50.77  |
+| sup-SimCSE-VietNamese-phobert_base          | 20.45   | 35.99   | 26.73    | 29.59      | 28.19  |
+| vietnamese-bi-encoder                       | 31.89   | 54.62   | 40.26    | 42.53      | 42.33  |
+| **GreenNode-Embedding (Our Work)**          |         |         |          |            |        |
+| M3-GN-VN                                     | 42.85   | 71.98   | 52.90    | 54.25      | 55.50  |
+| **M3-GN-VN-Mixed**                           | **44.20** | **72.64** | **54.30** | **56.30** | **56.86** |
+
+### Table: Performance comparison of various models on GreenNodeTableRetrieval (Hit Rate)
+
+| Model Name                                     | Hit Rate@1 ↑ | Hit Rate@5 ↑ | Hit Rate@10 ↑ | Hit Rate@20 ↑ |
+|------------------------------------------------|--------------|--------------|---------------|---------------|
+| **Multilingual Embedding models**              |              |              |               |               |
+| me5_small                                      | 38.99        | 53.37        | 59.28         | 65.09         |
+| me5_large                                      | 43.99        | 59.74        | 65.74         | 71.59         |
+| bge-m3                                         | 42.15        | 57.00        | 63.05         | 68.96         |
+| OpenAI-embedding-v3                            | -            | -            | -             | -             |
+| **Vietnamese Embedding models (Prior Work)**   |              |              |               |               |
+| halong-embedding                               | 37.22        | 52.49        | 58.57         | 64.64         |
+| sup-SimCSE-VietNamese-phobert_base             | 14.00        | 24.74        | 30.32         | 36.44         |
+| vietnamese-bi-encoder                          | 16.89        | 25.94        | 30.50         | 35.70         |
+| **GreenNode-Embedding (Our Work)**             |              |              |               |               |
+| **M3-GN-VN**                                    | **48.31**    | **64.60**    | **70.83**     | **76.46**     |
+| *M3-GN-VN-Mixed*                               | _47.94_      | _64.24_      | _70.43_       | _76.14_       |
+
+
+### Framework Versions
+- Python: 3.10.14
+- Sentence Transformers: 3.0.1
+- Transformers: 4.42.4
+- PyTorch: 2.3.1
+- Accelerate: 0.33.0
+- Datasets: 2.20.0
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "GreenNode/GreenNode-Embedding-Large-VN-Mixed-V1",
+  "architectures": [
+    "XLMRobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 8194,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.42.4",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 250002
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.42.4",
+    "pytorch": "2.3.1"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fc784e75e3d2a5e66751bfd59863638976149a9ba0fe7b9439df21ac5ad4799
+size 2271064456

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

sentencepiece.bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
+size 5069051

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:249df0778f236f6ece390de0de746838ef25b9d6954b68c2ee71249e0a9d8fd4
+size 17082799

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "250001": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 8192,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "XLMRobertaTokenizer",
+  "unk_token": "<unk>"
+}