Add new SentenceTransformer model

README.md

@@ -7,63 +7,64 @@ tags:
 - sentence-similarity
 - feature-extraction
 - generated_from_trainer
-- dataset_size:44
+- dataset_size:46
 - loss:MatryoshkaLoss
 - loss:MultipleNegativesRankingLoss
 base_model: nomic-ai/modernbert-embed-base
 widget:
-- source_sentence: Which technologies are mentioned in the title of the text?
+- source_sentence: What two factors contribute to the power of LLMs?
   sentences:
-  - Introduction to AI, Machine Learning, LLMs, and Their Integration
-  - Introduction to AI, Machine Learning, LLMs, and Their Integration
   - Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific
     data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG
     systems, the model first retrieves relevant documents from a database (like a
     knowledge base), then generates a response using that context—significantly improving
     the relevance and accuracy of the answers.
-- source_sentence: What is the ability of machines to understand and generate human
-    language called?
-  sentences:
+  - LLMs work by learning statistical relationships between words and phrases, allowing
+    them to predict and generate language that feels natural. The power of these models
+    lies not only in their size but also in the diversity of tasks they can perform
+    with little to no task-specific training
   - . For example, integrating an LLM into a customer support chatbot might involve
     connecting it to a company’s internal knowledge base, enabling it to answer customer
     questions using accurate, up-to-date information.
-  - Over the past few years, the field of ML has advanced rapidly, especially in the
-    area of Natural Language Processing (NLP)—the ability of machines to understand
-    and generate human language. At the forefront of this progress are Large Language
-    Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s
-    PaLM, and Meta’s LLaMA
-  - A major subset of AI is Machine Learning (ML), which involves algorithms that
-    learn from data rather than being explicitly programmed. Instead of writing detailed
-    instructions for every task, ML models find patterns in large datasets and use
-    these patterns to make predictions or decisions
-- source_sentence: What is the purpose of embedding LLMs into systems?
+- source_sentence: What is one method mentioned for fine-tuning the LLM?
   sentences:
+  - Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific
+    data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG
+    systems, the model first retrieves relevant documents from a database (like a
+    knowledge base), then generates a response using that context—significantly improving
+    the relevance and accuracy of the answers.
   - However, deploying LLMs effectively in real-world applications often requires
     LLM integration. This means embedding these models into systems, workflows, or
     products where they can interact with other components like databases, APIs, user
     interfaces, or even custom business logic
-  - Over the past few years, the field of ML has advanced rapidly, especially in the
-    area of Natural Language Processing (NLP)—the ability of machines to understand
-    and generate human language. At the forefront of this progress are Large Language
-    Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s
-    PaLM, and Meta’s LLaMA
+  - . As organizations increasingly adopt these technologies, the ability to understand
+    and apply LLMs will be a critical skill in the AI-powered future.
+- source_sentence: What are some tasks that AI is capable of performing?
+  sentences:
+  - Artificial Intelligence (AI) is the broad field of computer science that focuses
+    on building systems capable of performing tasks that normally require human intelligence.
+    These tasks include learning from experience, understanding language, recognizing
+    patterns, and making decisions. AI powers everything from smart assistants like
+    Siri to recommendation systems on Netflix and self-driving cars.
+  - In summary, AI and ML form the foundation for intelligent automation, while LLMs
+    represent a breakthrough in language understanding and generation. Integrating
+    these models into real-world systems unlocks practical value, turning raw intelligence
+    into tangible solutions
   - Introduction to AI, Machine Learning, LLMs, and Their Integration
-- source_sentence: What do ML algorithms learn from?
+- source_sentence: What is the abbreviation for Large Language Models as mentioned
+    in the text?
   sentences:
-  - A major subset of AI is Machine Learning (ML), which involves algorithms that
-    learn from data rather than being explicitly programmed. Instead of writing detailed
-    instructions for every task, ML models find patterns in large datasets and use
-    these patterns to make predictions or decisions
   - LLMs work by learning statistical relationships between words and phrases, allowing
     them to predict and generate language that feels natural. The power of these models
     lies not only in their size but also in the diversity of tasks they can perform
     with little to no task-specific training
-  - Over the past few years, the field of ML has advanced rapidly, especially in the
-    area of Natural Language Processing (NLP)—the ability of machines to understand
-    and generate human language. At the forefront of this progress are Large Language
-    Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s
-    PaLM, and Meta’s LLaMA
-- source_sentence: What is required for effectively deploying LLMs in real-world applications?
+  - LLMs work by learning statistical relationships between words and phrases, allowing
+    them to predict and generate language that feels natural. The power of these models
+    lies not only in their size but also in the diversity of tasks they can perform
+    with little to no task-specific training
+  - . As organizations increasingly adopt these technologies, the ability to understand
+    and apply LLMs will be a critical skill in the AI-powered future.
+- source_sentence: What does the use of RAG systems improve according to the text?
   sentences:
   - . For instance, a spam filter doesn’t just block emails with specific keywords—it
     learns from thousands of examples what spam typically looks like.
@@ -72,10 +73,11 @@ widget:
     systems, the model first retrieves relevant documents from a database (like a
     knowledge base), then generates a response using that context—significantly improving
     the relevance and accuracy of the answers.
-  - However, deploying LLMs effectively in real-world applications often requires
-    LLM integration. This means embedding these models into systems, workflows, or
-    products where they can interact with other components like databases, APIs, user
-    interfaces, or even custom business logic
+  - Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific
+    data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG
+    systems, the model first retrieves relevant documents from a database (like a
+    knowledge base), then generates a response using that context—significantly improving
+    the relevance and accuracy of the answers.
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 metrics:
@@ -105,10 +107,10 @@ model-index:
       type: dim_768
     metrics:
     - type: cosine_accuracy@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
       value: 1.0
@@ -117,22 +119,22 @@ model-index:
       value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.3333333333333333
+      value: 0.27777777777777773
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.2
+      value: 0.19999999999999998
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.1
+      value: 0.09999999999999999
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Recall@3
     - type: cosine_recall@5
       value: 1.0
@@ -141,13 +143,13 @@ model-index:
       value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 1.0
+      value: 0.8436010519408085
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 1.0
+      value: 0.7916666666666666
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 1.0
+      value: 0.7916666666666666
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -157,7 +159,7 @@ model-index:
       type: dim_512
     metrics:
     - type: cosine_accuracy@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
       value: 1.0
@@ -169,19 +171,19 @@ model-index:
       value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Precision@1
     - type: cosine_precision@3
       value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.2
+      value: 0.19999999999999998
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.1
+      value: 0.09999999999999999
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Recall@1
     - type: cosine_recall@3
       value: 1.0
@@ -193,13 +195,13 @@ model-index:
       value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 1.0
+      value: 0.8769765845238192
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -209,7 +211,7 @@ model-index:
       type: dim_256
     metrics:
     - type: cosine_accuracy@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
       value: 1.0
@@ -221,19 +223,19 @@ model-index:
       value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Precision@1
     - type: cosine_precision@3
       value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.2
+      value: 0.19999999999999998
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.1
+      value: 0.09999999999999999
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 1.0
+      value: 0.6666666666666666
       name: Cosine Recall@1
     - type: cosine_recall@3
       value: 1.0
@@ -245,13 +247,13 @@ model-index:
       value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 1.0
+      value: 0.8551549589285763
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 1.0
+      value: 0.8055555555555557
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 1.0
+      value: 0.8055555555555557
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -261,10 +263,10 @@ model-index:
       type: dim_128
     metrics:
     - type: cosine_accuracy@1
-      value: 0.8
+      value: 0.6666666666666666
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
       value: 1.0
@@ -273,22 +275,22 @@ model-index:
       value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.8
+      value: 0.6666666666666666
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.3333333333333333
+      value: 0.27777777777777773
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.2
+      value: 0.19999999999999998
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.1
+      value: 0.09999999999999999
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.8
+      value: 0.6666666666666666
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 1.0
+      value: 0.8333333333333334
       name: Cosine Recall@3
     - type: cosine_recall@5
       value: 1.0
@@ -297,13 +299,13 @@ model-index:
       value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.9261859507142916
+      value: 0.8217794263455654
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.9
+      value: 0.763888888888889
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.9
+      value: 0.763888888888889
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -313,7 +315,7 @@ model-index:
       type: dim_64
     metrics:
     - type: cosine_accuracy@1
-      value: 0.8
+      value: 0.8333333333333334
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
       value: 1.0
@@ -325,19 +327,19 @@ model-index:
       value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.8
+      value: 0.8333333333333334
       name: Cosine Precision@1
     - type: cosine_precision@3
       value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.2
+      value: 0.19999999999999998
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.1
+      value: 0.09999999999999999
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.8
+      value: 0.8333333333333334
       name: Cosine Recall@1
     - type: cosine_recall@3
       value: 1.0
@@ -349,13 +351,13 @@ model-index:
       value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.9261859507142916
+      value: 0.9166666666666666
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.9
+      value: 0.888888888888889
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.9
+      value: 0.888888888888889
       name: Cosine Map@100
 ---
 
@@ -409,9 +411,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("Nuf-hugginface/modernbert-embed-quickb")
 # Run inference
 sentences = [
-    'What is required for effectively deploying LLMs in real-world applications?',
-    'However, deploying LLMs effectively in real-world applications often requires LLM integration. This means embedding these models into systems, workflows, or products where they can interact with other components like databases, APIs, user interfaces, or even custom business logic',
-    '. For instance, a spam filter doesn’t just block emails with specific keywords—it learns from thousands of examples what spam typically looks like.',
+    'What does the use of RAG systems improve according to the text?',
+    'Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG systems, the model first retrieves relevant documents from a database (like a knowledge base), then generates a response using that context—significantly improving the relevance and accuracy of the answers.',
+    'Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG systems, the model first retrieves relevant documents from a database (like a knowledge base), then generates a response using that context—significantly improving the relevance and accuracy of the answers.',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -456,23 +458,23 @@ You can finetune this model on your own dataset.
 * Datasets: `dim_768`, `dim_512`, `dim_256`, `dim_128` and `dim_64`
 * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
 
-| Metric              | dim_768 | dim_512 | dim_256 | dim_128    | dim_64     |
-|:--------------------|:--------|:--------|:--------|:-----------|:-----------|
-| cosine_accuracy@1   | 1.0     | 1.0     | 1.0     | 0.8        | 0.8        |
-| cosine_accuracy@3   | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| cosine_accuracy@5   | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| cosine_accuracy@10  | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| cosine_precision@1  | 1.0     | 1.0     | 1.0     | 0.8        | 0.8        |
-| cosine_precision@3  | 0.3333  | 0.3333  | 0.3333  | 0.3333     | 0.3333     |
-| cosine_precision@5  | 0.2     | 0.2     | 0.2     | 0.2        | 0.2        |
-| cosine_precision@10 | 0.1     | 0.1     | 0.1     | 0.1        | 0.1        |
-| cosine_recall@1     | 1.0     | 1.0     | 1.0     | 0.8        | 0.8        |
-| cosine_recall@3     | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| cosine_recall@5     | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| cosine_recall@10    | 1.0     | 1.0     | 1.0     | 1.0        | 1.0        |
-| **cosine_ndcg@10**  | **1.0** | **1.0** | **1.0** | **0.9262** | **0.9262** |
-| cosine_mrr@10       | 1.0     | 1.0     | 1.0     | 0.9        | 0.9        |
-| cosine_map@100      | 1.0     | 1.0     | 1.0     | 0.9        | 0.9        |
+| Metric              | dim_768    | dim_512   | dim_256    | dim_128    | dim_64     |
+|:--------------------|:-----------|:----------|:-----------|:-----------|:-----------|
+| cosine_accuracy@1   | 0.6667     | 0.6667    | 0.6667     | 0.6667     | 0.8333     |
+| cosine_accuracy@3   | 0.8333     | 1.0       | 1.0        | 0.8333     | 1.0        |
+| cosine_accuracy@5   | 1.0        | 1.0       | 1.0        | 1.0        | 1.0        |
+| cosine_accuracy@10  | 1.0        | 1.0       | 1.0        | 1.0        | 1.0        |
+| cosine_precision@1  | 0.6667     | 0.6667    | 0.6667     | 0.6667     | 0.8333     |
+| cosine_precision@3  | 0.2778     | 0.3333    | 0.3333     | 0.2778     | 0.3333     |
+| cosine_precision@5  | 0.2        | 0.2       | 0.2        | 0.2        | 0.2        |
+| cosine_precision@10 | 0.1        | 0.1       | 0.1        | 0.1        | 0.1        |
+| cosine_recall@1     | 0.6667     | 0.6667    | 0.6667     | 0.6667     | 0.8333     |
+| cosine_recall@3     | 0.8333     | 1.0       | 1.0        | 0.8333     | 1.0        |
+| cosine_recall@5     | 1.0        | 1.0       | 1.0        | 1.0        | 1.0        |
+| cosine_recall@10    | 1.0        | 1.0       | 1.0        | 1.0        | 1.0        |
+| **cosine_ndcg@10**  | **0.8436** | **0.877** | **0.8552** | **0.8218** | **0.9167** |
+| cosine_mrr@10       | 0.7917     | 0.8333    | 0.8056     | 0.7639     | 0.8889     |
+| cosine_map@100      | 0.7917     | 0.8333    | 0.8056     | 0.7639     | 0.8889     |
 
 <!--
 ## Bias, Risks and Limitations
@@ -492,19 +494,19 @@ You can finetune this model on your own dataset.
 
 #### Unnamed Dataset
 
-* Size: 44 training samples
+* Size: 46 training samples
 * Columns: <code>anchor</code> and <code>positive</code>
-* Approximate statistics based on the first 44 samples:
+* Approximate statistics based on the first 46 samples:
   |         | anchor                                                                            | positive                                                                           |
   |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
   | type    | string                                                                            | string                                                                             |
-  | details | <ul><li>min: 8 tokens</li><li>mean: 12.68 tokens</li><li>max: 16 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 47.25 tokens</li><li>max: 83 tokens</li></ul> |
+  | details | <ul><li>min: 9 tokens</li><li>mean: 13.28 tokens</li><li>max: 18 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 47.54 tokens</li><li>max: 83 tokens</li></ul> |
 * Samples:
-  | anchor                                                                       | positive                                                                                                                                                                                                                                                                                                      |
-  |:-----------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-  | <code>What role does generalization ability play in customer service?</code> | <code>. This generalization ability makes them incredibly useful across industries—from customer service and education to software development and healthcare.</code>                                                                                                                                         |
-  | <code>Do LLMs require task-specific training to perform tasks?</code>        | <code>LLMs work by learning statistical relationships between words and phrases, allowing them to predict and generate language that feels natural. The power of these models lies not only in their size but also in the diversity of tasks they can perform with little to no task-specific training</code> |
-  | <code>What does a spam filter learn from?</code>                             | <code>. For instance, a spam filter doesn’t just block emails with specific keywords—it learns from thousands of examples what spam typically looks like.</code>                                                                                                                                              |
+  | anchor                                                                              | positive                                                                                                                                                                                                                                                                                                                                                                                            |
+  |:------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What does RAG stand for in the context of the text?</code>                    | <code>Furthermore, advanced integrations might include fine-tuning the LLM on domain-specific data, or pairing it with retrieval-augmented generation (RAG) pipelines. In RAG systems, the model first retrieves relevant documents from a database (like a knowledge base), then generates a response using that context—significantly improving the relevance and accuracy of the answers.</code> |
+  | <code>What type of information can the LLM use to answer customer questions?</code> | <code>. For example, integrating an LLM into a customer support chatbot might involve connecting it to a company’s internal knowledge base, enabling it to answer customer questions using accurate, up-to-date information.</code>                                                                                                                                                                 |
+  | <code>What do AI and ML form the foundation for?</code>                             | <code>In summary, AI and ML form the foundation for intelligent automation, while LLMs represent a breakthrough in language understanding and generation. Integrating these models into real-world systems unlocks practical value, turning raw intelligence into tangible solutions</code>                                                                                                         |
 * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
   ```json
   {
@@ -665,8 +667,8 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch   | Step  | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
 |:-------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
-| **1.0** | **2** | **1.0**                | **1.0**                | **1.0**                | **0.9262**             | **0.9**               |
-| 2.0     | 4     | 1.0                    | 1.0                    | 1.0                    | 0.9262                 | 0.9262                |
+| 1.0     | 2     | 0.8244                 | 0.8770                 | 0.8244                 | 0.8029                 | 0.8552                |
+| **2.0** | **4** | **0.8436**             | **0.877**              | **0.8552**             | **0.8218**             | **0.9167**            |
 
 * The bold row denotes the saved checkpoint.
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f0e68ee945d1bbc67d5cc8f65e86d365fe1ff4f668bc04a3c9b4f7049249f7a0
+oid sha256:32dfd9d19e38eafcef8a97038c57c0f09c730e4f42354f9efc84c9b49b1aab11
 size 596070136