Update README.md
Browse files
README.md
CHANGED
|
@@ -3,6 +3,21 @@ library_name: transformers
|
|
| 3 |
tags: []
|
| 4 |
---
|
| 5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
# Model Card for Model ID
|
| 7 |
|
| 8 |
<!-- Provide a quick summary of what the model is/does. -->
|
|
@@ -35,18 +50,35 @@ WARMUP_STEPS = 50
|
|
| 35 |
|
| 36 |
#### Table 1 - Structural validity metrics (% of outputs meeting each requirement)
|
| 37 |
|
| 38 |
-
|Metric|Qwen2.5-0.5B-Instruct|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|----:|:-------------------:|:--------------------:|:--------------------:|:--------------------:|
|
| 40 |
|Tag structure|10.8%|99.9%|100%|99.9%|
|
| 41 |
|XML validity|41.0%|99.8%|99.8%|99.9%|
|
| 42 |
|Order|2.0%|99.9%|100%|99.9%|
|
| 43 |
|Confidence|3.1%|99.9%|100%|99.9%|
|
| 44 |
-
|Perfectly structured output|0.0%|99.8%|99.8%|99.9%|
|
| 45 |
-
|
| 46 |
|
| 47 |
#### Table 2 - Semantic scores
|
| 48 |
|
| 49 |
-
|Metric|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
|----:|:--------------------:|:--------------------:|:--------------------:|
|
| 51 |
|Domain accuracy|**98.97 ± 8.43%**|98.57 ± 10.25%|98.95 ± 8.46%|
|
| 52 |
|Intent accuracy|98.20 ± 8.84%|**98.85 ± 8.30%**|98.75 ± 8.05%|
|
|
@@ -55,6 +87,7 @@ WARMUP_STEPS = 50
|
|
| 55 |
|Intent preservation|95.09 ± 15.74%|95.52 ± 15.00%|**96.02 ± 14.72%**|
|
| 56 |
|Follows guidelines|97.07 ± 13.78%|97.07 ± 12.90%|**97.40 ± 12.20%**|
|
| 57 |
|Overall semantic score|96.10 ± 9.74%|96.45 ± 9.52%|**96.81 ± 9.18%**|
|
|
|
|
| 58 |
|
| 59 |
(Corrected for the SFT + GRPO-SFT models)
|
| 60 |
|
|
|
|
| 3 |
tags: []
|
| 4 |
---
|
| 5 |
|
| 6 |
+
# QueryRefiner-0.5B-v0.1-GRPO
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
<!-- RAG Query Analyzer 0.5B v0.1 -->
|
| 11 |
+
|
| 12 |
+
## Model Description
|
| 13 |
+
|
| 14 |
+
QueryRefiner-0.5B-v0.1-GRPO is a specialized query analysis and rephrasing model fine-tuned from [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for Retrieval-Augmented Generation (RAG) systems. The model has been trained to analyze user queries through a structured, ontology-inspired reasoning framework that produces XML outputs with domain classification, intent detection, conceptual analysis, and optimized query reformulations designed to improve document retrieval precision.
|
| 15 |
+
|
| 16 |
+
This model addresses a critical challenge in RAG systems: converting ambiguous, vague, or suboptimally-phrased user queries into retrieval-friendly formats through systematic decomposition and analysis. By employing a structured reasoning approach that explicitly identifies domains, intents, concepts, and relationships, the model produces higher-quality rephrasings informed by comprehensive query understanding. <!-- Rather than directly answering questions, the model performs query understanding and transformation, making it an ideal preprocessing component for information retrieval pipelines. -->
|
| 17 |
+
|
| 18 |
+
**Note:** This model is currently in an experimental, beta stage. It represents an initial release focused on establishing baseline capabilities. Future iterations may address current limitations, expand domain coverage, or incorporate additional output features based on real-world deployment feedback.
|
| 19 |
+
|
| 20 |
+
|
| 21 |
# Model Card for Model ID
|
| 22 |
|
| 23 |
<!-- Provide a quick summary of what the model is/does. -->
|
|
|
|
| 50 |
|
| 51 |
#### Table 1 - Structural validity metrics (% of outputs meeting each requirement)
|
| 52 |
|
| 53 |
+
|Metric|Qwen2.5-0.5B-Instruct|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO|
|
| 54 |
+
|----:|:-------------------:|:--------------------:|:--------------------:|:--------------------:|
|
| 55 |
+
|Tag structure|10.8%|99.9%|99.9%|
|
| 56 |
+
|XML validity|41.0%|99.8%|99.9%|
|
| 57 |
+
|Order|2.0%|99.9%|99.9%|
|
| 58 |
+
|Confidence|3.1%|99.9%|99.9%|
|
| 59 |
+
|Perfectly structured output|0.0%|99.8%|99.9%|
|
| 60 |
+
|
| 61 |
+
<!-- |Metric|Qwen2.5-0.5B-Instruct|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO-SFT|QueryRefiner-0.5B-v0.1-GRPO-SFT1-SFT2-v2|
|
| 62 |
|----:|:-------------------:|:--------------------:|:--------------------:|:--------------------:|
|
| 63 |
|Tag structure|10.8%|99.9%|100%|99.9%|
|
| 64 |
|XML validity|41.0%|99.8%|99.8%|99.9%|
|
| 65 |
|Order|2.0%|99.9%|100%|99.9%|
|
| 66 |
|Confidence|3.1%|99.9%|100%|99.9%|
|
| 67 |
+
|Perfectly structured output|0.0%|99.8%|99.8%|99.9%| -->
|
|
|
|
| 68 |
|
| 69 |
#### Table 2 - Semantic scores
|
| 70 |
|
| 71 |
+
|Metric|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO|
|
| 72 |
+
|----:|:--------------------:|:--------------------:|
|
| 73 |
+
|Domain accuracy|**98.97 ± 8.43%**|98.95 ± 8.46%|
|
| 74 |
+
|Intent accuracy|98.20 ± 8.84%|**98.75 ± 8.05%**|
|
| 75 |
+
|Ambiguity assessment|99.04 ± 6.84%|**99.40 ± 5.79%**|
|
| 76 |
+
|Rephrasing quality|88.23 ± 18.20%|**90.33 ± 17.61%**|
|
| 77 |
+
|Intent preservation|95.09 ± 15.74%|**96.02 ± 14.72%**|
|
| 78 |
+
|Follows guidelines|97.07 ± 13.78%|**97.40 ± 12.20%**|
|
| 79 |
+
|Overall semantic score|96.10 ± 9.74%|**96.81 ± 9.18%**|
|
| 80 |
+
|
| 81 |
+
<!-- |Metric|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO-SFT|QueryRefiner-0.5B-v0.1-GRPO-SFT1-SFT2-v2|
|
| 82 |
|----:|:--------------------:|:--------------------:|:--------------------:|
|
| 83 |
|Domain accuracy|**98.97 ± 8.43%**|98.57 ± 10.25%|98.95 ± 8.46%|
|
| 84 |
|Intent accuracy|98.20 ± 8.84%|**98.85 ± 8.30%**|98.75 ± 8.05%|
|
|
|
|
| 87 |
|Intent preservation|95.09 ± 15.74%|95.52 ± 15.00%|**96.02 ± 14.72%**|
|
| 88 |
|Follows guidelines|97.07 ± 13.78%|97.07 ± 12.90%|**97.40 ± 12.20%**|
|
| 89 |
|Overall semantic score|96.10 ± 9.74%|96.45 ± 9.52%|**96.81 ± 9.18%**|
|
| 90 |
+
-->
|
| 91 |
|
| 92 |
(Corrected for the SFT + GRPO-SFT models)
|
| 93 |
|