krogoldAI commited on
Commit
9fb6e6c
·
verified ·
1 Parent(s): 73b70fd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -13,25 +13,29 @@ This model addresses a critical challenge in RAG systems: converting ambiguous,
13
 
14
  ## Intended Use
15
 
16
- The model is designed to serve as a query analysis layer in RAG systems, where it processes user inputs before they are sent to retrieval components. It excels at handling queries with varying degrees of ambiguity and can identify when queries lack sufficient context for effective retrieval. The structured XML output enables downstream systems to make informed decisions about how to handle each query based on confidence scores, ambiguity flags, and the rephrased version.
17
 
18
  Typical deployment scenarios include conversational search systems, question-answering platforms, document retrieval services, and any application where understanding user intent and optimizing queries for semantic search is critical. The model's compact 0.5B parameter size makes it suitable for deployment in resource-constrained environments or as part of larger multi-component systems where latency is a concern.
19
 
20
  ## Training Data
21
 
22
- The model was trained on the [krogoldAI/Query_rephraser_SFT](https://huggingface.co/datasets/krogoldAI/Query_rephraser_SFT) dataset, which contains 7,305 high-quality query-analysis pairs. This dataset was carefully curated from three established sources: [rag-datasets/rag-mini-wikipedia](https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia), [razbit96/Ambiguity-Handling-in-User-Queries](https://huggingface.co/datasets/razbit96/Ambiguity-Handling-in-User-Queries), and [glaiveai/RAG-v1](https://huggingface.co/datasets/glaiveai/RAG-v1). Approximately 20% of the training examples include queries with systematically introduced ambiguity at varying levels (low, medium, and high) to ensure the model can handle realistic user inputs across the ambiguity spectrum.
 
 
 
 
23
 
24
  The training data underwent rigorous quality assurance through a dual evaluation framework. Each example was validated for strict XML schema conformance and semantically evaluated using an LLM-as-a-judge protocol with six quality dimensions. Only examples achieving both perfect structural validity and high semantic quality scores were included in the final dataset, ensuring the model was trained exclusively on gold-standard examples.
25
 
26
  ## Training Procedure
27
 
28
- The model underwent full fine-tuning (not parameter-efficient methods like LoRA) of all parameters in Qwen2.5-0.5B-Instruct. Training was conducted over three epochs with a per-device batch size of 4 and gradient accumulation over 4 steps, yielding an effective batch size of 16. The learning rate was set to 2e-5 with a weight decay of 0.01 to prevent overfitting. A warmup period of 100 steps allowed the model to gradually adapt to the task-specific distribution before reaching the full learning rate. Training was performed on an NVIDIA A100 SXM GPU.
29
 
30
  This training configuration was chosen to balance thorough adaptation to the structured output task while preserving the base model's language understanding capabilities. The full fine-tuning approach, rather than parameter-efficient alternatives, ensures the model fully internalizes the strict XML formatting requirements and the nuanced query analysis patterns present in the training data.
31
 
32
  ## Model Capabilities
33
 
34
- The model generates structured XML analyses following a specific schema that includes both mandatory and optional fields. For every query, it provides domain classification and intent detection with confidence scores that sum to 1.0, properly handling both unambiguous cases (single candidate with confidence 1.0) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
35
 
36
  The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as [PERSON] or [COMPANY]. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
37
 
 
13
 
14
  ## Intended Use
15
 
16
+ The model is designed to serve as a query analysis layer in RAG systems, where it processes user inputs before they are sent to retrieval components. It can handle queries with varying degrees of ambiguity and identify when queries lack sufficient context for effective retrieval. The structured XML output enables downstream systems to make informed decisions about how to handle each query based on confidence scores, ambiguity flags, and the rephrased version.
17
 
18
  Typical deployment scenarios include conversational search systems, question-answering platforms, document retrieval services, and any application where understanding user intent and optimizing queries for semantic search is critical. The model's compact 0.5B parameter size makes it suitable for deployment in resource-constrained environments or as part of larger multi-component systems where latency is a concern.
19
 
20
  ## Training Data
21
 
22
+ The model was trained on the [krogoldAI/Query_rephraser_SFT](https://huggingface.co/datasets/krogoldAI/Query_rephraser_SFT) dataset, which contains 7,305 high-quality query-analysis pairs. This dataset was carefully curated from three sources:
23
+ - [rag-datasets/rag-mini-wikipedia](https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia)
24
+ - [razbit96/Ambiguity-Handling-in-User-Queries](https://huggingface.co/datasets/razbit96/Ambiguity-Handling-in-User-Queries)
25
+ - [glaiveai/RAG-v1](https://huggingface.co/datasets/glaiveai/RAG-v1).
26
+ Approximately 20% of the training examples include queries with systematically introduced ambiguity at varying levels to ensure the model can handle realistic user inputs across the ambiguity spectrum.
27
 
28
  The training data underwent rigorous quality assurance through a dual evaluation framework. Each example was validated for strict XML schema conformance and semantically evaluated using an LLM-as-a-judge protocol with six quality dimensions. Only examples achieving both perfect structural validity and high semantic quality scores were included in the final dataset, ensuring the model was trained exclusively on gold-standard examples.
29
 
30
  ## Training Procedure
31
 
32
+ The model underwent full fine-tuning (not parameter-efficient methods like LoRA) of all parameters in [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). Training was conducted over three epochs with a per-device batch size of \\( 4 \\) and gradient accumulation over \\( 4 \\) steps, yielding an effective batch size of \\( 16 \\). The learning rate was set to \\( 2\times 10^{-5} \\) with a weight decay of \\( 0.01 \\) to prevent overfitting. A warmup period of \\( 100 \\) steps allowed the model to gradually adapt to the task-specific distribution before reaching the full learning rate. Training was performed on an NVIDIA A100 SXM GPU.
33
 
34
  This training configuration was chosen to balance thorough adaptation to the structured output task while preserving the base model's language understanding capabilities. The full fine-tuning approach, rather than parameter-efficient alternatives, ensures the model fully internalizes the strict XML formatting requirements and the nuanced query analysis patterns present in the training data.
35
 
36
  ## Model Capabilities
37
 
38
+ The model generates structured XML analyses following a specific schema that includes both mandatory and optional fields. For every query, it provides domain classification and intent detection with confidence scores that sum to `1.0`, properly handling both unambiguous cases (single candidate with confidence 1.0) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
39
 
40
  The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as [PERSON] or [COMPANY]. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
41