Instructions to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="krogoldAI/QueryRefiner-0.5B-v0.1-SFT") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-SFT") model = AutoModelForCausalLM.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-SFT", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-SFT
- SGLang
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with Docker Model Runner:
docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-SFT
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,25 +13,29 @@ This model addresses a critical challenge in RAG systems: converting ambiguous,
|
|
| 13 |
|
| 14 |
## Intended Use
|
| 15 |
|
| 16 |
-
The model is designed to serve as a query analysis layer in RAG systems, where it processes user inputs before they are sent to retrieval components. It
|
| 17 |
|
| 18 |
Typical deployment scenarios include conversational search systems, question-answering platforms, document retrieval services, and any application where understanding user intent and optimizing queries for semantic search is critical. The model's compact 0.5B parameter size makes it suitable for deployment in resource-constrained environments or as part of larger multi-component systems where latency is a concern.
|
| 19 |
|
| 20 |
## Training Data
|
| 21 |
|
| 22 |
-
The model was trained on the [krogoldAI/Query_rephraser_SFT](https://huggingface.co/datasets/krogoldAI/Query_rephraser_SFT) dataset, which contains 7,305 high-quality query-analysis pairs. This dataset was carefully curated from three
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
The training data underwent rigorous quality assurance through a dual evaluation framework. Each example was validated for strict XML schema conformance and semantically evaluated using an LLM-as-a-judge protocol with six quality dimensions. Only examples achieving both perfect structural validity and high semantic quality scores were included in the final dataset, ensuring the model was trained exclusively on gold-standard examples.
|
| 25 |
|
| 26 |
## Training Procedure
|
| 27 |
|
| 28 |
-
The model underwent full fine-tuning (not parameter-efficient methods like LoRA) of all parameters in Qwen2.5-0.5B-Instruct. Training was conducted over three epochs with a per-device batch size of 4 and gradient accumulation over 4 steps, yielding an effective batch size of 16. The learning rate was set to
|
| 29 |
|
| 30 |
This training configuration was chosen to balance thorough adaptation to the structured output task while preserving the base model's language understanding capabilities. The full fine-tuning approach, rather than parameter-efficient alternatives, ensures the model fully internalizes the strict XML formatting requirements and the nuanced query analysis patterns present in the training data.
|
| 31 |
|
| 32 |
## Model Capabilities
|
| 33 |
|
| 34 |
-
The model generates structured XML analyses following a specific schema that includes both mandatory and optional fields. For every query, it provides domain classification and intent detection with confidence scores that sum to 1.0, properly handling both unambiguous cases (single candidate with confidence 1.0) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
|
| 35 |
|
| 36 |
The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as [PERSON] or [COMPANY]. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
|
| 37 |
|
|
|
|
| 13 |
|
| 14 |
## Intended Use
|
| 15 |
|
| 16 |
+
The model is designed to serve as a query analysis layer in RAG systems, where it processes user inputs before they are sent to retrieval components. It can handle queries with varying degrees of ambiguity and identify when queries lack sufficient context for effective retrieval. The structured XML output enables downstream systems to make informed decisions about how to handle each query based on confidence scores, ambiguity flags, and the rephrased version.
|
| 17 |
|
| 18 |
Typical deployment scenarios include conversational search systems, question-answering platforms, document retrieval services, and any application where understanding user intent and optimizing queries for semantic search is critical. The model's compact 0.5B parameter size makes it suitable for deployment in resource-constrained environments or as part of larger multi-component systems where latency is a concern.
|
| 19 |
|
| 20 |
## Training Data
|
| 21 |
|
| 22 |
+
The model was trained on the [krogoldAI/Query_rephraser_SFT](https://huggingface.co/datasets/krogoldAI/Query_rephraser_SFT) dataset, which contains 7,305 high-quality query-analysis pairs. This dataset was carefully curated from three sources:
|
| 23 |
+
- [rag-datasets/rag-mini-wikipedia](https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia)
|
| 24 |
+
- [razbit96/Ambiguity-Handling-in-User-Queries](https://huggingface.co/datasets/razbit96/Ambiguity-Handling-in-User-Queries)
|
| 25 |
+
- [glaiveai/RAG-v1](https://huggingface.co/datasets/glaiveai/RAG-v1).
|
| 26 |
+
Approximately 20% of the training examples include queries with systematically introduced ambiguity at varying levels to ensure the model can handle realistic user inputs across the ambiguity spectrum.
|
| 27 |
|
| 28 |
The training data underwent rigorous quality assurance through a dual evaluation framework. Each example was validated for strict XML schema conformance and semantically evaluated using an LLM-as-a-judge protocol with six quality dimensions. Only examples achieving both perfect structural validity and high semantic quality scores were included in the final dataset, ensuring the model was trained exclusively on gold-standard examples.
|
| 29 |
|
| 30 |
## Training Procedure
|
| 31 |
|
| 32 |
+
The model underwent full fine-tuning (not parameter-efficient methods like LoRA) of all parameters in [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). Training was conducted over three epochs with a per-device batch size of \\( 4 \\) and gradient accumulation over \\( 4 \\) steps, yielding an effective batch size of \\( 16 \\). The learning rate was set to \\( 2\times 10^{-5} \\) with a weight decay of \\( 0.01 \\) to prevent overfitting. A warmup period of \\( 100 \\) steps allowed the model to gradually adapt to the task-specific distribution before reaching the full learning rate. Training was performed on an NVIDIA A100 SXM GPU.
|
| 33 |
|
| 34 |
This training configuration was chosen to balance thorough adaptation to the structured output task while preserving the base model's language understanding capabilities. The full fine-tuning approach, rather than parameter-efficient alternatives, ensures the model fully internalizes the strict XML formatting requirements and the nuanced query analysis patterns present in the training data.
|
| 35 |
|
| 36 |
## Model Capabilities
|
| 37 |
|
| 38 |
+
The model generates structured XML analyses following a specific schema that includes both mandatory and optional fields. For every query, it provides domain classification and intent detection with confidence scores that sum to `1.0`, properly handling both unambiguous cases (single candidate with confidence 1.0) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
|
| 39 |
|
| 40 |
The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as [PERSON] or [COMPANY]. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
|
| 41 |
|