Instructions to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="krogoldAI/QueryRefiner-0.5B-v0.1-SFT") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-SFT") model = AutoModelForCausalLM.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-SFT", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-SFT
- SGLang
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "krogoldAI/QueryRefiner-0.5B-v0.1-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use krogoldAI/QueryRefiner-0.5B-v0.1-SFT with Docker Model Runner:
docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-SFT
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,7 @@ base_model:
|
|
| 14 |
|
| 15 |
## Model Description
|
| 16 |
|
| 17 |
-
Rephraser-0.5B-v0.1-SFT is a specialized query analysis and rephrasing model fine-tuned from [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for Retrieval-Augmented Generation (RAG) systems. The model has been trained to analyze user queries
|
| 18 |
|
| 19 |
**Note:** This model is currently in an experimental, beta stage. It represents an initial release focused on establishing baseline capabilities. Future iterations may address current limitations, expand domain coverage, or incorporate additional output features based on real-world deployment feedback.
|
| 20 |
|
|
@@ -43,10 +43,12 @@ This training configuration was chosen to balance thorough adaptation to the str
|
|
| 43 |
|
| 44 |
## Model Capabilities
|
| 45 |
|
| 46 |
-
The model
|
| 47 |
|
| 48 |
The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as `[PERSON]` or `[COMPANY]`. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
|
| 49 |
|
|
|
|
|
|
|
| 50 |
The default output schema is the following:
|
| 51 |
|
| 52 |
```xml
|
|
|
|
| 14 |
|
| 15 |
## Model Description
|
| 16 |
|
| 17 |
+
Rephraser-0.5B-v0.1-SFT is a specialized query analysis and rephrasing model fine-tuned from [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for Retrieval-Augmented Generation (RAG) systems. The model has been trained to analyze user queries through a structured, ontology-inspired reasoning framework that produces XML outputs with domain classification, intent detection, conceptual analysis, and optimized query reformulations designed to improve document retrieval precision. This model addresses a critical challenge in RAG systems: converting ambiguous, vague, or suboptimally-phrased user queries into retrieval-friendly formats through systematic decomposition and analysis. By employing a structured reasoning approach that explicitly identifies domains, intents, concepts, and relationships, the model produces higher-quality rephrasings informed by comprehensive query understanding. <!-- Rather than directly answering questions, the model performs query understanding and transformation, making it an ideal preprocessing component for information retrieval pipelines. -->
|
| 18 |
|
| 19 |
**Note:** This model is currently in an experimental, beta stage. It represents an initial release focused on establishing baseline capabilities. Future iterations may address current limitations, expand domain coverage, or incorporate additional output features based on real-world deployment feedback.
|
| 20 |
|
|
|
|
| 43 |
|
| 44 |
## Model Capabilities
|
| 45 |
|
| 46 |
+
The model employs a systematic, ontology-inspired analysis framework that decomposes queries into structured XML representations. This analytical approach—which identifies domains, intents, concepts, relations, and ambiguities—directly informs and enhances the quality of the rephrased query output. For every query, it provides domain classification and intent detection with confidence scores that sum to `1.0`, properly handling both unambiguous cases (single candidate with confidence `1.0`) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
|
| 47 |
|
| 48 |
The rephrasing capability focuses on retrieval optimization rather than query answering. The model transforms queries by using specific terminology likely to appear in relevant documents, expanding acronyms when contextually appropriate, adding disambiguating context, and making implicit references explicit through placeholder notation such as `[PERSON]` or `[COMPANY]`. Importantly, the model has learned to preserve already-optimal queries unchanged, recognizing when rephrasing would not improve retrieval effectiveness.
|
| 49 |
|
| 50 |
+
This structured analytical framework ensures that rephrasings are informed by comprehensive query understanding rather than surface-level transformations, leading to more semantically precise retrieval-optimized queries.
|
| 51 |
+
|
| 52 |
The default output schema is the following:
|
| 53 |
|
| 54 |
```xml
|