Instructions to use krogoldAI/QueryRefiner-0.5B-v0.1-GRPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use krogoldAI/QueryRefiner-0.5B-v0.1-GRPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="krogoldAI/QueryRefiner-0.5B-v0.1-GRPO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-GRPO")
model = AutoModelForCausalLM.from_pretrained("krogoldAI/QueryRefiner-0.5B-v0.1-GRPO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use krogoldAI/QueryRefiner-0.5B-v0.1-GRPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-GRPO

SGLang

How to use krogoldAI/QueryRefiner-0.5B-v0.1-GRPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "krogoldAI/QueryRefiner-0.5B-v0.1-GRPO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use krogoldAI/QueryRefiner-0.5B-v0.1-GRPO with Docker Model Runner:
```
docker model run hf.co/krogoldAI/QueryRefiner-0.5B-v0.1-GRPO
```

krogoldAI commited on Oct 14, 2025

Commit

d6f64b2

verified ·

1 Parent(s): 63683a0

Update README.md

Browse files

Files changed (1) hide show

README.md +19 -17

README.md CHANGED Viewed

@@ -457,17 +457,7 @@ Output only valid JSON. Do not include any explanations, comments, or text outsi
 </details>
-### Performance Considerations
-Performance characteristics will vary based on query type, domain, and ambiguity level. The model is expected to perform strongest on queries similar to those in the training distribution and may require additional fine-tuning or prompt engineering for specialized applications or domains underrepresented in the training data.
-## Acknowledgments
-This model builds upon [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) by the Qwen team at Alibaba Cloud. The training data incorporates queries from [rag-datasets/rag-mini-wikipedia](https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia), [razbit96/Ambiguity-Handling-in-User-Queries](https://huggingface.co/datasets/razbit96/Ambiguity-Handling-in-User-Queries), and [glaiveai/RAG-v1](https://huggingface.co/datasets/glaiveai/RAG-v1).
----
-#### A. Per-ambiguity breakdown + sample sizes
 |                      | NONE           | LOW            | MEDIUM         | HIGH           |
 |:---------------------|:---------------|:---------------|:---------------|:---------------|
@@ -479,11 +469,8 @@ This model builds upon [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2
 | follows_guidelines   | 97.14 ± 13.26% | 97.05 ± 12.91% | **98.89 ± 7.17%**  | 95.67 ± 13.76% |
 | semantic_score       | 96.82 ± 9.58%  | 96.73 ± 9.80%  | **97.31 ± 6.82%**  | 95.11 ± 10.06% |
-Performance analysis across four ambiguity levels shows that QueryRefiner-0.5B-v0.1-GRPO sustains near-saturated accuracy for domain and intent identification (~99%) and achieves perfect structural and ambiguity assessment at the highest ambiguity tier. These results confirm that the model effectively internalized the XML schema and ambiguity-recognition criteria learned during GRPO pretraining.
-More nuanced metrics reveal moderate, expected declines in rephrasing quality (from 90.8 % to 87.5 %) and intent preservation (from ~96 % to 92.8 %) as ambiguity increases, indicating that the model’s semantic generation remains robust but not immune to underspecified inputs. The consistent yet non-flat degradation, coupled with realistic standard-deviation ranges, suggests healthy generalization rather than overfitting: the model adapts sensibly to rising query uncertainty while preserving adherence to output-format and guideline constraints.
-### Discussion of Results
 The evaluation results reveal a clear task difficulty hierarchy that aligns with the inherent complexity of each component. Structural and classification metrics (domain accuracy, intent accuracy, ambiguity assessment) achieve 98-99% performance, while the generative rephrasing task scores lower at ~90%. This gap reflects the fundamental difference in task complexity rather than a training deficiency.
@@ -493,4 +480,19 @@ Rephrasing quality, by contrast, requires the model to make nuanced judgments ab
 The per-ambiguity breakdown supports this interpretation: rephrasing quality remains relatively stable across ambiguity levels (87.5-90.8%), with the expected slight degradation on highly ambiguous queries. This consistency across query types suggests the model has learned generalizable rephrasing strategies rather than memorizing domain-specific patterns. The ~18% standard deviation across all metrics indicates natural variance in query difficulty rather than systematic failures on particular query types.
-For production deployments, users should anticipate that the model will perform most reliably on structural conformance and classification tasks, while rephrasing decisions may occasionally require human review, particularly for edge cases or highly specialized domains underrepresented in the training data.

 </details>
+Here's a per-ambiguity breakdown:
 |                      | NONE           | LOW            | MEDIUM         | HIGH           |
 |:---------------------|:---------------|:---------------|:---------------|:---------------|
 | follows_guidelines   | 97.14 ± 13.26% | 97.05 ± 12.91% | **98.89 ± 7.17%**  | 95.67 ± 13.76% |
 | semantic_score       | 96.82 ± 9.58%  | 96.73 ± 9.80%  | **97.31 ± 6.82%**  | 95.11 ± 10.06% |
+<details>
+  <summary><i>Unroll for a further discussion of results</i></summary>
 The evaluation results reveal a clear task difficulty hierarchy that aligns with the inherent complexity of each component. Structural and classification metrics (domain accuracy, intent accuracy, ambiguity assessment) achieve 98-99% performance, while the generative rephrasing task scores lower at ~90%. This gap reflects the fundamental difference in task complexity rather than a training deficiency.
 The per-ambiguity breakdown supports this interpretation: rephrasing quality remains relatively stable across ambiguity levels (87.5-90.8%), with the expected slight degradation on highly ambiguous queries. This consistency across query types suggests the model has learned generalizable rephrasing strategies rather than memorizing domain-specific patterns. The ~18% standard deviation across all metrics indicates natural variance in query difficulty rather than systematic failures on particular query types.
+For production deployments, users should anticipate that the model will perform most reliably on structural conformance and classification tasks, while rephrasing decisions may occasionally require human review, particularly for edge cases or highly specialized domains underrepresented in the training data.
+</details>
+<!-- Performance analysis across four ambiguity levels shows that QueryRefiner-0.5B-v0.1-GRPO sustains near-saturated accuracy for domain and intent identification (~99%) and achieves perfect structural and ambiguity assessment at the highest ambiguity tier. These results confirm that the model effectively internalized the XML schema and ambiguity-recognition criteria learned during GRPO pretraining.
+More nuanced metrics reveal moderate, expected declines in rephrasing quality (from 90.8 % to 87.5 %) and intent preservation (from ~96 % to 92.8 %) as ambiguity increases, indicating that the model’s semantic generation remains robust but not immune to underspecified inputs. The consistent yet non-flat degradation, coupled with realistic standard-deviation ranges, suggests healthy generalization rather than overfitting: the model adapts sensibly to rising query uncertainty while preserving adherence to output-format and guideline constraints.
+-->
+### Performance Considerations
+Performance characteristics will vary based on query type, domain, and ambiguity level. The model is expected to perform strongest on queries similar to those in the training distribution and may require additional fine-tuning or prompt engineering for specialized applications or domains underrepresented in the training data.
+## Acknowledgments
+This model builds upon [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) by the Qwen team at Alibaba Cloud. The training data incorporates queries from [rag-datasets/rag-mini-wikipedia](https://huggingface.co/datasets/rag-datasets/rag-mini-wikipedia), [razbit96/Ambiguity-Handling-in-User-Queries](https://huggingface.co/datasets/razbit96/Ambiguity-Handling-in-User-Queries), and [glaiveai/RAG-v1](https://huggingface.co/datasets/glaiveai/RAG-v1).