Text Generation
Transformers
Safetensors
English
qwen2
conversational
text-generation-inference
krogoldAI commited on
Commit
cb98cf3
·
verified ·
1 Parent(s): 5c38700

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -31
README.md CHANGED
@@ -146,6 +146,32 @@ The model underwent full fine-tuning (not parameter-efficient methods like LoRA)
146
 
147
  The training code can be found [here](Code/Fine-tuning.py).
148
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
149
  ## Model Capabilities
150
 
151
  The model employs a systematic, ontology-inspired analysis framework that decomposes queries into structured XML representations. This analytical approach (which identifies domains, intents, concepts, relations, and ambiguities) aims to enhance the quality of the rephrased query output. For every query, it provides domain classification and intent detection with confidence scores that sum to `1.0`, properly handling both unambiguous cases (single candidate with confidence `1.0`) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
@@ -312,37 +338,6 @@ print(analysis)
312
 
313
  To assess the model's ability to produce correctly formatted outputs, we evaluated structural conformance across 1,000 examples from the test split of [krogoldAI/rag-ambiguous-queries](https://huggingface.co/datasets/krogoldAI/rag-ambiguous-queries), comparing QueryRefiner-0.5B-v0.1 against its base model [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). The evaluation measured adherence to the required XML schema, including tag presence, well-formedness, element ordering, and confidence score formatting.
314
 
315
-
316
- # Model Card for Model ID
317
-
318
- <!-- Provide a quick summary of what the model is/does. -->
319
-
320
- GRPO + SFT on 7.3k data from [krogoldAI/rag-query-analysis](https://huggingface.co/datasets/krogoldAI/rag-query-analysis/viewer) with:
321
-
322
- ```py
323
- NUM_EPOCHS = 2
324
- PER_DEVICE_BATCH = 4
325
- GRADIENT_ACCUMULATION = 4
326
- LEARNING_RATE = 2e-5
327
- WEIGHT_DECAY = 0.01
328
- WARMUP_STEPS = 100
329
- ```
330
-
331
- (--> Breadth)
332
-
333
- Second SFT on ~3k examples with:
334
-
335
- ```py
336
- NUM_EPOCHS = 3
337
- PER_DEVICE_BATCH = 4
338
- GRADIENT_ACCUMULATION = 4
339
- LEARNING_RATE = 2e-5
340
- WEIGHT_DECAY = 0.01
341
- WARMUP_STEPS = 50
342
- ```
343
-
344
- (--> Depth)
345
-
346
  #### Table 1 - Structural validity metrics (% of outputs meeting each requirement)
347
 
348
  |Metric|Qwen2.5-0.5B-Instruct|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO|
 
146
 
147
  The training code can be found [here](Code/Fine-tuning.py).
148
 
149
+ GRPO + SFT on 7.3k data from [krogoldAI/rag-query-analysis](https://huggingface.co/datasets/krogoldAI/rag-query-analysis/viewer) with:
150
+
151
+ ```py
152
+ NUM_EPOCHS = 2
153
+ PER_DEVICE_BATCH = 4
154
+ GRADIENT_ACCUMULATION = 4
155
+ LEARNING_RATE = 2e-5
156
+ WEIGHT_DECAY = 0.01
157
+ WARMUP_STEPS = 100
158
+ ```
159
+
160
+ (--> Breadth)
161
+
162
+ Second SFT on ~3k examples with:
163
+
164
+ ```py
165
+ NUM_EPOCHS = 3
166
+ PER_DEVICE_BATCH = 4
167
+ GRADIENT_ACCUMULATION = 4
168
+ LEARNING_RATE = 2e-5
169
+ WEIGHT_DECAY = 0.01
170
+ WARMUP_STEPS = 50
171
+ ```
172
+
173
+ (--> Depth)
174
+
175
  ## Model Capabilities
176
 
177
  The model employs a systematic, ontology-inspired analysis framework that decomposes queries into structured XML representations. This analytical approach (which identifies domains, intents, concepts, relations, and ambiguities) aims to enhance the quality of the rephrased query output. For every query, it provides domain classification and intent detection with confidence scores that sum to `1.0`, properly handling both unambiguous cases (single candidate with confidence `1.0`) and ambiguous cases (multiple candidates with distributed confidence). The model can optionally extract explicit and implicit concepts, identify relations between entities using subject-predicate-object triples, and normalize ambiguous terms when disambiguation would improve retrieval.
 
338
 
339
  To assess the model's ability to produce correctly formatted outputs, we evaluated structural conformance across 1,000 examples from the test split of [krogoldAI/rag-ambiguous-queries](https://huggingface.co/datasets/krogoldAI/rag-ambiguous-queries), comparing QueryRefiner-0.5B-v0.1 against its base model [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). The evaluation measured adherence to the required XML schema, including tag presence, well-formedness, element ordering, and confidence score formatting.
340
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
341
  #### Table 1 - Structural validity metrics (% of outputs meeting each requirement)
342
 
343
  |Metric|Qwen2.5-0.5B-Instruct|QueryRefiner-0.5B-v0.1-SFT|QueryRefiner-0.5B-v0.1-GRPO|