Update README.md
Browse files
README.md
CHANGED
|
@@ -110,3 +110,40 @@ inputs = tokenizer(prompt, return_tensors="pt")
|
|
| 110 |
outputs = model.generate(**inputs, max_new_tokens=120)
|
| 111 |
|
| 112 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
outputs = model.generate(**inputs, max_new_tokens=120)
|
| 111 |
|
| 112 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 113 |
+
```
|
| 114 |
+
---
|
| 115 |
+
## Evaluation
|
| 116 |
+
## Testing Data, Factors & Metrics
|
| 117 |
+
## Testing Data
|
| 118 |
+
|
| 119 |
+
Held-out automotive technician QA samples from the same domain.
|
| 120 |
+
|
| 121 |
+
## Factors
|
| 122 |
+
|
| 123 |
+
Customer complaint handling
|
| 124 |
+
|
| 125 |
+
Diagnostic reasoning
|
| 126 |
+
|
| 127 |
+
Spare part replacement logic
|
| 128 |
+
|
| 129 |
+
Service workflow understanding
|
| 130 |
+
|
| 131 |
+
## Metrics
|
| 132 |
+
|
| 133 |
+
Perplexity
|
| 134 |
+
|
| 135 |
+
Instruction-following accuracy
|
| 136 |
+
|
| 137 |
+
Manual domain evaluation
|
| 138 |
+
|
| 139 |
+
## Results
|
| 140 |
+
|
| 141 |
+
Strong performance on workshop troubleshooting queries
|
| 142 |
+
|
| 143 |
+
Accurate step-by-step diagnostic suggestions
|
| 144 |
+
|
| 145 |
+
Fast inference on CPU
|
| 146 |
+
|
| 147 |
+
## Summary
|
| 148 |
+
|
| 149 |
+
The fine-tuned model shows clear domain adaptation compared to the base SmolLM2 model, especially for automotive service workflows.
|