Spaces:
Running
Running
Update app.py
Browse files
app.py
CHANGED
|
@@ -86,7 +86,7 @@ class LLMEvaluator:
|
|
| 86 |
def evaluate(self, context, question, student_answer, max_marks):
|
| 87 |
# OPTIMIZED PROMPT FOR SMALL MODELS (0.5B)
|
| 88 |
messages = [
|
| 89 |
-
{"role": "system", "content": "You are a strictest, literal academic grader. You ONLY grade based on the provided text. You DO NOT use outside knowledge."},
|
| 90 |
{"role": "user", "content": f"""
|
| 91 |
Task: Grade the student answer based ONLY on the Reference Text.
|
| 92 |
|
|
@@ -101,8 +101,8 @@ class LLMEvaluator:
|
|
| 101 |
|
| 102 |
-----------------------------
|
| 103 |
GRADING LOGIC:
|
| 104 |
-
1. READ the Reference Text. What does it actually say about the Question?
|
| 105 |
-
2. COMPARE it to the Student Answer.
|
| 106 |
3 START with 0 marks and IF the answers line up to the reference text in a meaningful way, then add marks porportionally. ONLY GIVE MARKS FOR CORRECT STATEMENT STRICTLY BASED ON THE REFERENCE TEXT AND NOTHING ELSE IN THIS WORLD.
|
| 107 |
4. IF the Student Answer claims things not found in the text , he is incorrect and HALLUCINATING. Do not give marks for that statment/phrase
|
| 108 |
5. IF the Student Answer contradicts the text (e.g., Text says "hide personality" but Student says "show personality"), Do not give marks for that statment/phrase
|
|
|
|
| 86 |
def evaluate(self, context, question, student_answer, max_marks):
|
| 87 |
# OPTIMIZED PROMPT FOR SMALL MODELS (0.5B)
|
| 88 |
messages = [
|
| 89 |
+
{"role": "system", "content": "You are a strictest, literal academic grader in the whole. You ONLY grade based on the provided text. You DO NOT use outside knowledge."},
|
| 90 |
{"role": "user", "content": f"""
|
| 91 |
Task: Grade the student answer based ONLY on the Reference Text.
|
| 92 |
|
|
|
|
| 101 |
|
| 102 |
-----------------------------
|
| 103 |
GRADING LOGIC:
|
| 104 |
+
1. READ the Reference Text and use that as the ground truth. What does it actually say about the Question?
|
| 105 |
+
2. COMPARE it to the Student Answer, do not forcefully agree with the answer by seeing things that are not there. You are to penalise irrelevant text and contradictions whenever you encounter them.
|
| 106 |
3 START with 0 marks and IF the answers line up to the reference text in a meaningful way, then add marks porportionally. ONLY GIVE MARKS FOR CORRECT STATEMENT STRICTLY BASED ON THE REFERENCE TEXT AND NOTHING ELSE IN THIS WORLD.
|
| 107 |
4. IF the Student Answer claims things not found in the text , he is incorrect and HALLUCINATING. Do not give marks for that statment/phrase
|
| 108 |
5. IF the Student Answer contradicts the text (e.g., Text says "hide personality" but Student says "show personality"), Do not give marks for that statment/phrase
|