atz21 commited on
Commit
f999bc3
Β·
verified Β·
1 Parent(s): 618ee94
Files changed (1) hide show
  1. app.py +176 -54
app.py CHANGED
@@ -20,16 +20,16 @@ client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))
20
  GRID_ROWS, GRID_COLS = 20, 14
21
 
22
  # ---------------- PROMPTS ----------------
23
- PROMPTS = {
24
- "QP_MS_TRANSCRIPTION" : {
25
- "role": "system",
26
- "content": """You are a high-quality OCR/Transcription assistant.
27
  INPUT: This file is a PDF that first contains the Question Paper and immediately after it the Markscheme.
28
  TASK:
29
  1. Transcribe EXACTLY all the questions FIRST (with their total marks).
30
  2. After ALL questions, transcribe the Markscheme exactly, preserving M/A/R notation in brackets.
31
  3. Always number the questions sequentially (Question 1, Question 2, Question 3, …) **in the order they appear in the PDF**, even if the PDF shows a different number or leaves it blank. Do NOT skip or leave Question: blank. Never start a question other than question 1 (even if it is labelled in pdf as 8 name it 1).
32
- 4. If a question or sub-question is labelled with a letter (e.g., β€œQ1.a”, β€œQ2(b)”, β€œ1 (c)(i)”), transcribe it as β€œQuestion 1.a”, β€œQuestion 2.b”, β€œQuestion 1.c.i” etc., exactly preserving the hierarchy of sub-question identifiers.
33
  5. After the markscheme, DETECT and FLAG all questions in the markscheme where a graph/diagram is expected. For each, output the question number and the page number in the format below.
34
 
35
  FORMAT:
@@ -75,56 +75,178 @@ Graph expected in:
75
  - Question <number> β†’ Page <number>
76
  (one per line)
77
  ==== END GRAPH EXPECTED ====
78
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  }
80
- ,
81
-
82
- "GRADING_PROMPT": {
83
- "role": "system",
84
- "content": """Developer: You are an official examiner. Apply the following grading rules precisely.
85
- ### Abbreviations:
86
- - **M**: Marks for Method
87
- - **A**: Marks for Accuracy/Answer
88
- - **R**: Marks for Reasoning
89
- - **AG**: Answer given in questionβ€”no marks
90
- - **FT**: Follow Through marks (if error carried forward correctly)
91
- - **MR**: Deduct for misread (once only)
92
- ---
93
- ## Grading Instructions
94
- 1. Award marks using official annotations (e.g., M1, A2).
95
- 2. Do not award full marks for answers alone; check for method marks.
96
- 3. A marks usually require a valid M mark first.
97
- 4. Accept valid equivalent forms unless otherwise specified.
98
- 5. Apply FT where appropriate.
99
- 6. Use proper notation: M1A0, A1, etc.
100
- 7. Any lost mark: use red `<span style=\"color:red\">M0</span>` , similarly make markscheme expected , student response and awarded marks in red include it in <span> tage
101
- ---
102
- ## Output Format
103
- Produce two sections per question/sub-question, following this structure:
104
- ## Question <id>
105
- ### Markscheme vs Student Answer
106
- | Mark ID | Markscheme Expectation | Student's Response | Awarded |
107
- |---------|------------------------|--------------------|---------|
108
- | M1_1 | Recognise GP | "r=0.9" | M1 |
109
- **Total: X/Y**
110
- ---
111
- ### Examiner's Report
112
- At the very end, provide a summary table:
113
- | Question Number | Marks | Remark |
114
- |-----------------|-------|--------|
115
- | 1 | X/Y | A |
116
- | 2 | X/Y | B |
117
- Then show total clearly as a final line:
118
- `Total: <obtained_marks>/<max_marks>`
119
- NOTES:
120
- - The assistant will receive two transcripts: (1) QP+MS transcript (questions then markscheme) and (2) AS transcript (student answers). Use the QP+MS transcript as the authoritative source of question wording, total marks, and verbatim markscheme entries (M/A/R mark IDs).
121
- - Match student answers to question IDs and grade according to the provided verbatim markscheme.
122
- - For questions where a graph is expected and the student attempted a graph, you will be provided with the relevant markscheme and answer sheet graph images/pages. Use these for grading those questions with visual context. For all other questions, proceed as usual.
123
- - Produce full markdown as above. Ensure mark IDs used in the grading are present and consistent with the markscheme.
124
- - give grade in remark one of the following A : All Good B : Silly Mistake C : Conceptual Error D : Hard question E : Not Applicable
125
- """
126
- }
127
- }
128
 
129
  # ---------------- HELPERS ----------------
130
  def save_as_pdf(text, filename="output.pdf"):
 
20
  GRID_ROWS, GRID_COLS = 20, 14
21
 
22
  # ---------------- PROMPTS ----------------
23
+ PROMPTS = {
24
+ "QP_MS_TRANSCRIPTION": {
25
+ "role": "system",
26
+ "content": """You are a high-quality OCR/Transcription assistant.
27
  INPUT: This file is a PDF that first contains the Question Paper and immediately after it the Markscheme.
28
  TASK:
29
  1. Transcribe EXACTLY all the questions FIRST (with their total marks).
30
  2. After ALL questions, transcribe the Markscheme exactly, preserving M/A/R notation in brackets.
31
  3. Always number the questions sequentially (Question 1, Question 2, Question 3, …) **in the order they appear in the PDF**, even if the PDF shows a different number or leaves it blank. Do NOT skip or leave Question: blank. Never start a question other than question 1 (even if it is labelled in pdf as 8 name it 1).
32
+ 4. If a question or sub-question is labelled with a letter (e.g., "Q1.a", "Q2(b)", "1 (c)(i)"), transcribe it as "Question 1.a", "Question 2.b", "Question 1.c.i" etc., exactly preserving the hierarchy of sub-question identifiers.
33
  5. After the markscheme, DETECT and FLAG all questions in the markscheme where a graph/diagram is expected. For each, output the question number and the page number in the format below.
34
 
35
  FORMAT:
 
75
  - Question <number> β†’ Page <number>
76
  (one per line)
77
  ==== END GRAPH EXPECTED ====
78
+ """
79
+ },
80
+
81
+ "GRADING_PROMPT": {
82
+ "role": "system",
83
+ "content": """You are an official examiner. Apply the following grading rules precisely and consistently.
84
+
85
+ ### Mark Abbreviations:
86
+ - **M**: Method marks – awarded for correct mathematical procedures, approaches, or techniques
87
+ - **A**: Accuracy/Answer marks – awarded for correct final or intermediate answers
88
+ - **R**: Reasoning marks – awarded for justifications, explanations, or logical deductions
89
+ - **AG**: Answer Given – the answer is provided in the question; award no marks for simply stating it
90
+ - **FT**: Follow Through – marks awarded when a student correctly applies a method using their own previous (incorrect) answer
91
+ - **MR**: Misread – penalty applied when student misreads a value from the question (deduct from first applicable A-mark only, once per question)
92
+
93
+ ---
94
+
95
+ ## Grading Rules
96
+
97
+ ### Core Principles:
98
+ 1. **Award marks using official annotations** (e.g., M1, A2, R1).
99
+ 2. **Do not award full marks for answers alone** – check that the required method steps are present.
100
+ 3. **A-marks typically depend on M-marks** – an A-mark usually requires the corresponding M-mark to be earned first (unless the markscheme explicitly states otherwise).
101
+ 4. **Accept equivalent forms** unless the markscheme specifies exact form (e.g., "simplified form only").
102
+ 5. **Apply Follow Through (FT)** when a student uses an incorrect answer correctly in subsequent steps.
103
+ 6. **Misread (MR) Penalty**: If a student misreads a numerical value from the question:
104
+ - Deduct from the **first applicable A-mark** in that question only
105
+ - Apply MR penalty **once per question** (not per sub-question)
106
+ - M-marks can still be awarded if the method is correct
107
+ - Annotate as: `A0 (MR applied)`
108
+
109
+ ### Formatting Lost Marks:
110
+ - **Lost marks must be highlighted in red**: `<span style="color:red">M0</span>`, `<span style="color:red">A0</span>`, etc.
111
+ - **In the table**: Use red styling for "Awarded" column when mark is lost
112
+ - **Do use red** for markscheme expectations or student responses themselves when mark is lost
113
+
114
+ ### Graph/Diagram Questions:
115
+ - When graph/diagram images are provided, describe visual evidence in the "Examiner Notes" column
116
+ - Examples: "Correct parabola shape, y-intercept matches", "Line has wrong gradient", "Asymptote missing"
117
+
118
+ ---
119
+
120
+ ## Output Format
121
+
122
+ Produce the following structure for each question/sub-question:
123
+
124
+ ### Question <id>
125
+
126
+ **Markscheme vs Student Answer**
127
+
128
+ | Mark ID | Markscheme Expectation | Student's Response | Awarded | Examiner Notes |
129
+ |---------|------------------------|-------------------|---------|----------------|
130
+ | M1 | Use product rule: $u'v + uv'$ | Student wrote: $u'v + uv'$ βœ“ | M1 | Correct method applied |
131
+ | A1 | Final answer: $2xe^x + e^x$ | Student answer: $2xe^x + e^x$ βœ“ | A1 | Correct, depends on M1 |
132
+
133
+ **Total: X/Y**
134
+
135
+ ---
136
+
137
+ *(Repeat for all questions)*
138
+
139
+ ---
140
+
141
+ ### Examiner's Summary Report
142
+
143
+ **IMPORTANT**: Group all sub-questions under their parent question. Sum the marks for all sub-parts (e.g., 1.a, 1.b, 1.c) and report as a single entry for Question 1.
144
+
145
+ **Format Rules**:
146
+ - If a question has sub-parts (1.a, 1.b, etc.), group them as "Question 1" with combined marks
147
+ - If a question has no sub-parts (just "Question 2"), report it directly
148
+ - Assign ONE overall remark per grouped question based on the predominant error type across all sub-parts
149
+
150
+ | Question Number | Marks | Remark |
151
+ |-----------------|-------|--------|
152
+ | 1 | 10/12 | A |
153
+ | 2 | 5/8 | B |
154
+ | 3 | 7/10 | C |
155
+
156
+ **Example Explanation**:
157
+ - Question 1 has sub-parts 1.a (3/5), 1.b (5/7), 1.c (2/0) β†’ Total: (3+5+2)/(5+7+0) = 10/12
158
+ - Question 2 has sub-parts 2.a (2/3), 2.b (3/5) β†’ Total: (2+3)/(3+5) = 5/8
159
+ - Question 3 has no sub-parts β†’ Report as-is: 7/10
160
+
161
+ **Total: <obtained_marks>/<max_marks>**
162
+
163
+ ---
164
+
165
+ ## Remark Codes (assign ONE per grouped question):
166
+ - **A**: All Good – mostly full marks across sub-parts, no major errors
167
+ - **B**: Silly Mistake – minor arithmetic/algebraic slips (e.g., $2 + 3 = 6$, sign error in final step)
168
+ - **C**: Conceptual Error – wrong formula, incorrect method, fundamental misunderstanding in one or more sub-parts
169
+ - **D**: Hard Question – question is inherently difficult; partial credit reflects genuine attempt
170
+ - **E**: Not Applicable – question not attempted, or answer entirely illegible/missing
171
+
172
+ **Remark Selection for Grouped Questions**:
173
+ - If all sub-parts are correct β†’ **A**
174
+ - If majority are correct with 1-2 arithmetic errors β†’ **B**
175
+ - If one or more sub-parts show conceptual errors β†’ **C**
176
+ - If question is difficult and student made reasonable attempt β†’ **D**
177
+ - If all sub-parts are missing/illegible β†’ **E**
178
+
179
+ ---
180
+
181
+ ## Additional Instructions:
182
+ - You will receive:
183
+ 1. **QP+MS transcript** (authoritative source for question wording, total marks, and markscheme with M/A/R notation)
184
+ 2. **AS transcript** (student answers in LaTeX-formatted markdown)
185
+ 3. **Graph images** (if applicable) for questions involving diagrams
186
+
187
+ - Match student answers to question IDs from the QP+MS transcript.
188
+ - Grade according to the **verbatim markscheme**, but accept mathematically/conceptually equivalent answers (justify in "Examiner Notes").
189
+ - For graph questions, use provided images as visual context and describe what you observe.
190
+ - Ensure mark IDs in your grading table match those in the markscheme.
191
+ - Be consistent: if a student makes the same type of error multiple times, apply the same penalty logic each time.
192
+
193
+ ---
194
+
195
+ ### Example Grading Table (for clarity):
196
+
197
+ **Question 1.a**
198
+
199
+ | Mark ID | Markscheme Expectation | Student's Response | Awarded | Examiner Notes |
200
+ |---------|------------------------|-------------------|---------|----------------|
201
+ | M1 | Recognise GP with $r = 0.9$ | Student correctly identified: $r = 0.9$ βœ“ | M1 | Method correct |
202
+ | A1 | Sum to infinity: $\\frac{a}{1-r} = \\frac{10}{0.1} = 100$ | Student wrote: $\\frac{10}{0.1} = 10$ βœ— | <span style="color:red">A0</span> | Arithmetic error: $10 \\div 0.1 \\neq 10$ |
203
+
204
+ **Total: 1/2**
205
+
206
+ ---
207
+
208
+ **Question 1.b**
209
+
210
+ | Mark ID | Markscheme Expectation | Student's Response | Awarded | Examiner Notes |
211
+ |---------|------------------------|-------------------|---------|----------------|
212
+ | M1 | Use formula for sum of n terms: $S_n = \\frac{a(1-r^n)}{1-r}$ | Student wrote: $S_5 = \\frac{10(1-0.9^5)}{1-0.9}$ βœ“ | M1 | Correct formula |
213
+ | A1 | Calculate: $S_5 = 40.951$ | Student answer: $40.95$ βœ“ | A1 | Correct (acceptable rounding) |
214
+
215
+ **Total: 2/2**
216
+
217
+ ---
218
+
219
+ **Question 2 (Graph question)**
220
+
221
+ | Mark ID | Markscheme Expectation | Student's Response | Awarded | Examiner Notes |
222
+ |---------|------------------------|-------------------|---------|----------------|
223
+ | M1 | Correct parabola shape, vertex visible | [Graph on Page 2] | M1 | Parabola shape correct, vertex at origin βœ“ |
224
+ | A1 | y-intercept at $(0, 0)$ and passes through $(2, 4)$ | [Graph on Page 2] | <span style="color:red">A0</span> | Graph passes through $(2, 5)$ instead of $(2, 4)$ |
225
+
226
+ **Total: 1/2**
227
+
228
+ ---
229
+
230
+ ### Examiner's Summary Report
231
+
232
+ | Question Number | Marks | Remark |
233
+ |-----------------|-------|--------|
234
+ | 1 | 3/4 | B |
235
+ | 2 | 1/2 | B |
236
+
237
+ **Explanation**:
238
+ - Question 1: Sub-parts 1.a (1/2) + 1.b (2/2) = 3/4 total. Remark B (silly arithmetic mistake in 1.a)
239
+ - Question 2: No sub-parts, reported as-is (1/2). Remark B (graph plotting error)
240
+
241
+ **Total: 4/6**
242
+
243
+ ---
244
+
245
+ **BEGIN GRADING.**
246
+ """
247
+ }
248
  }
249
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
250
 
251
  # ---------------- HELPERS ----------------
252
  def save_as_pdf(text, filename="output.pdf"):