QUESTION_CATEGORIZATION.md

# ClassLens — Question Categorization & Bank System

## Overview

When a teacher uploads a questions PDF, ClassLens automatically classifies every question into a pre-defined taxonomy using an LLM. This classification feeds three downstream features:

1. **Richer student reports** — wrong-answer blocks show the specific grammar/vocabulary tag that was tested, and the Weakness Summary groups errors by tag.
2. **Shared question bank** — classified questions accumulate across all uploads and all teachers, organized by category and tag.
3. **Practice question generation** — teachers can generate new questions by selecting a category and tag; the system rephrases questions already in the bank rather than hallucinating new ones.

---

## Taxonomy

The taxonomy has **6 main categories**. Two of them (字詞理解 and 語法結構) have sub-tags; the other four are used as-is.

### 字詞理解 (Vocabulary & Word Usage)

| Sub-group | Tags |
|---|---|
| 詞性 | 形容詞用法, 情狀副詞, 頻率副詞, 數量形容詞, 介系詞 |
| 花費動詞 | spend/cost/take/pay 用法 |
| 字彙 | 動詞與動詞片語, 名詞與名詞片語, 形容詞與副詞, 介系詞與連接詞片語, 情境字彙推斷 |

### 語法結構 (Grammar Structures)

| Sub-group | Tags |
|---|---|
| 時態 | 現在簡單式, 現在簡單式第三人稱單數, 現在進行式, 現在完成式, 過去式be動詞, 過去簡單式, 過去進行式, 過去完成式, 未來式 |
| 語態 | 被動語態 |
| 代名詞 | 人稱代名詞主格, 人稱代名詞受格, 所有格代名詞, 反身代名詞, 不定代名詞, 指示代名詞 |
| 連接詞 | 對等連接詞, when/before/after, if/although, why/because/so, as long as/as soon as, when/while, 相關連接詞 |
| 關係子句 | 關係代名詞, 關係代名詞省略 |
| 動詞型態 | 動名詞, 不定詞, 感官動詞, 使役動詞, 連綴動詞, 授與動詞, 助動詞, used to |
| 句型結構 | 祈使句, 附加問句, 附和句, 間接問句, 比較級, 最高級, There be 句型, It takes 句型, Too...to 句型, So...that 句型, Enough...to 句型, 動名詞當主詞, 虛主詞it |
| 疑問句型 | Who/What/Where/When/Which/How/Why/How often/How much/How long 問答句 |

### 文意推論 (Inference)
No sub-tags. Covers questions that require reading between the lines.

### 篇章大意 (Main Idea)
No sub-tags. Covers questions about the overall topic or gist of a passage.

### 篇章細節 (Reading for Detail)
No sub-tags. Covers questions about specific facts stated in a passage.

### 篇章結構 (Text Organization)
No sub-tags. Covers questions about how a passage is organized (e.g., ordering paragraphs, identifying transitions).

---

## How Categorization Works

### Trigger
Categorization runs **once per questions PDF upload**, in the background after the upload response is returned to the client. It does not block the upload or report generation.

### Flow

```
Teacher uploads questions PDF
        │
        ▼
Parser extracts [{number, text}, ...] from PDF
        │
        ▼
process_uploaded_files() saves rows to ParsedData (main_category = "", tags = [])
        │
        ▼
Upload response returned to client immediately
        │
        ▼  (asyncio.create_task — background)
categorize_questions(session_id, teacher_id, questions)
        │
        ├── Builds LLM prompt with full taxonomy + all question texts
        ├── Calls gpt-4o-mini → returns JSON [{question_num, main_category, tags}, ...]
        ├── Validates: main_category must be one of the 6; tags must match that category
        ├── update_parsed_data_categories() — writes main_category + tags to ParsedData rows
        └── save_question_bank_batch() — inserts into shared question_bank table
```

If the LLM call or DB write fails, the error is logged and the upload is unaffected.

### Files
| File | Purpose |
|---|---|
| `chatkit/backend/app/taxonomy.py` | Single source of truth for all categories and tags |
| `chatkit/backend/app/question_categorizer.py` | LLM categorization + DB writes |
| `chatkit/backend/app/models.py` | `QuestionBank` ORM model, `ParsedData` columns `main_category` + `tags` |
| `chatkit/backend/app/database.py` | `update_parsed_data_categories`, `save_question_bank_batch`, `query_question_bank` |
| `chatkit/backend/alembic/versions/0003_question_bank.py` | Migration: creates the `question_bank` table |

---

## Question Bank

The `question_bank` table is **shared across all teachers**. Questions persist even if the original quiz or teacher account is deleted (FK uses `ON DELETE SET NULL`).

### Schema
| Column | Type | Notes |
|---|---|---|
| `id` | integer PK | |
| `quiz_id` | integer, nullable | Source quiz (SET NULL on delete) |
| `teacher_id` | integer, nullable | Uploading teacher (SET NULL on delete) |
| `question_text` | text | Full question as extracted from PDF |
| `answer` | varchar(10) | Correct answer (may be empty if uploaded without answer key) |
| `main_category` | varchar(64), indexed | One of the 6 taxonomy categories |
| `tags` | JSON array | Tags from the category's sub-list |
| `created_at` | timestamptz | |

### Querying the bank
`query_question_bank(main_category, tags, limit)` in `database.py`:
- Fetches up to `limit×3` candidates by category (most recent first)
- If tags are specified, filters in Python to rows that share at least one tag
- Falls back to category-only results if no tag match is found

---

## Practice Question Generation

When a teacher requests practice questions for a category/tag combination:

1. `query_question_bank(category, tags)` fetches matching source questions from the bank.
2. If no bank questions match the filter, the endpoint returns an empty list — no AI fallback.
3. If matches are found, `gpt-4o-mini` is asked to **rephrase** the source questions (same grammar point, varied vocabulary and context) — not to generate new topics.

This ensures generated questions always test real skills from past exams and never drift outside the taxonomy.

---

## How Categories Appear in Reports

When a student report is generated, `main.py` fetches the `ParsedData` rows for the session and builds a `categories` dict (`question_num → (main_category, tags)`). This is passed to `generate_student_report`.

Each wrong-answer block in the LLM prompt includes a **分類** line:
```
**分類**: 語法結構 > 時態, 過去簡單式
```
Or for tag-free categories:
```
**分類**: 篇章細節
```

The **Weakness Summary** section of the report instructs the LLM to:
- Name the specific **tags** that recurred most (e.g., 過去簡單式, 被動語態)
- Fall back to the **category name** for the four tag-free categories
- Group errors by these tags — no question numbers, no vague summaries