QUESTION_CATEGORIZATION.md

ClassLens — Question Categorization & Bank System

Overview

When a teacher uploads a questions PDF, ClassLens automatically classifies every question into a pre-defined taxonomy using an LLM. This classification feeds three downstream features:

1. Richer student reports — wrong-answer blocks show the specific grammar/vocabulary tag that was tested, and the Weakness Summary groups errors by tag.
2. Shared question bank — classified questions accumulate across all uploads and all teachers, organized by category and tag.
3. Practice question generation — teachers can generate new questions by selecting a category and tag; the system rephrases questions already in the bank rather than hallucinating new ones.

---

Taxonomy

The taxonomy has 6 main categories. Two of them (字詞理解 and 語法結構) have sub-tags; the other four are used as-is.

字詞理解 (Vocabulary & Word Usage)

Sub-group	Tags
詞性	形容詞用法, 情狀副詞, 頻率副詞, 數量形容詞, 介系詞
花費動詞	spend/cost/take/pay 用法
字彙	動詞與動詞片語, 名詞與名詞片語, 形容詞與副詞, 介系詞與連接詞片語, 情境字彙推斷

語法結構 (Grammar Structures)

Sub-group	Tags
時態	現在簡單式, 現在簡單式第三人稱單數, 現在進行式, 現在完成式, 過去式be動詞, 過去簡單式, 過去進行式, 過去完成式, 未來式
語態	被動語態
代名詞	人稱代名詞主格, 人稱代名詞受格, 所有格代名詞, 反身代名詞, 不定代名詞, 指示代名詞
連接詞	對等連接詞, when/before/after, if/although, why/because/so, as long as/as soon as, when/while, 相關連接詞
關係子句	關係代名詞, 關係代名詞省略
動詞型態	動名詞, 不定詞, 感官動詞, 使役動詞, 連綴動詞, 授與動詞, 助動詞, used to
句型結構	祈使句, 附加問句, 附和句, 間接問句, 比較級, 最高級, There be 句型, It takes 句型, Too...to 句型, So...that 句型, Enough...to 句型, 動名詞當主詞, 虛主詞it
疑問句型	Who/What/Where/When/Which/How/Why/How often/How much/How long 問答句

文意推論 (Inference)

No sub-tags. Covers questions that require reading between the lines.

篇章大意 (Main Idea)

No sub-tags. Covers questions about the overall topic or gist of a passage.

篇章細節 (Reading for Detail)

No sub-tags. Covers questions about specific facts stated in a passage.

篇章結構 (Text Organization)

No sub-tags. Covers questions about how a passage is organized (e.g., ordering paragraphs, identifying transitions).

---

How Categorization Works

Trigger

Categorization runs once per questions PDF upload, in the background after the upload response is returned to the client. It does not block the upload or report generation.

Flow

Teacher uploads questions PDF
        │
        ▼
Parser extracts [{number, text}, ...] from PDF
        │
        ▼
process_uploaded_files() saves rows to ParsedData (main_category = "", tags = [])
        │
        ▼
Upload response returned to client immediately
        │
        ▼  (asyncio.create_task — background)
categorize_questions(session_id, teacher_id, questions)
        │
        ├── Builds LLM prompt with full taxonomy + all question texts
        ├── Calls gpt-4o-mini → returns JSON [{question_num, main_category, tags}, ...]
        ├── Validates: main_category must be one of the 6; tags must match that category
        ├── update_parsed_data_categories() — writes main_category + tags to ParsedData rows
        └── save_question_bank_batch() — inserts into shared question_bank table

If the LLM call or DB write fails, the error is logged and the upload is unaffected.

Files

File	Purpose
chatkit/backend/app/taxonomy.py	Single source of truth for all categories and tags
chatkit/backend/app/question_categorizer.py	LLM categorization + DB writes
chatkit/backend/app/models.py	QuestionBank ORM model, ParsedData columns main_category + tags
chatkit/backend/app/database.py	update_parsed_data_categories, save_question_bank_batch, query_question_bank
chatkit/backend/alembic/versions/0003_question_bank.py	Migration: creates the question_bank table

---

Question Bank

The question_bank table is shared across all teachers. Questions persist even if the original quiz or teacher account is deleted (FK uses ON DELETE SET NULL).

Schema

Column	Type	Notes
id	integer PK
quiz_id	integer, nullable	Source quiz (SET NULL on delete)
teacher_id	integer, nullable	Uploading teacher (SET NULL on delete)
question_text	text	Full question as extracted from PDF
answer	varchar(10)	Correct answer (may be empty if uploaded without answer key)
main_category	varchar(64), indexed	One of the 6 taxonomy categories
tags	JSON array	Tags from the category's sub-list
created_at	timestamptz

Querying the bank

query_question_bank(main_category, tags, limit) in database.py:

• Fetches up to limit×3 candidates by category (most recent first)
• If tags are specified, filters in Python to rows that share at least one tag
• Falls back to category-only results if no tag match is found

---

Practice Question Generation

When a teacher requests practice questions for a category/tag combination:

1. query_question_bank(category, tags) fetches matching source questions from the bank.
2. If no bank questions match the filter, the endpoint returns an empty list — no AI fallback.
3. If matches are found, gpt-4o-mini is asked to rephrase the source questions (same grammar point, varied vocabulary and context) — not to generate new topics.

This ensures generated questions always test real skills from past exams and never drift outside the taxonomy.

---

How Categories Appear in Reports

When a student report is generated, main.py fetches the ParsedData rows for the session and builds a categories dict (question_num → (main_category, tags)). This is passed to generate_student_report.

Each wrong-answer block in the LLM prompt includes a 分類 line:

**分類**: 語法結構 > 時態, 過去簡單式

Or for tag-free categories:

**分類**: 篇章細節

The Weakness Summary section of the report instructs the LLM to:

• Name the specific tags that recurred most (e.g., 過去簡單式, 被動語態)
• Fall back to the category name for the four tag-free categories
• Group errors by these tags — no question numbers, no vague summaries