n rm

README.md

@@ -1,232 +1,226 @@
 # AI Course Assessment Generator
 
-This application generates learning objectives and multiple-choice questions for AI course materials based on uploaded content files. It uses OpenAI's language models to create high-quality educational assessments that adhere to specified quality standards.
+An AI-powered tool that creates learning objectives and multiple-choice quiz questions from course materials. Supports both **automatic generation** from uploaded content and **manual entry** of learning objectives, producing fully enriched outputs with correct and incorrect answer suggestions ready for quiz generation.
+
+---
 
 ## Features
 
-- Upload course materials in various formats (.vtt, .srt, .ipynb)
-- Generate customizable number of learning objectives
-- Create multiple-choice questions based on learning objectives
-- Evaluate question quality using an LLM judge
-- Save assessments to JSON format
-- Track source references for each learning objective and question
+### Tab 1 — Generate Learning Objectives
 
-## Setup
+**Two modes of operation:**
 
-1. Clone this repository
-2. Install the required dependencies:
-   ```
-   pip install -r requirements.txt
-   ```
-3. Create a `.env` file in the project root with your OpenAI API key:
-   ```
-   OPENAI_API_KEY=your_api_key_here
-   ```
-
-## Usage
-
-1. Run the application:
-   ```
-   python app.py
-   ```
-2. Open the Gradio interface in your web browser (typically at http://127.0.0.1:7860)
-3. Upload your course materials (.vtt, .srt, .ipynb files)
-4. Specify the number of learning objectives to generate
-5. Select the OpenAI model to use
-6. Generate learning objectives
-7. Review and provide feedback on the generated objectives
-8. Generate multiple-choice questions based on the approved objectives
-9. Review the generated questions and their quality assessments
-10. The final assessment will be saved as `assessment.json` in the project directory
-
-## Project Structure
-
-- `app.py`: Entry point for the application
-
-### Modules
-
-- `models/`: Pydantic data models
-  - `__init__.py`: Exports all models
-  - `learning_objectives.py`: Learning objective data models
-  - `questions.py`: Question and option data models
-  - `assessment.py`: Assessment data models
-
-- `ui/`: User interface components
-  - `__init__.py`: Package initialization
-  - `app.py`: Gradio UI implementation
-  - `content_processor.py`: Processes uploaded files and extracts content
-  - `objective_handlers.py`: Handlers for learning objective generation
-  - `question_handlers.py`: Handlers for question generation
-  - `feedback_handlers.py`: Handlers for feedback and regeneration
-  - `formatting.py`: Formatting utilities for UI display
-  - `state.py`: State management for the UI
-
-- `quiz_generator/`: Quiz generation components
-  - `__init__.py`: Package initialization
-  - `generator.py`: Main QuizGenerator class
-  - `assessment.py`: Assessment generation logic
-  - `question_generation.py`: Question generation logic
-  - `question_improvement.py`: Question quality improvement logic
-  - `question_ranking.py`: Question ranking and grouping logic
-  - `feedback_questions.py`: Feedback-based question generation
-
-- `learning_objective_generator/`: Learning objective generation components
-  - `__init__.py`: Package initialization
-  - `generator.py`: Main generator class
-  - `base_generation.py`: Base generation logic
-  - `enhancement.py`: Enhancement logic
-  - `grouping_and_ranking.py`: Grouping and ranking logic
-
-- `prompts/`: Prompt templates and components
-  - `questions.py`: Question generation prompts
-  - `incorrect_answers.py`: Incorrect answer generation prompts
-  - `learning_objectives.py`: Learning objective generation prompts
-
-- `obsolete/`: Deprecated files (not used in current implementation)
-
-- `specs.md`: Project specifications
-- `project_flow.md`: Detailed description of the project architecture and workflow
+- **Generate from course materials** — Upload course files and let the AI extract and generate learning objectives automatically through a multi-run, multi-stage pipeline.
+- **Use my own learning objectives** — Enter your own learning objectives in a text field (one per line). The app searches the uploaded course materials for relevant source references, generates a correct answer for each objective, and produces incorrect answer options — the same full pipeline as automatic generation.
 
-## Requirements
+**Shared capabilities (both modes):**
+- Configurable AI model and temperature for both generation and incorrect answer suggestion steps
+- All output in the same JSON format, ready to feed directly into Tab 2
+- "Generate all" button runs the full end-to-end pipeline (learning objectives → quiz questions) in a single click, in either mode
+
+### Tab 2 — Generate Questions
+
+- Takes the learning objectives JSON produced in Tab 1 as input
+- Generates multiple-choice questions with 4 options, per-option feedback, and source references
+- Configurable number of questions and generation runs
+- Automatic ranking and grouping of generated questions by quality
+- Outputs: ranked best-in-group questions, all grouped questions, and a human-readable formatted quiz
+
+### Tab 3 — Propose / Edit Question
+
+- Load the formatted quiz from Tab 2 or upload a `.md` / `.yml` quiz file
+- Review and edit questions one at a time with Previous / Accept & Next navigation
+- Download the final edited quiz
+
+---
+
+## Generation Pipeline (Learning Objectives)
 
-- Python 3.8+
-- Gradio 4.19.2+
-- Pydantic 2.8.0+
-- OpenAI 1.52.0+
-- nbformat 5.9.2+
-- instructor 1.7.9+
-- python-dotenv 1.0.0+
+### Automatic generation mode
 
-Install dependencies using uv (recommended):
+1. **Content extraction** — Uploads are parsed (`.vtt`, `.srt`, `.ipynb`, `.md`) and wrapped with XML source tags for full traceability
+2. **Multi-run base generation** — Multiple independent runs produce candidate objectives (Bloom's taxonomy aware, one action verb, multiple-choice assessable)
+3. **Correct answer generation** — A concise correct answer (~20 words) is generated for each objective from the course content
+4. **Grouping & ranking** — Similar objectives are clustered; the best representative in each group is selected
+5. **Incorrect answer generation** — Three plausible distractors are generated for each best-in-group objective, matching the correct answer in length, style, and complexity
+6. **Iterative improvement** — Each distractor is evaluated and regenerated until it meets quality standards
+
+### User-provided objectives mode
+
+1. **Objective parsing** — Text is split by newlines; common leading labels are stripped automatically:
+   - Numbered: `1.`, `2)`, `3:`
+   - Lettered: `a.`, `b)`, `c:`
+   - Plain (no label)
+2. **Source finding** — For each objective, the LLM searches the uploaded course materials to identify the most relevant source file(s)
+3. **Correct answer generation** — Same function as the automatic flow, grounded in the course content
+4. **Incorrect answer generation** — Same three-distractor generation as automatic flow
+5. **Iterative improvement** — Same quality improvement loop
+6. All objectives are treated as best-in-group (the user has already curated them), so no grouping/filtering step is applied
+
+**Example accepted input formats:**
 ```
-uv venv -p 3.12
-source .venv/bin/activate  # On Windows use: .venv\Scripts\activate
-uv pip install -r requirements.txt
+Identify key upstream and downstream collaborators for data engineers
+Identify the stages of the data engineering lifecycle
+Articulate a mental framework for building data engineering solutions
+```
+```
+1. Identify key upstream and downstream collaborators for data engineers
+2. Identify the stages of the data engineering lifecycle
+3. Articulate a mental framework for building data engineering solutions
+```
+```
+a. Identify key upstream and downstream collaborators for data engineers
+b. Identify the stages of the data engineering lifecycle
+c. Articulate a mental framework for building data engineering solutions
 ```
 
-## Notes
-
-- The application uses XML-style source tags to track which file each piece of content comes from
-- Questions are evaluated against quality standards to ensure they meet educational requirements
-- Each question includes feedback for both correct and incorrect answers
+---
 
-## Prompt Structure
+## Setup
 
-The application's prompt system in `prompts.py` has been refactored into modular components for better maintainability:
+### Prerequisites
 
-- `GENERAL_QUALITY_STANDARDS`: Overall quality standards for all generated content
-- `QUESTION_SPECIFIC_QUALITY_STANDARDS`: Standards specific to question generation
-- `CORRECT_ANSWER_SPECIFIC_QUALITY_STANDARDS`: Standards for correct answer options
-- `INCORRECT_ANSWER_SPECIFIC_QUALITY_STANDARDS`: Standards for creating plausible incorrect answers
-- `EXAMPLE_QUESTIONS`: A collection of high-quality example questions for model guidance
-- `MULTIPLE_CHOICE_STANDARDS`: Standards specific to multiple-choice question format
-- `BLOOMS_TAXONOMY_LEVELS`: Educational taxonomy for different levels of learning
-- `ANSWER_FEEDBACK_QUALITY_STANDARDS`: Standards for providing helpful feedback
-- `LEARNING_OBJECTIVES_PROMPT`: Template for generating learning objectives
-- `LEARNING_OBJECTIVE_EXAMPLES`: Examples of well-formulated learning objectives
+- Python 3.12 (recommended) or 3.8+
+- An OpenAI API key
 
-These components are imported and combined in `quiz_generator.py` to create comprehensive prompts for different generation tasks. This modular approach makes it easier to:
+### Installation
 
-1. Update individual aspects of the prompt without affecting others
-2. Reuse common standards across different generation tasks
-3. Maintain consistent quality across all generated content
+**Using uv (recommended):**
+```bash
+uv venv -p 3.12
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+uv pip install -r requirements.txt
+```
 
-## Detailed Project Flow
+**Using pip:**
+```bash
+pip install -r requirements.txt
+```
 
-### Overview
+### Environment variables
 
-This section provides a more detailed look at how the various components of the system work together to generate educational assessments.
+Create a `.env` file in the project root:
+```
+OPENAI_API_KEY=your_api_key_here
+```
 
-### Core Components
+---
 
-1. **Content Processing**: Handles ingestion of course materials from different file formats
-2. **Learning Objective Generation**: Creates learning objectives from the processed content
-3. **Question Generation**: Produces multiple-choice questions for each learning objective
-4. **Quality Assessment**: Evaluates the generated questions for quality
-5. **UI Interface**: Provides a Gradio-based web interface for user interaction
+## Running the app
 
-### Application Entry Point (`app.py`)
+```bash
+python app.py
+```
 
-- Serves as the entry point for the application
-- Loads environment variables (including OpenAI API key)
-- Creates and launches the Gradio UI
+Opens the Gradio interface at [http://127.0.0.1:7860](http://127.0.0.1:7860).
 
-### User Interface (`ui/` module)
+---
 
-- Creates the Gradio interface for user interaction
-- Organizes functionality into tabs:
-  - File upload and learning objective generation
-  - Question generation
-  - Preview and export
+## Supported file formats
 
-- Key components:
-  - `app.py`: Creates the Gradio interface and defines the UI layout
-  - `objective_handlers.py`: Handles learning objective generation and regeneration
-  - `question_handlers.py`: Handles question generation and regeneration
-  - `feedback_handlers.py`: Handles user feedback and custom question generation
-  - `formatting.py`: Formats quiz data for UI display
-  - `state.py`: Manages state between UI components
+| Format | Description |
+|--------|-------------|
+| `.vtt` | WebVTT subtitle files (timestamps stripped) |
+| `.srt` | SRT subtitle files (timestamps stripped) |
+| `.ipynb` | Jupyter notebooks (markdown and code cells extracted) |
+| `.md` | Markdown files |
 
-### Content Processing (`ui/content_processor.py`)
+All content is wrapped with XML source tags (`<source file="filename">…</source>`) so every generated objective and question can be traced back to its origin file.
 
-- `ContentProcessor` class processes different file types:
-  - `.vtt` and `.srt` subtitle files
-  - `.ipynb` Jupyter notebook files
-- For each file, adds XML source tags to track the origin of content
-- Returns structured content for further processing
+---
 
-### Quiz Generation (`quiz_generator/` module)
+## Project structure
 
-- `QuizGenerator` class is the central component that:
-  - Generates learning objectives from processed content
-  - Creates multiple-choice questions for each objective
-  - Judges question quality
-  - Saves assessments to JSON
+```
+quiz_generator_ECM/
+│
+├── app.py                              # Entry point — loads .env and launches Gradio
+│
+├── models/                             # Pydantic data models
+│   ├── learning_objectives.py          # BaseLearningObjective → LearningObjective → Grouped*
+│   ├── questions.py                    # MultipleChoiceQuestion → Ranked* → Grouped*
+│   ├── assessment.py                   # Assessment (objectives + questions)
+│   └── config.py                       # Model list and temperature availability map
+│
+├── prompts/                            # Reusable prompt components
+│   ├── learning_objectives.py          # Bloom's taxonomy, quality standards, examples
+│   ├── incorrect_answers.py            # Distractor guidelines and examples
+│   ├── questions.py                    # Question and answer quality standards
+│   └── all_quality_standards.py        # General quality standards
+│
+├── learning_objective_generator/       # Learning objective pipeline
+│   ├── generator.py                    # LearningObjectiveGenerator orchestrator
+│   ├── base_generation.py              # Base generation, correct answers, source finding
+│   ├── enhancement.py                  # Incorrect answer generation
+│   ├── grouping_and_ranking.py         # Similarity grouping and best-in-group selection
+│   └── suggestion_improvement.py       # Iterative distractor quality improvement
+│
+├── quiz_generator/                     # Question generation pipeline
+│   ├── generator.py                    # QuizGenerator orchestrator
+│   ├── question_generation.py          # Multiple-choice question generation
+│   ├── question_improvement.py         # Question quality assessment and improvement
+│   ├── question_ranking.py             # Ranking and grouping of questions
+│   ├── feedback_questions.py           # Feedback-based question regeneration
+│   └── assessment.py                   # Assessment compilation and export
+│
+└── ui/                                 # Gradio interface and handlers
+    ├── app.py                          # UI layout, mode toggle, event wiring
+    ├── objective_handlers.py           # Handlers for both objective modes + Generate all
+    ├── question_handlers.py            # Question generation handler
+    ├── content_processor.py            # File parsing and XML source tagging
+    ├── edit_handlers.py                # Question editing flow (Tab 3)
+    ├── formatting.py                   # Quiz formatting for UI display
+    ├── state.py                        # Global state (file contents, objectives)
+    └── run_manager.py                  # Run tracking and output saving
+```
 
-#### Learning Objective Generation
+---
 
-1. Takes processed file contents as input
-2. Combines content and creates a prompt (utilizing modular components from `prompts.py`)
-3. Uses OpenAI's API with instructor to generate learning objectives
-4. Returns structured `LearningObjective` objects
+## Data models
 
-#### Question Generation
+Learning objectives progress through these stages:
 
-1. For each learning objective:
-   - Retrieves relevant content from source files
-   - Creates a prompt by combining modular components from `prompts.py`
-   - Generates a multiple-choice question with feedback for each option
-   - Returns a structured `MultipleChoiceQuestion` object
+```
+BaseLearningObjectiveWithoutCorrectAnswer
+  └─ id, learning_objective, source_reference
+      ↓
+BaseLearningObjective
+  └─ + correct_answer
+      ↓
+LearningObjective  (output of Tab 1, input to Tab 2)
+  └─ + incorrect_answer_options, in_group, group_members, best_in_group
+```
 
-### Data Models (`models/` module)
+Questions follow an equivalent progression:
 
-Defines the data structures used throughout the application:
-- `LearningObjective`: Represents a learning objective with ID, text, and source references
-- `MultipleChoiceOption`: Represents an answer option with text, correctness flag, and feedback
-- `MultipleChoiceQuestion`: Represents a complete question with options, linked to learning objectives
-- `RankedMultipleChoiceQuestion`: Extends MultipleChoiceQuestion with ranking information
-- `GroupedMultipleChoiceQuestion`: Extends RankedMultipleChoiceQuestion with grouping information
-- `Assessment`: Collection of learning objectives and questions
+```
+MultipleChoiceQuestion
+  └─ id, question_text, options (text + is_correct + feedback),
+     learning_objective_id, correct_answer, source_reference
+      ↓
+RankedMultipleChoiceQuestion
+  └─ + rank, ranking_reasoning, in_group, group_members, best_in_group
+```
 
-### Prompt Component Integration
+---
 
-The modular prompt components in the `prompts/` directory are imported into the quiz generation modules and assembled into complete prompts as needed:
+## Model configuration
 
-1. **Learning Objective Generation**:
-   - Components like `LEARNING_OBJECTIVES_PROMPT`, `LEARNING_OBJECTIVE_EXAMPLES`, and `BLOOMS_TAXONOMY_LEVELS` are combined with course content
-   - This creates a comprehensive prompt that guides the LLM in generating relevant and well-structured learning objectives
+Default model: `gpt-5.2`
+Default temperature: `1.0` (ignored for models that do not support it, such as `o1`, `o3-mini`, `gpt-5`, `gpt-5.1`, `gpt-5.2`)
 
-2. **Question Generation**:
-   - Components like `GENERAL_QUALITY_STANDARDS`, `MULTIPLE_CHOICE_STANDARDS`, `QUESTION_SPECIFIC_QUALITY_STANDARDS`, etc. are combined
-   - Along with the learning objective and course content, these form a detailed prompt that ensures high-quality question generation
+You can set different models for the main generation step and the incorrect answer suggestion step, which is useful for using a more creative model for distractors.
 
-### Workflow Summary
+---
 
-1. User uploads content files (notebooks, subtitles) through the UI
-2. System processes files and extracts content with source references
-3. LLM generates learning objectives based on content
-4. User reviews and approves learning objectives
-5. System generates multiple-choice questions for each approved objective
-6. Questions are presented to the user for review and export
+## Requirements
 
-This modular approach makes it easier to maintain, update, and experiment with different prompt components without disrupting the overall system. Any changes to the components in `prompts.py` will affect how learning objectives and questions are generated, potentially changing the style, format, and quality of the output.
+| Package | Version |
+|---------|---------|
+| Python | 3.8+ (3.12 recommended) |
+| gradio | 4.19.2+ |
+| pydantic | 2.8.0+ |
+| openai | 1.52.0+ |
+| nbformat | 5.9.2+ |
+| instructor | 1.7.9+ |
+| python-dotenv | 1.0.0+ |