Initial commit: LLM Code Deployment System

This commit includes:
- Student API for receiving tasks and deploying to GitHub Pages
- Instructor system for task generation and evaluation
- Docker configuration for Hugging Face Spaces deployment
- AIPipe integration for cost-effective LLM access
- Comprehensive documentation for deployment and usage

Features:
- FastAPI-based REST APIs for student and instructor systems
- LLM-powered code generation (supports Anthropic, OpenAI, AIPipe)
- Automated GitHub repository creation and Pages deployment
- Multi-level evaluation (static, dynamic, LLM-based)
- PostgreSQL database for task and result tracking
- Playwright-based dynamic testing
- Round 1 and Round 2 task support

Documentation:
- README.md: Project overview and setup
- DEPLOYMENT.md: Step-by-step student deployment guide
- README_SPACES.md: Hugging Face Spaces configuration
- QUICKSTART.md: Quick start guide for local development

🚀 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

.env.example

@@ -0,0 +1,38 @@
+# Student Configuration
+STUDENT_SECRET=your-secret-key-here
+STUDENT_EMAIL=student@example.com
+STUDENT_API_PORT=8000
+
+# GitHub Configuration
+GITHUB_TOKEN=ghp_your_github_personal_access_token
+GITHUB_USERNAME=your-github-username
+
+# LLM Configuration (Choose one)
+# For AIPipe (recommended for IIT Madras students - $2/month free)
+LLM_PROVIDER=aipipe  # aipipe, anthropic, or openai
+LLM_MODEL=google/gemini-2.0-flash-lite-001  # Model to use with AIPipe
+AIPIPE_TOKEN=your-aipipe-token-from-https://aipipe.org/login
+AIPIPE_BASE_URL=https://aipipe.org/openrouter/v1  # or https://aipipe.org/openai/v1
+
+# Alternative: Direct API access
+ANTHROPIC_API_KEY=sk-ant-your-anthropic-api-key
+OPENAI_API_KEY=sk-your-openai-api-key
+
+# Instructor Configuration
+INSTRUCTOR_API_PORT=8001
+DATABASE_URL=postgresql://user:password@localhost:5432/llm_deployment
+EVALUATION_API_URL=http://localhost:8001/api/evaluate
+
+# Task Configuration
+TASK_TIMEOUT_MINUTES=10
+MAX_RETRY_ATTEMPTS=3
+RETRY_DELAYS=1,2,4,8
+
+# Paths
+GENERATED_REPOS_DIR=./generated_repos
+TASK_TEMPLATES_DIR=./templates
+SUBMISSIONS_CSV=./submissions.csv
+
+# Logging
+LOG_LEVEL=INFO
+LOG_FILE=./logs/app.log

.gitignore

@@ -0,0 +1,40 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+venv/
+env/
+
+# Environment variables and secrets
+.env
+.env.local
+*.key
+*.pem
+
+# Database
+*.db
+*.sqlite
+*.sqlite3
+
+# Generated repos
+generated_repos/
+
+# Logs
+logs/
+*.log
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db

.python-version

@@ -0,0 +1 @@
+3.13

DEPLOYMENT.md

@@ -0,0 +1,317 @@
+# Deployment Guide for Students
+
+This guide will help you deploy your LLM Code Deployment API to Hugging Face Spaces so instructors can send you tasks and grade your work.
+
+## Prerequisites
+
+1. ✅ GitHub account with a personal access token
+2. ✅ Hugging Face account (free)
+3. ✅ AIPipe token from https://aipipe.org/login
+4. ✅ IIT Madras email (@ds.study.iitm.ac.in) for free AIPipe access
+
+## Step-by-Step Deployment
+
+### Step 1: Get Your AIPipe Token
+
+1. Visit **https://aipipe.org/login**
+2. Sign in with your **@ds.study.iitm.ac.in** email
+3. Copy your API token (starts with `pk_`)
+4. **Important**: You get $2/month free. Don't exceed this limit!
+
+### Step 2: Create GitHub Personal Access Token
+
+1. Go to https://github.com/settings/tokens
+2. Click **"Generate new token"** → **"Generate new token (classic)"**
+3. Give it a name like "LLM Code Deployment"
+4. Select scopes: ✅ **repo** (all permissions)
+5. Click **"Generate token"**
+6. **Copy the token immediately** (you won't see it again)
+
+### Step 3: Create a Hugging Face Space
+
+1. Go to **https://huggingface.co/new-space**
+2. Choose a name (e.g., `llm-code-deploy-yourname`)
+3. Select **Docker** as the SDK
+4. Make it **Public**
+5. Click **Create Space**
+
+### Step 4: Configure Environment Variables
+
+In your Space, go to **Settings** → **Variables and secrets**:
+
+Add these **environment variables** (not secrets):
+
+| Variable | Value |
+|----------|-------|
+| `STUDENT_EMAIL` | Your email (e.g., `yourname@ds.study.iitm.ac.in`) |
+| `STUDENT_SECRET` | A secret phrase (e.g., `mySecretKey123`) |
+| `GITHUB_USERNAME` | Your GitHub username |
+| `LLM_PROVIDER` | `aipipe` |
+| `LLM_MODEL` | `google/gemini-2.0-flash-lite-001` |
+
+Add these as **secrets** (hidden values):
+
+| Secret | Value |
+|--------|-------|
+| `GITHUB_TOKEN` | Your GitHub personal access token |
+| `AIPIPE_TOKEN` | Your AIPipe token |
+
+### Step 5: Deploy Your Code
+
+**Option A: Push from Local Repository**
+
+```bash
+# Clone your repository
+git clone https://github.com/YOUR_USERNAME/tds-p1.git
+cd tds-p1
+
+# Add Hugging Face Space as remote
+git remote add space https://huggingface.co/spaces/YOUR_HF_USERNAME/YOUR_SPACE_NAME
+
+# Push to Space
+git push space main
+```
+
+**Option B: Upload Files Directly**
+
+1. In your Space, click **Files** → **Add file**
+2. Upload these files:
+   - `Dockerfile`
+   - `requirements.txt`
+   - All `.py` files
+   - `templates/` folder
+   - `.gitignore`
+
+### Step 6: Wait for Build
+
+1. Go to the **Logs** tab
+2. Wait for the build to complete (5-10 minutes)
+3. Look for: `Application startup complete` in the logs
+
+### Step 7: Test Your Deployment
+
+Your API endpoint will be:
+```
+https://YOUR_HF_USERNAME-YOUR_SPACE_NAME.hf.space/api/build
+```
+
+Test it:
+```bash
+curl https://YOUR_HF_USERNAME-YOUR_SPACE_NAME.hf.space/health
+```
+
+Expected response:
+```json
+{
+  "status": "healthy",
+  "active_tasks": 0,
+  "timestamp": "2025-01-16T10:30:00.123456"
+}
+```
+
+### Step 8: Submit to Instructors
+
+1. Go to the instructor's Google Form
+2. Submit:
+   - **API Endpoint**: `https://YOUR_HF_USERNAME-YOUR_SPACE_NAME.hf.space/api/build`
+   - **Secret**: Same value you used in `STUDENT_SECRET`
+   - **Email**: Same value you used in `STUDENT_EMAIL`
+
+## What Happens Next?
+
+1. **Instructors send tasks** → Your API receives a JSON request
+2. **Code generation** → LLM generates a web app based on the task
+3. **GitHub deployment** → New repo is created and pushed
+4. **Pages deployment** → GitHub Pages is enabled
+5. **Notification** → Your API notifies the evaluation endpoint
+6. **Grading** → Instructors run automated checks
+
+## Monitoring Your Deployment
+
+### View Logs
+
+In your Space:
+- **Logs** tab shows real-time activity
+- Look for task requests, code generation, and deployment status
+
+### Check GitHub
+
+Generated repos will appear at:
+```
+https://github.com/YOUR_GITHUB_USERNAME/
+```
+
+Each task creates a new repo like:
+```
+sum-of-sales-abc12
+markdown-to-html-def34
+```
+
+### Monitor AIPipe Usage
+
+Visit https://aipipe.org/usage to see:
+- How much of your $2/month quota you've used
+- Number of API calls made
+- Estimated cost
+
+## Troubleshooting
+
+### Space won't build
+**Error**: Build fails with dependency errors
+
+**Solution**:
+1. Check `Dockerfile` is present and correct
+2. Verify `requirements.txt` has all dependencies
+3. Review **Logs** for specific error messages
+
+### Invalid secret error
+**Error**: `401 Unauthorized: Invalid secret`
+
+**Solution**:
+1. Check `STUDENT_SECRET` matches what you submitted in the form
+2. Verify `STUDENT_EMAIL` is correct
+3. Test with the exact same values
+
+### GitHub authentication fails
+**Error**: `Failed to create repository`
+
+**Solution**:
+1. Verify `GITHUB_TOKEN` is set as a **secret** (not variable)
+2. Ensure token has `repo` permissions
+3. Check token hasn't expired (they last 1 year by default)
+
+### LLM API errors
+**Error**: `LLM generation failed: 401 Unauthorized`
+
+**Solution**:
+1. Verify `AIPIPE_TOKEN` is correct
+2. Check you haven't exceeded $2/month quota
+3. Try a different model: `anthropic/claude-3-haiku`
+
+### GitHub Pages not deploying
+**Error**: `Pages URL returns 404`
+
+**Solution**:
+1. Wait 60-120 seconds for Pages to activate
+2. Check repo is **public** (not private)
+3. Verify Pages is enabled in repo settings
+
+## Cost Optimization
+
+### Recommended Models (Cheapest to Most Expensive)
+
+1. **google/gemini-2.0-flash-lite-001** ✅ Recommended
+   - $0.00001/1K tokens (cheapest)
+   - ~$0.02 per task
+   - ~100 tasks per $2
+
+2. **anthropic/claude-3-haiku**
+   - $0.00025/1K tokens
+   - ~$0.50 per task
+   - ~4 tasks per $2
+
+3. **openai/gpt-4.1-nano**
+   - $0.0004/1K tokens
+   - ~$0.80 per task
+   - ~2.5 tasks per $2
+
+### Tips to Reduce Costs
+
+- ✅ Use Gemini Flash Lite (default)
+- ✅ Test locally before deploying
+- ✅ Monitor usage at https://aipipe.org/usage
+- ❌ Don't use GPT-4 or Claude Sonnet (expensive)
+- ❌ Don't test excessively (each test costs money)
+
+## Alternative: Run Locally (No Cost)
+
+If you prefer not to use AIPipe, run the instructor evaluation locally:
+
+```bash
+# Set up with your own Anthropic/OpenAI key
+echo "LLM_PROVIDER=anthropic" >> .env
+echo "ANTHROPIC_API_KEY=sk-ant-your-key" >> .env
+
+# Run student API locally
+uv run python main.py student-api
+
+# Use ngrok to expose it publicly
+ngrok http 8000
+```
+
+Then submit the ngrok URL to instructors.
+
+## FAQ
+
+### Can I use a different LLM provider?
+
+Yes! Change these environment variables:
+
+**For Anthropic Claude:**
+```
+LLM_PROVIDER=anthropic
+ANTHROPIC_API_KEY=sk-ant-your-key
+LLM_MODEL=claude-3-5-sonnet-20241022
+```
+
+**For OpenAI:**
+```
+LLM_PROVIDER=openai
+OPENAI_API_KEY=sk-your-key
+LLM_MODEL=gpt-4-turbo-preview
+```
+
+### How long does code generation take?
+
+- Gemini Flash Lite: 2-5 seconds
+- Claude Haiku: 3-7 seconds
+- GPT-4: 5-15 seconds
+
+Plus:
+- GitHub deployment: 30-60 seconds
+- Pages activation: 30-120 seconds
+- **Total**: ~2-5 minutes per task
+
+### Can I see the generated code before deployment?
+
+Yes! Check the logs:
+```bash
+# View logs in real-time
+curl https://YOUR_SPACE.hf.space/api/status/TASK_ID
+```
+
+Or check the generated repo on GitHub after deployment.
+
+### What if I exceed the $2 limit?
+
+1. AIPipe will block your requests
+2. Use your own API key (see alternative providers above)
+3. Contact instructors for assistance
+
+### How many tasks will I receive?
+
+Instructors may send:
+- **Round 1**: 1-3 tasks
+- **Round 2**: 1-3 update tasks per Round 1 task
+- **Total**: 2-9 tasks
+
+With Gemini Flash Lite (~$0.02/task), you'll use ~$0.18 total. Well within the $2 limit!
+
+## Support
+
+If you encounter issues:
+
+1. **Check logs**: Space → Logs tab
+2. **Review this guide**: Especially troubleshooting section
+3. **Test locally**: Run the API locally to debug
+4. **Ask instructors**: In the course forum or office hours
+
+## Next Steps
+
+After deployment:
+1. ✅ Submit your endpoint to the Google Form
+2. ✅ Wait for tasks from instructors
+3. ✅ Monitor logs and GitHub for activity
+4. ✅ Check evaluation results after deadline
+
+Good luck! 🚀

Dockerfile

@@ -0,0 +1,41 @@
+FROM python:3.11-slim
+
+# Create the same UID (1000) that Hugging Face Spaces uses
+RUN useradd -m -u 1000 user
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    build-essential \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Set working directory
+WORKDIR /home/user/app
+
+# Copy requirements and install Python dependencies
+COPY --chown=user requirements.txt .
+RUN pip install --no-cache-dir --upgrade pip && \
+    pip install --no-cache-dir -r requirements.txt
+
+# Install Playwright and chromium browser
+RUN pip install --no-cache-dir playwright && \
+    playwright install chromium && \
+    playwright install-deps chromium
+
+# Copy the rest of the application
+COPY --chown=user . .
+
+# Switch to non-root user
+USER user
+
+# Set environment variables
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH \
+    PYTHONUNBUFFERED=1
+
+# Expose the port that Hugging Face Spaces expects
+EXPOSE 7860
+
+# Start the student API (this is what students will deploy)
+CMD ["uvicorn", "student.api:app", "--host", "0.0.0.0", "--port", "7860"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 LLM Code Deployment System
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

QUICKSTART.md

@@ -0,0 +1,185 @@
+# Quick Start Guide
+
+## Installation
+
+1. **Install dependencies with uv**
+```bash
+uv sync
+```
+
+2. **Install Playwright browsers**
+```bash
+uv run playwright install chromium
+```
+
+3. **Set up environment variables**
+```bash
+cp .env.example .env
+```
+
+Edit `.env` and configure at minimum:
+- `STUDENT_SECRET` - Your secret key
+- `STUDENT_EMAIL` - Your email
+- `GITHUB_TOKEN` - GitHub personal access token
+- `GITHUB_USERNAME` - Your GitHub username
+- `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` - LLM API key
+
+## Running the System
+
+### Option 1: Using main.py CLI
+
+**Start Student API:**
+```bash
+uv run python main.py student-api
+```
+
+**Start Instructor API:**
+```bash
+uv run python main.py instructor-api
+```
+
+**Initialize Database:**
+```bash
+uv run python main.py init-db
+```
+
+**Run Round 1:**
+```bash
+uv run python main.py round1
+```
+
+**Run Evaluation:**
+```bash
+uv run python main.py evaluate
+```
+
+**Run Round 2:**
+```bash
+uv run python main.py round2
+```
+
+### Option 2: Direct module execution
+
+**Start Student API:**
+```bash
+uv run python -m student.api
+```
+
+**Start Instructor API:**
+```bash
+uv run python -m instructor.api
+```
+
+## Testing the System
+
+### 1. Test Student API
+
+Start the student API:
+```bash
+uv run python main.py student-api
+```
+
+In another terminal, send a test request:
+```bash
+curl -X POST http://localhost:8000/api/build \
+  -H "Content-Type: application/json" \
+  -d '{
+    "email": "your-email@example.com",
+    "secret": "your-secret",
+    "task": "test-task-abc",
+    "round": 1,
+    "nonce": "unique-nonce-123",
+    "brief": "Create a simple Hello World page with Bootstrap",
+    "checks": ["Page displays Hello World", "Bootstrap is loaded"],
+    "evaluation_url": "http://localhost:8001/api/evaluate",
+    "attachments": []
+  }'
+```
+
+Check status:
+```bash
+curl http://localhost:8000/api/status/test-task-abc
+```
+
+### 2. Test Instructor Workflow
+
+**Start Instructor API:**
+```bash
+uv run python main.py instructor-api
+```
+
+**Initialize Database:**
+```bash
+uv run python main.py init-db
+```
+
+**Create submissions.csv:**
+```csv
+timestamp,email,endpoint,secret
+2025-01-15T10:00:00,student@example.com,http://localhost:8000/api/build,your-secret
+```
+
+**Run Round 1:**
+```bash
+uv run python main.py round1
+```
+
+**Run Evaluation:**
+```bash
+uv run python main.py evaluate
+```
+
+**Check Results:**
+```bash
+curl http://localhost:8001/api/results/student@example.com
+```
+
+## Project Structure Overview
+
+```
+student/              # Student side (receives tasks, generates code)
+├── api.py           # API endpoint
+├── code_generator.py # LLM code generation
+├── github_manager.py # GitHub operations
+└── notification_client.py # Notify evaluation
+
+instructor/          # Instructor side (generates tasks, evaluates)
+├── api.py          # Evaluation endpoint
+├── database.py     # Database operations
+├── task_templates.py # Template management
+├── round1.py       # Generate round 1 tasks
+├── round2.py       # Generate round 2 tasks
+├── evaluate.py     # Run evaluations
+└── checks/         # Evaluation checks
+    ├── static_checks.py
+    ├── dynamic_checks.py
+    └── llm_checks.py
+
+shared/             # Shared utilities
+├── config.py       # Configuration
+├── models.py       # Data models
+├── logger.py       # Logging
+└── utils.py        # Utilities
+
+templates/          # Task templates (YAML)
+```
+
+## Next Steps
+
+1. Configure your `.env` file with actual credentials
+2. Set up a PostgreSQL database (if using instructor features)
+3. Review the task templates in `templates/`
+4. Test the student API with a simple request
+5. Set up the instructor system for evaluation
+
+## Common Issues
+
+**Import errors**: Make sure you run commands with `uv run` prefix
+
+**GitHub auth errors**: Verify GITHUB_TOKEN in .env has proper permissions
+
+**Database errors**: Make sure PostgreSQL is running and DATABASE_URL is correct
+
+**LLM errors**: Check your API key and quota
+
+For more details, see README.md

README.md

@@ -0,0 +1,411 @@
+# LLM Code Deployment System
+
+An automated system for building, deploying, and evaluating LLM-generated web applications with GitHub Pages integration.
+
+## Overview
+
+This project implements a complete workflow for:
+- **Students**: Receive task requests, use LLMs to generate code, deploy to GitHub Pages, and submit for evaluation
+- **Instructors**: Generate task requests, receive submissions, run automated evaluations (static, dynamic, and LLM-based)
+
+## 🚀 Quick Deployment for Students
+
+**Deploy to Hugging Face Spaces in 10 minutes:**
+
+1. Read **[DEPLOYMENT.md](DEPLOYMENT.md)** for complete step-by-step instructions
+2. Read **[README_SPACES.md](README_SPACES.md)** for Hugging Face Spaces configuration
+3. Get your AIPipe token from https://aipipe.org/login ($2/month free for IIT Madras students)
+4. Create a Space at https://huggingface.co/new-space
+5. Configure environment variables and deploy!
+
+**Already deployed?** Just submit your endpoint URL to the instructor's Google Form!
+
+## Architecture
+
+```
+┌─────────────┐         ┌──────────────┐         ┌─────────────┐
+│  Instructor │         │   Student    │         │   GitHub    │
+│   System    │────────▶│     API      │────────▶│    Pages    │
+│             │  POST   │              │  Deploy │             │
+└─────────────┘  Task   └──────────────┘         └─────────────┘
+      │                        │                         │
+      │                        │                         │
+      │                        └────────POST─────────────┘
+      │                         Submission               │
+      │                                                  │
+      ▼                                                  ▼
+┌─────────────┐                                   ┌─────────────┐
+│  Evaluation │◀──────────────────────────────────│  Validation │
+│   Database  │                                   │   & Checks  │
+└─────────────┘                                   └─────────────┘
+```
+
+## Features
+
+### Student-Side
+- **API Endpoint**: Receives task requests via HTTP POST
+- **LLM Code Generation**: Uses Claude/GPT to generate complete web apps
+- **GitHub Integration**: Automatically creates repos, pushes code, enables Pages
+- **Automatic Notification**: Sends repo details to evaluation endpoint
+- **Round 2 Support**: Handles update requests for existing repos
+
+### Instructor-Side
+- **Task Templates**: YAML-based parametrizable task definitions
+- **Round 1 & 2 Scripts**: Automated task generation and distribution
+- **Evaluation API**: Receives and validates student submissions
+- **Multi-Level Checks**:
+  - Static: License, README, repo creation time, secrets detection
+  - LLM: Code quality, documentation quality
+  - Dynamic: Playwright-based functional testing
+- **Database**: PostgreSQL storage for tasks, repos, and results
+
+## Project Structure
+
+```
+tds-p1/
+├── shared/                 # Shared utilities and models
+│   ├── config.py          # Configuration management
+│   ├── models.py          # Pydantic data models
+│   ├── logger.py          # Logging setup
+│   └── utils.py           # Utility functions
+├── student/               # Student-side components
+│   ├── api.py             # FastAPI endpoint
+│   ├── code_generator.py  # LLM-based code generation
+│   ├── github_manager.py  # GitHub operations
+│   └── notification_client.py  # Evaluation notification
+├── instructor/            # Instructor-side components
+│   ├── api.py             # Evaluation endpoint
+│   ├── database.py        # Database models and operations
+│   ├── task_templates.py  # Template management
+│   ├── round1.py          # Round 1 task generation
+│   ├── round2.py          # Round 2 task generation
+│   ├── evaluate.py        # Main evaluation script
+│   └── checks/            # Evaluation modules
+│       ├── static_checks.py   # Static analysis
+│       ├── dynamic_checks.py  # Playwright tests
+│       └── llm_checks.py      # LLM evaluations
+├── templates/             # Task template YAML files
+│   ├── sum-of-sales.yaml
+│   ├── markdown-to-html.yaml
+│   └── github-user-created.yaml
+├── pyproject.toml         # Project dependencies
+├── .env.example           # Environment variables template
+└── README.md              # This file
+```
+
+## Setup
+
+### Prerequisites
+- Python 3.10+
+- PostgreSQL database
+- GitHub account with personal access token
+- Anthropic or OpenAI API key
+
+### Installation
+
+1. **Clone the repository**
+```bash
+git clone <your-repo-url>
+cd tds-p1
+```
+
+2. **Install dependencies**
+```bash
+pip install -e .
+```
+
+3. **Install Playwright browsers**
+```bash
+playwright install chromium
+```
+
+4. **Configure environment**
+```bash
+cp .env.example .env
+# Edit .env with your credentials
+```
+
+5. **Set up database**
+```bash
+# Create PostgreSQL database
+createdb llm_deployment
+
+# Initialize tables
+python -c "from instructor.database import Database; Database().create_tables()"
+```
+
+## Configuration
+
+Edit `.env` with your settings:
+
+### Student Configuration
+```bash
+STUDENT_SECRET=your-secret-key
+STUDENT_EMAIL=your-email@example.com
+STUDENT_API_PORT=8000
+```
+
+### GitHub
+```bash
+GITHUB_TOKEN=ghp_your_personal_access_token
+GITHUB_USERNAME=your-username
+```
+
+### LLM Provider
+```bash
+# Choose one
+LLM_PROVIDER=anthropic  # or openai
+ANTHROPIC_API_KEY=sk-ant-...
+# OR
+OPENAI_API_KEY=sk-...
+LLM_MODEL=claude-3-5-sonnet-20241022
+```
+
+### Instructor
+```bash
+DATABASE_URL=postgresql://user:password@localhost:5432/llm_deployment
+EVALUATION_API_URL=http://your-server:8001/api/evaluate
+```
+
+## Usage
+
+### For Students
+
+1. **Start the Student API**
+```bash
+python -m student.api
+```
+
+The API will listen on `http://localhost:8000/api/build`
+
+2. **Test with a sample request**
+```bash
+curl -X POST http://localhost:8000/api/build \
+  -H "Content-Type: application/json" \
+  -d '{
+    "email": "your-email@example.com",
+    "secret": "your-secret",
+    "task": "test-task-abc",
+    "round": 1,
+    "nonce": "unique-nonce-123",
+    "brief": "Create a simple Hello World page",
+    "checks": ["Page displays Hello World"],
+    "evaluation_url": "http://localhost:8001/api/evaluate",
+    "attachments": []
+  }'
+```
+
+### For Instructors
+
+1. **Start the Evaluation API**
+```bash
+python -m instructor.api
+```
+
+2. **Prepare submissions.csv**
+```csv
+timestamp,email,endpoint,secret
+2025-01-15T10:00:00,student1@example.com,http://student1.com/api/build,secret1
+2025-01-15T10:05:00,student2@example.com,http://student2.com/api/build,secret2
+```
+
+3. **Run Round 1 task generation**
+```bash
+python -m instructor.round1
+```
+
+This will:
+- Load submissions from CSV
+- Generate unique tasks from templates
+- POST tasks to student endpoints
+- Log results to database
+
+4. **Run evaluations**
+```bash
+python -m instructor.evaluate
+```
+
+This will:
+- Fetch pending submissions
+- Clone repositories
+- Run static, LLM, and Playwright checks
+- Save results to database
+
+5. **Run Round 2 task generation**
+```bash
+python -m instructor.round2
+```
+
+This will:
+- Find all Round 1 submissions
+- Generate Round 2 update tasks
+- POST to student endpoints
+
+## Task Templates
+
+Task templates are YAML files in the `templates/` directory. Example:
+
+```yaml
+id: sum-of-sales
+brief: Publish a single-page site that fetches data.csv from attachments...
+attachments:
+  - name: data.csv
+    url: data:text/csv;base64,placeholder
+checks:
+  - "Page title equals 'Sales Summary {{ seed }}'"
+  - "Bootstrap 5 CSS loaded from jsdelivr"
+round2:
+  - brief: Add a Bootstrap table #product-sales...
+    checks:
+      - "Table #product-sales exists"
+```
+
+### Template Variables
+- `{{ seed }}`: Unique seed based on email and timestamp
+- `{{ hash }}`: Deterministic hash value
+- `{{ result }}`: Generated numeric value
+
+## API Endpoints
+
+### Student API
+
+**POST /api/build**
+- Receives task request
+- Returns 200 on acceptance
+- Processes in background
+
+**GET /api/status/{task_id}**
+- Returns task status
+
+**GET /health**
+- Health check
+
+### Instructor API
+
+**POST /api/evaluate**
+- Receives repo submission
+- Validates against task record
+- Returns 200 on success
+
+**GET /api/submissions/{email}**
+- Returns all submissions for email
+
+**GET /api/results/{email}**
+- Returns all evaluation results
+
+## Evaluation Criteria
+
+### Static Checks
+- ✓ Repository created after task sent
+- ✓ MIT LICENSE exists in root
+- ✓ README.md present with good structure
+- ✓ No secrets in git history
+
+### LLM Checks
+- ✓ README.md professional quality (0-1 score)
+- ✓ Code quality and best practices (0-1 score)
+
+### Dynamic Checks
+- ✓ Task-specific requirements (from template)
+- ✓ Page loads successfully
+- ✓ JavaScript evaluations
+- ✓ Element presence and content
+
+## Database Schema
+
+### Tasks Table
+- Task requests sent to students
+- Fields: email, task, round, nonce, brief, checks, etc.
+
+### Repos Table
+- Repository submissions from students
+- Fields: email, task, round, repo_url, commit_sha, pages_url
+
+### Results Table
+- Evaluation results
+- Fields: email, task, round, check, score, reason, logs
+
+## Troubleshooting
+
+### Common Issues
+
+**Student API not receiving requests**
+- Check firewall settings
+- Ensure port 8000 is accessible
+- Verify endpoint URL in submissions.csv
+
+**GitHub Pages not deploying**
+- Verify GITHUB_TOKEN has repo permissions
+- Check repository is public
+- Wait up to 60 seconds for Pages to activate
+
+**LLM generation fails**
+- Check API key is valid
+- Verify API quota/credits
+- Review logs for error details
+
+**Playwright tests fail**
+- Ensure chromium is installed: `playwright install chromium`
+- Check Pages URL is accessible
+- Increase timeout if needed
+
+**Database connection errors**
+- Verify PostgreSQL is running
+- Check DATABASE_URL credentials
+- Ensure database exists
+
+## Development
+
+### Running Tests
+```bash
+pytest tests/
+```
+
+### Code Formatting
+```bash
+black .
+ruff check .
+```
+
+### Type Checking
+```bash
+mypy .
+```
+
+## License
+
+MIT License
+
+Copyright (c) 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Submit a pull request
+
+## Support
+
+For issues and questions:
+- Check the troubleshooting section
+- Review logs in `logs/app.log`
+- Open an issue on GitHub

README_SPACES.md

@@ -0,0 +1,180 @@
+---
+title: LLM Code Deployment API
+emoji: 🚀
+colorFrom: blue
+colorTo: green
+sdk: docker
+app_port: 7860
+---
+
+# LLM Code Deployment - Student API
+
+This is a Hugging Face Space that hosts the student API endpoint for the LLM Code Deployment project. It receives build requests, generates code using LLMs, and deploys to GitHub Pages.
+
+## Setup on Hugging Face Spaces
+
+### 1. Create a New Space
+
+1. Go to https://huggingface.co/new-space
+2. Choose a name for your Space
+3. Select **Docker** as the SDK
+4. Click **Create Space**
+
+### 2. Configure Environment Variables
+
+Go to your Space's **Settings** → **Variables and secrets** tab and add:
+
+**Required:**
+- `STUDENT_EMAIL` - Your email address (matching the submission form)
+- `STUDENT_SECRET` - Your secret key (matching the submission form)
+- `GITHUB_TOKEN` - Your GitHub personal access token (with `repo` permissions)
+- `GITHUB_USERNAME` - Your GitHub username
+- `AIPIPE_TOKEN` - Your AIPipe token from https://aipipe.org/login
+
+**Optional (advanced):**
+- `LLM_PROVIDER` - `aipipe` (default), `anthropic`, or `openai`
+- `LLM_MODEL` - Model name (default: `google/gemini-2.0-flash-lite-001`)
+- `AIPIPE_BASE_URL` - AIPipe endpoint (default: `https://aipipe.org/openrouter/v1`)
+
+### 3. Get Your AIPipe Token
+
+1. Visit https://aipipe.org/login
+2. Sign in with your `@ds.study.iitm.ac.in` email
+3. Copy your API token
+4. You get **$2 per month free** - don't exceed this
+
+### 4. Deploy the Code
+
+**Option A: Clone from GitHub**
+```bash
+git clone https://github.com/YOUR_USERNAME/YOUR_REPO.git
+cd YOUR_REPO
+git remote add space https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE
+git push space main
+```
+
+**Option B: Direct Upload**
+1. Upload all project files to your Space via the web interface
+2. Ensure `Dockerfile`, `requirements.txt`, and all code files are present
+
+### 5. Wait for Build
+
+The Space will automatically build using the Dockerfile. This may take 5-10 minutes.
+
+### 6. Get Your API Endpoint
+
+Once deployed, your endpoint will be:
+```
+https://YOUR_USERNAME-YOUR_SPACE.hf.space/api/build
+```
+
+Use this URL when submitting to the instructor's Google Form.
+
+## Testing Your Deployment
+
+Test your endpoint locally first:
+
+```bash
+curl -X POST https://YOUR_USERNAME-YOUR_SPACE.hf.space/api/build \
+  -H "Content-Type: application/json" \
+  -d '{
+    "email": "your-email@ds.study.iitm.ac.in",
+    "secret": "your-secret",
+    "task": "test-task-123",
+    "round": 1,
+    "nonce": "test-nonce-456",
+    "brief": "Create a simple Hello World page with Bootstrap",
+    "checks": ["Page displays Hello World"],
+    "evaluation_url": "https://example.com/evaluate",
+    "attachments": []
+  }'
+```
+
+Expected response:
+```json
+{
+  "status": "accepted",
+  "message": "Task test-task-123-1 accepted for processing",
+  "task": "test-task-123",
+  "round": 1
+}
+```
+
+## Health Check
+
+Check if your Space is running:
+```bash
+curl https://YOUR_USERNAME-YOUR_SPACE.hf.space/health
+```
+
+## Monitoring
+
+View logs in the **Logs** tab of your Space to monitor:
+- Task requests received
+- Code generation progress
+- GitHub deployment status
+- Evaluation notification results
+
+## Troubleshooting
+
+### Space won't start
+- Check the **Logs** tab for build errors
+- Ensure all environment variables are set correctly
+- Verify Dockerfile and requirements.txt are present
+
+### Authentication errors
+- Verify `STUDENT_SECRET` matches what you submitted in the form
+- Check `STUDENT_EMAIL` is correct
+- Ensure `GITHUB_TOKEN` has repo permissions
+
+### LLM generation fails
+- Verify `AIPIPE_TOKEN` is set and valid
+- Check you haven't exceeded the $2/month quota
+- Review logs for specific error messages
+
+### GitHub deployment fails
+- Ensure `GITHUB_TOKEN` has correct permissions
+- Check `GITHUB_USERNAME` is correct
+- Verify token hasn't expired
+
+## Cost Management
+
+**AIPipe Limits:**
+- Free tier: $2 per month for @ds.study.iitm.ac.in emails
+- Models recommended:
+  - `google/gemini-2.0-flash-lite-001` (cheapest)
+  - `anthropic/claude-3-haiku` (good quality)
+  - `openai/gpt-4.1-nano` (balanced)
+
+**Monitoring usage:**
+- Check https://aipipe.org/usage
+- Each code generation uses ~1000-2000 tokens
+- Budget for ~50-100 task submissions per month
+
+## Local Development
+
+To test locally before deploying to Spaces:
+
+```bash
+# Set environment variables
+cp .env.example .env
+# Edit .env with your credentials
+
+# Run with Docker
+docker build -t llm-code-deploy .
+docker run -p 7860:7860 --env-file .env llm-code-deploy
+
+# Or run directly with Python
+uv run python main.py student-api
+```
+
+## Support
+
+For issues:
+1. Check the project README.md
+2. Review Hugging Face Spaces documentation
+3. Contact the course instructors
+
+## License
+
+MIT License - See LICENSE file

instructor/__init__.py

@@ -0,0 +1 @@
+"""Instructor-side modules for task generation and evaluation."""

instructor/api.py

@@ -0,0 +1,168 @@
+"""Instructor API endpoint for receiving repo submissions."""
+from datetime import datetime
+
+from fastapi import FastAPI, HTTPException
+from fastapi.responses import JSONResponse
+
+from instructor.database import Database
+from shared.logger import setup_logger
+from shared.models import RepoSubmission
+
+logger = setup_logger(__name__)
+
+app = FastAPI(
+    title="LLM Code Deployment - Instructor API",
+    description="API endpoint for receiving and validating repo submissions",
+    version="1.0.0",
+)
+
+# Initialize database
+db = Database()
+
+
+@app.on_event("startup")
+async def startup_event():
+    """Create database tables on startup."""
+    db.create_tables()
+    logger.info("Database initialized")
+
+
+@app.post("/api/evaluate")
+async def evaluate_submission(submission: RepoSubmission) -> JSONResponse:
+    """Receive and validate repo submission.
+
+    Args:
+        submission: Repository submission from student
+
+    Returns:
+        HTTP 200 on success, HTTP 400 on validation failure
+    """
+    logger.info(
+        f"Received submission: {submission.email}, "
+        f"task {submission.task}, round {submission.round}"
+    )
+
+    # Validate submission against task record
+    task = db.get_task_by_nonce(submission.nonce)
+
+    if not task:
+        logger.error(f"Invalid nonce: {submission.nonce}")
+        raise HTTPException(status_code=400, detail="Invalid nonce")
+
+    # Validate fields match
+    if task.email != submission.email:
+        logger.error(f"Email mismatch: expected {task.email}, got {submission.email}")
+        raise HTTPException(status_code=400, detail="Email mismatch")
+
+    if task.task != submission.task:
+        logger.error(f"Task mismatch: expected {task.task}, got {submission.task}")
+        raise HTTPException(status_code=400, detail="Task mismatch")
+
+    if task.round != submission.round:
+        logger.error(f"Round mismatch: expected {task.round}, got {submission.round}")
+        raise HTTPException(status_code=400, detail="Round mismatch")
+
+    # Check if already submitted
+    try:
+        existing = db.get_session().query(db.Repo).filter_by(nonce=submission.nonce).first()
+        if existing:
+            logger.warning(f"Duplicate submission for nonce {submission.nonce}")
+            return JSONResponse(
+                status_code=200,
+                content={
+                    "status": "duplicate",
+                    "message": "Submission already received",
+                },
+            )
+    finally:
+        db.get_session().close()
+
+    # Add to repos table
+    try:
+        repo = db.add_repo(
+            {
+                "timestamp": datetime.utcnow(),
+                "email": submission.email,
+                "task": submission.task,
+                "round": submission.round,
+                "nonce": submission.nonce,
+                "repo_url": submission.repo_url,
+                "commit_sha": submission.commit_sha,
+                "pages_url": submission.pages_url,
+            }
+        )
+
+        logger.info(
+            f"Added repo submission: {submission.task}, round {submission.round}, "
+            f"repo {submission.repo_url}"
+        )
+
+        return JSONResponse(
+            status_code=200,
+            content={
+                "status": "success",
+                "message": "Submission received and queued for evaluation",
+                "repo_id": repo.id,
+            },
+        )
+
+    except Exception as e:
+        logger.error(f"Failed to add repo submission: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Internal server error")
+
+
+@app.get("/api/submissions/{email}")
+async def get_submissions(email: str) -> JSONResponse:
+    """Get all submissions for an email.
+
+    Args:
+        email: Student email
+
+    Returns:
+        List of submissions
+    """
+    repos = db.get_repos(email=email)
+    return JSONResponse(content=[repo.to_dict() for repo in repos])
+
+
+@app.get("/api/results/{email}")
+async def get_results(email: str) -> JSONResponse:
+    """Get all evaluation results for an email.
+
+    Args:
+        email: Student email
+
+    Returns:
+        List of evaluation results
+    """
+    results = db.get_results(email=email)
+    return JSONResponse(content=[result.to_dict() for result in results])
+
+
+@app.get("/health")
+async def health_check() -> JSONResponse:
+    """Health check endpoint.
+
+    Returns:
+        Health status
+    """
+    return JSONResponse(
+        content={
+            "status": "healthy",
+            "timestamp": datetime.utcnow().isoformat(),
+        }
+    )
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    from shared.config import settings
+
+    logger.info(f"Starting Instructor API on port {settings.instructor_api_port}")
+    uvicorn.run(
+        "instructor.api:app",
+        host="0.0.0.0",
+        port=settings.instructor_api_port,
+        reload=True,
+    )

instructor/checks/__init__.py

@@ -0,0 +1 @@
+"""Evaluation check modules."""

instructor/checks/dynamic_checks.py

@@ -0,0 +1,216 @@
+"""Dynamic checks using Playwright."""
+import asyncio
+import json
+import re
+from typing import Any
+
+from playwright.async_api import async_playwright
+
+from shared.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+
+class DynamicChecker:
+    """Perform dynamic checks using Playwright."""
+
+    def __init__(self, pages_url: str, checks: list[str]) -> None:
+        """Initialize dynamic checker.
+
+        Args:
+            pages_url: GitHub Pages URL
+            checks: List of check descriptions
+        """
+        self.pages_url = pages_url
+        self.checks = checks
+
+    async def run_checks(self) -> list[dict[str, Any]]:
+        """Run all dynamic checks.
+
+        Returns:
+            List of check results
+        """
+        results = []
+
+        async with async_playwright() as p:
+            browser = await p.chromium.launch()
+            page = await browser.new_page()
+
+            try:
+                # Load page
+                await page.goto(self.pages_url, timeout=30000, wait_until="networkidle")
+
+                # Wait a bit for JS to execute
+                await page.wait_for_timeout(2000)
+
+                # Run each check
+                for check in self.checks:
+                    result = await self._run_single_check(page, check)
+                    results.append(result)
+
+            except Exception as e:
+                logger.error(f"Error loading page {self.pages_url}: {e}")
+                results.append(
+                    {
+                        "check": "Page Load",
+                        "passed": False,
+                        "score": 0.0,
+                        "reason": f"Failed to load page: {e}",
+                    }
+                )
+
+            finally:
+                await browser.close()
+
+        return results
+
+    async def _run_single_check(self, page, check_desc: str) -> dict[str, Any]:
+        """Run a single check.
+
+        Args:
+            page: Playwright page
+            check_desc: Check description
+
+        Returns:
+            Check result
+        """
+        try:
+            # Parse check type
+            if check_desc.startswith("js:"):
+                # JavaScript evaluation check
+                js_code = check_desc[3:].strip()
+                return await self._run_js_check(page, js_code)
+
+            else:
+                # Text-based check
+                return await self._run_text_check(page, check_desc)
+
+        except Exception as e:
+            logger.error(f"Error running check '{check_desc}': {e}")
+            return {
+                "check": check_desc,
+                "passed": False,
+                "score": 0.0,
+                "reason": f"Error: {e}",
+            }
+
+    async def _run_js_check(self, page, js_code: str) -> dict[str, Any]:
+        """Run JavaScript evaluation check.
+
+        Args:
+            page: Playwright page
+            js_code: JavaScript code to evaluate
+
+        Returns:
+            Check result
+        """
+        try:
+            result = await page.evaluate(js_code)
+
+            if isinstance(result, bool):
+                return {
+                    "check": f"JS: {js_code[:50]}...",
+                    "passed": result,
+                    "score": 1.0 if result else 0.0,
+                    "reason": f"JavaScript evaluated to {result}",
+                }
+            else:
+                return {
+                    "check": f"JS: {js_code[:50]}...",
+                    "passed": True,
+                    "score": 1.0,
+                    "reason": f"JavaScript returned: {result}",
+                }
+
+        except Exception as e:
+            return {
+                "check": f"JS: {js_code[:50]}...",
+                "passed": False,
+                "score": 0.0,
+                "reason": f"JavaScript error: {e}",
+            }
+
+    async def _run_text_check(self, page, check_desc: str) -> dict[str, Any]:
+        """Run text-based check.
+
+        Args:
+            page: Playwright page
+            check_desc: Check description
+
+        Returns:
+            Check result
+        """
+        # Extract selector and expected condition
+        # Examples:
+        # - "Page title equals 'Sales Summary xyz'"
+        # - "Element #total-sales exists and displays numeric value"
+        # - "Bootstrap 5 CSS loaded from jsdelivr"
+
+        content = await page.content()
+
+        # Check for page title
+        if "page title" in check_desc.lower():
+            title = await page.title()
+            match = re.search(r"['\"]([^'\"]+)['\"]", check_desc)
+            if match:
+                expected = match.group(1)
+                passed = expected in title
+                return {
+                    "check": check_desc,
+                    "passed": passed,
+                    "score": 1.0 if passed else 0.0,
+                    "reason": f"Title is '{title}', expected to contain '{expected}'",
+                }
+
+        # Check for element existence
+        if "element" in check_desc.lower():
+            # Extract selector
+            selector_match = re.search(r"(#[\w-]+|\.[a-z-]+|[a-z]+)", check_desc, re.IGNORECASE)
+            if selector_match:
+                selector = selector_match.group(1)
+                try:
+                    element = await page.query_selector(selector)
+                    if element:
+                        text = await element.text_content()
+                        return {
+                            "check": check_desc,
+                            "passed": True,
+                            "score": 1.0,
+                            "reason": f"Element {selector} found with content: {text[:50]}...",
+                        }
+                    else:
+                        return {
+                            "check": check_desc,
+                            "passed": False,
+                            "score": 0.0,
+                            "reason": f"Element {selector} not found",
+                        }
+                except Exception as e:
+                    return {
+                        "check": check_desc,
+                        "passed": False,
+                        "score": 0.0,
+                        "reason": f"Error finding element: {e}",
+                    }
+
+        # Check for Bootstrap/library loading
+        if "bootstrap" in check_desc.lower() or "cdn" in check_desc.lower():
+            has_bootstrap = "bootstrap" in content.lower()
+            return {
+                "check": check_desc,
+                "passed": has_bootstrap,
+                "score": 1.0 if has_bootstrap else 0.0,
+                "reason": f"Bootstrap {'found' if has_bootstrap else 'not found'} in page",
+            }
+
+        # Default: check if description keywords appear in content
+        keywords = re.findall(r"\b\w{4,}\b", check_desc.lower())
+        matches = sum(1 for kw in keywords if kw in content.lower())
+        score = min(matches / max(len(keywords), 1), 1.0)
+
+        return {
+            "check": check_desc,
+            "passed": score >= 0.5,
+            "score": score,
+            "reason": f"Matched {matches}/{len(keywords)} keywords",
+        }

instructor/checks/llm_checks.py

@@ -0,0 +1,209 @@
+"""LLM-based code quality checks."""
+from pathlib import Path
+from typing import Any
+
+import anthropic
+import openai
+
+from shared.config import settings
+from shared.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+
+class LLMChecker:
+    """Perform LLM-based quality checks."""
+
+    def __init__(self) -> None:
+        """Initialize LLM checker."""
+        self.provider = settings.llm_provider
+        self.model = settings.llm_model
+
+        if self.provider == "anthropic":
+            self.client = anthropic.Anthropic(api_key=settings.anthropic_api_key)
+        elif self.provider == "openai":
+            self.client = openai.OpenAI(api_key=settings.openai_api_key)
+        elif self.provider == "aipipe":
+            # Use OpenAI client with AIPipe endpoints
+            self.client = openai.OpenAI(
+                api_key=settings.aipipe_token,
+                base_url=settings.aipipe_base_url
+            )
+
+        logger.info(f"Initialized LLMChecker with {self.provider}/{self.model}")
+
+    def check_readme_quality(self, readme_path: Path) -> dict[str, Any]:
+        """Evaluate README.md quality using LLM.
+
+        Args:
+            readme_path: Path to README.md
+
+        Returns:
+            Check result
+        """
+        try:
+            if not readme_path.exists():
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "README.md not found",
+                }
+
+            content = readme_path.read_text()
+
+            prompt = f"""Evaluate this README.md for quality. Rate it on a scale of 0.0 to 1.0.
+
+Criteria:
+- Has a clear title and description
+- Explains setup/installation
+- Explains usage
+- Has code examples or explanations
+- Is well-formatted and professional
+- Mentions license
+
+README content:
+{content}
+
+Respond with ONLY a JSON object in this format:
+{{
+  "score": 0.85,
+  "reason": "Clear title and good structure, but missing detailed setup instructions"
+}}"""
+
+            response = self._call_llm(prompt)
+
+            # Parse JSON from response
+            import json
+            import re
+
+            json_match = re.search(r"\{.*\}", response, re.DOTALL)
+            if json_match:
+                result = json.loads(json_match.group(0))
+                score = float(result.get("score", 0.0))
+                reason = result.get("reason", "No reason provided")
+
+                return {
+                    "passed": score >= 0.7,
+                    "score": score,
+                    "reason": reason,
+                }
+
+            return {
+                "passed": False,
+                "score": 0.0,
+                "reason": "Could not parse LLM response",
+            }
+
+        except Exception as e:
+            logger.error(f"Error in README quality check: {e}")
+            return {"passed": False, "score": 0.0, "reason": f"Error: {e}"}
+
+    def check_code_quality(self, code_dir: Path) -> dict[str, Any]:
+        """Evaluate code quality using LLM.
+
+        Args:
+            code_dir: Directory containing code
+
+        Returns:
+            Check result
+        """
+        try:
+            # Gather code files
+            code_files = {}
+            for ext in [".html", ".js", ".css"]:
+                for file in code_dir.rglob(f"*{ext}"):
+                    if ".git" not in str(file):
+                        rel_path = file.relative_to(code_dir)
+                        code_files[str(rel_path)] = file.read_text()[:2000]  # Limit size
+
+            if not code_files:
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "No code files found",
+                }
+
+            # Format code for LLM
+            code_text = "\n\n".join(
+                f"=== {name} ===\n{content}" for name, content in code_files.items()
+            )
+
+            prompt = f"""Evaluate this web application code for quality. Rate it on a scale of 0.0 to 1.0.
+
+Criteria:
+- Code is clean and well-organized
+- Proper use of HTML semantics
+- Good JavaScript practices
+- Reasonable styling
+- Comments where helpful
+- No obvious bugs or security issues
+
+Code:
+{code_text}
+
+Respond with ONLY a JSON object in this format:
+{{
+  "score": 0.8,
+  "reason": "Clean code with good structure, minor improvements possible"
+}}"""
+
+            response = self._call_llm(prompt)
+
+            # Parse JSON
+            import json
+            import re
+
+            json_match = re.search(r"\{.*\}", response, re.DOTALL)
+            if json_match:
+                result = json.loads(json_match.group(0))
+                score = float(result.get("score", 0.0))
+                reason = result.get("reason", "No reason provided")
+
+                return {
+                    "passed": score >= 0.6,
+                    "score": score,
+                    "reason": reason,
+                }
+
+            return {
+                "passed": False,
+                "score": 0.0,
+                "reason": "Could not parse LLM response",
+            }
+
+        except Exception as e:
+            logger.error(f"Error in code quality check: {e}")
+            return {"passed": False, "score": 0.0, "reason": f"Error: {e}"}
+
+    def _call_llm(self, prompt: str) -> str:
+        """Call LLM API.
+
+        Args:
+            prompt: Prompt text
+
+        Returns:
+            LLM response
+        """
+        try:
+            if self.provider == "anthropic":
+                response = self.client.messages.create(
+                    model=self.model,
+                    max_tokens=1024,
+                    temperature=0.3,
+                    messages=[{"role": "user", "content": prompt}],
+                )
+                return response.content[0].text
+
+            elif self.provider in ["openai", "aipipe"]:
+                # Both OpenAI and AIPipe use the same API format
+                response = self.client.chat.completions.create(
+                    model=self.model,
+                    messages=[{"role": "user", "content": prompt}],
+                    temperature=0.3,
+                    max_tokens=1024,
+                )
+                return response.choices[0].message.content
+
+        except Exception as e:
+            logger.error(f"LLM API call failed: {e}")
+            raise

instructor/checks/static_checks.py

@@ -0,0 +1,204 @@
+"""Static code analysis checks."""
+import re
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import git
+import requests
+from bs4 import BeautifulSoup
+
+from shared.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+
+class StaticChecker:
+    """Perform static checks on repository."""
+
+    def __init__(self, repo_url: str, commit_sha: str, clone_dir: Path) -> None:
+        """Initialize static checker.
+
+        Args:
+            repo_url: Repository URL
+            commit_sha: Commit SHA to check
+            clone_dir: Directory to clone repo
+        """
+        self.repo_url = repo_url
+        self.commit_sha = commit_sha
+        self.clone_dir = clone_dir
+        self.repo = None
+
+    def clone_repo(self) -> None:
+        """Clone repository and checkout specific commit."""
+        logger.info(f"Cloning {self.repo_url} to {self.clone_dir}")
+
+        # Clean up if exists
+        if self.clone_dir.exists():
+            import shutil
+
+            shutil.rmtree(self.clone_dir)
+
+        # Clone
+        self.repo = git.Repo.clone_from(self.repo_url, self.clone_dir)
+
+        # Checkout specific commit
+        self.repo.git.checkout(self.commit_sha)
+
+        logger.info(f"Checked out commit {self.commit_sha}")
+
+    def check_created_after_task(self, task_timestamp: datetime) -> dict[str, Any]:
+        """Check if repo was created after task was sent.
+
+        Args:
+            task_timestamp: When task was sent
+
+        Returns:
+            Check result
+        """
+        try:
+            # Get first commit timestamp
+            commits = list(self.repo.iter_commits())
+            if not commits:
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "No commits found",
+                }
+
+            first_commit = commits[-1]
+            first_commit_time = datetime.fromtimestamp(first_commit.committed_date)
+
+            if first_commit_time > task_timestamp:
+                return {
+                    "passed": True,
+                    "score": 1.0,
+                    "reason": f"Repo created after task ({first_commit_time} > {task_timestamp})",
+                }
+            else:
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": f"Repo existed before task ({first_commit_time} <= {task_timestamp})",
+                }
+
+        except Exception as e:
+            logger.error(f"Error checking repo creation time: {e}")
+            return {"passed": False, "score": 0.0, "reason": f"Error: {e}"}
+
+    def check_mit_license(self) -> dict[str, Any]:
+        """Check if repo has MIT LICENSE in root.
+
+        Returns:
+            Check result
+        """
+        try:
+            license_path = self.clone_dir / "LICENSE"
+
+            if not license_path.exists():
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "No LICENSE file found in root",
+                }
+
+            content = license_path.read_text()
+
+            if "MIT License" in content or "MIT" in content[:200]:
+                return {
+                    "passed": True,
+                    "score": 1.0,
+                    "reason": "MIT LICENSE found",
+                }
+            else:
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "LICENSE file exists but is not MIT",
+                }
+
+        except Exception as e:
+            logger.error(f"Error checking LICENSE: {e}")
+            return {"passed": False, "score": 0.0, "reason": f"Error: {e}"}
+
+    def check_readme(self) -> dict[str, Any]:
+        """Check README.md quality.
+
+        Returns:
+            Check result
+        """
+        try:
+            readme_path = self.clone_dir / "README.md"
+
+            if not readme_path.exists():
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": "No README.md found",
+                }
+
+            content = readme_path.read_text()
+
+            # Basic quality checks
+            has_title = bool(re.search(r"^#\s+.+", content, re.MULTILINE))
+            has_sections = content.count("#") >= 3
+            has_enough_content = len(content) >= 200
+            has_code_blocks = "```" in content
+
+            score = sum([has_title, has_sections, has_enough_content, has_code_blocks]) / 4
+
+            return {
+                "passed": score >= 0.75,
+                "score": score,
+                "reason": f"README quality: {score:.0%} (title={has_title}, "
+                f"sections={has_sections}, content={has_enough_content}, code={has_code_blocks})",
+            }
+
+        except Exception as e:
+            logger.error(f"Error checking README: {e}")
+            return {"passed": False, "score": 0.0, "reason": f"Error: {e}"}
+
+    def check_no_secrets(self) -> dict[str, Any]:
+        """Check for common secret patterns in git history.
+
+        Returns:
+            Check result
+        """
+        try:
+            # Check for common secret patterns
+            secret_patterns = [
+                r"sk-[a-zA-Z0-9]{32,}",  # API keys
+                r"ghp_[a-zA-Z0-9]{36,}",  # GitHub tokens
+                r"AKIA[0-9A-Z]{16}",  # AWS keys
+                r"password\s*=\s*['\"][^'\"]{8,}",  # Hardcoded passwords
+            ]
+
+            found_secrets = []
+
+            # Check all files in current commit
+            for item in self.clone_dir.rglob("*"):
+                if item.is_file() and not item.name.startswith(".git"):
+                    try:
+                        content = item.read_text(errors="ignore")
+                        for pattern in secret_patterns:
+                            if re.search(pattern, content, re.IGNORECASE):
+                                found_secrets.append(str(item.relative_to(self.clone_dir)))
+                    except Exception:
+                        pass
+
+            if found_secrets:
+                return {
+                    "passed": False,
+                    "score": 0.0,
+                    "reason": f"Potential secrets found in: {', '.join(found_secrets[:3])}",
+                }
+            else:
+                return {
+                    "passed": True,
+                    "score": 1.0,
+                    "reason": "No obvious secrets detected",
+                }
+
+        except Exception as e:
+            logger.error(f"Error checking secrets: {e}")
+            return {"passed": True, "score": 1.0, "reason": "Could not check secrets"}

instructor/database.py

@@ -0,0 +1,307 @@
+"""Database models and operations for instructor system."""
+import json
+from datetime import datetime
+from typing import Any
+
+from sqlalchemy import (
+    Column,
+    DateTime,
+    Float,
+    Integer,
+    String,
+    Text,
+    create_engine,
+)
+from sqlalchemy.orm import declarative_base, sessionmaker
+
+from shared.config import settings
+from shared.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+Base = declarative_base()
+
+
+class Task(Base):
+    """Task records sent to students."""
+
+    __tablename__ = "tasks"
+
+    id = Column(Integer, primary_key=True)
+    timestamp = Column(DateTime, default=datetime.utcnow, nullable=False)
+    email = Column(String(255), nullable=False, index=True)
+    task = Column(String(255), nullable=False, index=True)
+    round = Column(Integer, nullable=False)
+    nonce = Column(String(255), nullable=False, unique=True)
+    brief = Column(Text, nullable=False)
+    attachments = Column(Text, nullable=False)  # JSON serialized
+    checks = Column(Text, nullable=False)  # JSON serialized
+    evaluation_url = Column(String(512), nullable=False)
+    endpoint = Column(String(512), nullable=False)
+    statuscode = Column(Integer, nullable=True)
+    secret = Column(String(255), nullable=False)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "id": self.id,
+            "timestamp": self.timestamp.isoformat() if self.timestamp else None,
+            "email": self.email,
+            "task": self.task,
+            "round": self.round,
+            "nonce": self.nonce,
+            "brief": self.brief,
+            "attachments": json.loads(self.attachments) if self.attachments else [],
+            "checks": json.loads(self.checks) if self.checks else [],
+            "evaluation_url": self.evaluation_url,
+            "endpoint": self.endpoint,
+            "statuscode": self.statuscode,
+        }
+
+
+class Repo(Base):
+    """Repository submissions from students."""
+
+    __tablename__ = "repos"
+
+    id = Column(Integer, primary_key=True)
+    timestamp = Column(DateTime, default=datetime.utcnow, nullable=False)
+    email = Column(String(255), nullable=False, index=True)
+    task = Column(String(255), nullable=False, index=True)
+    round = Column(Integer, nullable=False)
+    nonce = Column(String(255), nullable=False, unique=True)
+    repo_url = Column(String(512), nullable=False)
+    commit_sha = Column(String(255), nullable=False)
+    pages_url = Column(String(512), nullable=False)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "id": self.id,
+            "timestamp": self.timestamp.isoformat() if self.timestamp else None,
+            "email": self.email,
+            "task": self.task,
+            "round": self.round,
+            "nonce": self.nonce,
+            "repo_url": self.repo_url,
+            "commit_sha": self.commit_sha,
+            "pages_url": self.pages_url,
+        }
+
+
+class Result(Base):
+    """Evaluation results."""
+
+    __tablename__ = "results"
+
+    id = Column(Integer, primary_key=True)
+    timestamp = Column(DateTime, default=datetime.utcnow, nullable=False)
+    email = Column(String(255), nullable=False, index=True)
+    task = Column(String(255), nullable=False, index=True)
+    round = Column(Integer, nullable=False)
+    repo_url = Column(String(512), nullable=False)
+    commit_sha = Column(String(255), nullable=False)
+    pages_url = Column(String(512), nullable=False)
+    check = Column(String(512), nullable=False)
+    score = Column(Float, nullable=False)
+    reason = Column(Text, nullable=False)
+    logs = Column(Text, nullable=True)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "id": self.id,
+            "timestamp": self.timestamp.isoformat() if self.timestamp else None,
+            "email": self.email,
+            "task": self.task,
+            "round": self.round,
+            "repo_url": self.repo_url,
+            "commit_sha": self.commit_sha,
+            "pages_url": self.pages_url,
+            "check": self.check,
+            "score": self.score,
+            "reason": self.reason,
+            "logs": self.logs,
+        }
+
+
+class Database:
+    """Database manager for instructor system."""
+
+    def __init__(self, database_url: str | None = None) -> None:
+        """Initialize database connection.
+
+        Args:
+            database_url: Database URL (uses settings if not provided)
+        """
+        self.database_url = database_url or settings.database_url
+        self.engine = create_engine(self.database_url, echo=False)
+        self.SessionLocal = sessionmaker(bind=self.engine)
+        logger.info(f"Initialized database: {self.database_url}")
+
+    def create_tables(self) -> None:
+        """Create all tables."""
+        Base.metadata.create_all(self.engine)
+        logger.info("Created database tables")
+
+    def drop_tables(self) -> None:
+        """Drop all tables (use with caution)."""
+        Base.metadata.drop_all(self.engine)
+        logger.warning("Dropped all database tables")
+
+    def get_session(self):
+        """Get database session."""
+        return self.SessionLocal()
+
+    # Task operations
+    def add_task(self, task_data: dict[str, Any]) -> Task:
+        """Add a task record.
+
+        Args:
+            task_data: Task data dictionary
+
+        Returns:
+            Created task record
+        """
+        session = self.get_session()
+        try:
+            task = Task(
+                email=task_data["email"],
+                task=task_data["task"],
+                round=task_data["round"],
+                nonce=task_data["nonce"],
+                brief=task_data["brief"],
+                attachments=json.dumps(task_data.get("attachments", [])),
+                checks=json.dumps(task_data.get("checks", [])),
+                evaluation_url=task_data["evaluation_url"],
+                endpoint=task_data["endpoint"],
+                statuscode=task_data.get("statuscode"),
+                secret=task_data["secret"],
+            )
+            session.add(task)
+            session.commit()
+            session.refresh(task)
+            logger.info(f"Added task: {task.task}, round {task.round}")
+            return task
+        finally:
+            session.close()
+
+    def get_task_by_nonce(self, nonce: str) -> Task | None:
+        """Get task by nonce.
+
+        Args:
+            nonce: Task nonce
+
+        Returns:
+            Task or None
+        """
+        session = self.get_session()
+        try:
+            return session.query(Task).filter(Task.nonce == nonce).first()
+        finally:
+            session.close()
+
+    def task_exists(self, email: str, task: str, round: int) -> bool:
+        """Check if task exists.
+
+        Args:
+            email: Student email
+            task: Task ID
+            round: Round number
+
+        Returns:
+            True if exists
+        """
+        session = self.get_session()
+        try:
+            return (
+                session.query(Task)
+                .filter(Task.email == email, Task.task == task, Task.round == round)
+                .first()
+                is not None
+            )
+        finally:
+            session.close()
+
+    # Repo operations
+    def add_repo(self, repo_data: dict[str, Any]) -> Repo:
+        """Add a repo submission.
+
+        Args:
+            repo_data: Repo data dictionary
+
+        Returns:
+            Created repo record
+        """
+        session = self.get_session()
+        try:
+            repo = Repo(**repo_data)
+            session.add(repo)
+            session.commit()
+            session.refresh(repo)
+            logger.info(f"Added repo: {repo.task}, round {repo.round}")
+            return repo
+        finally:
+            session.close()
+
+    def get_repos(self, email: str | None = None) -> list[Repo]:
+        """Get repo submissions.
+
+        Args:
+            email: Filter by email (optional)
+
+        Returns:
+            List of repos
+        """
+        session = self.get_session()
+        try:
+            query = session.query(Repo)
+            if email:
+                query = query.filter(Repo.email == email)
+            return query.all()
+        finally:
+            session.close()
+
+    # Result operations
+    def add_result(self, result_data: dict[str, Any]) -> Result:
+        """Add an evaluation result.
+
+        Args:
+            result_data: Result data dictionary
+
+        Returns:
+            Created result record
+        """
+        session = self.get_session()
+        try:
+            result = Result(**result_data)
+            session.add(result)
+            session.commit()
+            session.refresh(result)
+            logger.debug(f"Added result: {result.task}, {result.check}")
+            return result
+        finally:
+            session.close()
+
+    def get_results(
+        self, email: str | None = None, task: str | None = None
+    ) -> list[Result]:
+        """Get evaluation results.
+
+        Args:
+            email: Filter by email (optional)
+            task: Filter by task (optional)
+
+        Returns:
+            List of results
+        """
+        session = self.get_session()
+        try:
+            query = session.query(Result)
+            if email:
+                query = query.filter(Result.email == email)
+            if task:
+                query = query.filter(Result.task == task)
+            return query.all()
+        finally:
+            session.close()

instructor/evaluate.py

@@ -0,0 +1,258 @@
+"""Main evaluation script for running all checks."""
+import asyncio
+import json
+import shutil
+from datetime import datetime
+from pathlib import Path
+
+from instructor.checks.dynamic_checks import DynamicChecker
+from instructor.checks.llm_checks import LLMChecker
+from instructor.checks.static_checks import StaticChecker
+from instructor.database import Database
+from shared.config import settings
+from shared.logger import setup_logger
+
+logger = setup_logger(__name__)
+
+
+class Evaluator:
+    """Main evaluator for running all checks on submissions."""
+
+    def __init__(self) -> None:
+        """Initialize evaluator."""
+        self.db = Database()
+        self.llm_checker = LLMChecker()
+        self.temp_dir = Path("./temp_evaluations")
+        self.temp_dir.mkdir(parents=True, exist_ok=True)
+
+    def get_pending_repos(self) -> list[dict]:
+        """Get repos that need evaluation.
+
+        Returns:
+            List of repo dictionaries
+        """
+        all_repos = self.db.get_repos()
+        pending = []
+
+        for repo in all_repos:
+            repo_dict = repo.to_dict()
+
+            # Check if already evaluated
+            existing_results = self.db.get_results(
+                email=repo_dict["email"], task=repo_dict["task"]
+            )
+
+            # Filter to this round
+            round_results = [
+                r for r in existing_results if r.to_dict()["round"] == repo_dict["round"]
+            ]
+
+            if not round_results:
+                pending.append(repo_dict)
+
+        logger.info(f"Found {len(pending)} pending repos for evaluation")
+        return pending
+
+    async def evaluate_repo(self, repo: dict) -> None:
+        """Evaluate a single repository.
+
+        Args:
+            repo: Repository submission data
+        """
+        logger.info(f"Evaluating {repo['email']}/{repo['task']}, round {repo['round']}")
+
+        # Get task details
+        session = self.db.get_session()
+        try:
+            task = (
+                session.query(self.db.Task)
+                .filter_by(
+                    email=repo["email"],
+                    task=repo["task"],
+                    round=repo["round"],
+                )
+                .first()
+            )
+
+            if not task:
+                logger.error(f"No task found for {repo['task']}")
+                return
+
+            task_dict = task.to_dict()
+            checks_list = task_dict["checks"]
+
+        finally:
+            session.close()
+
+        # Clone repo
+        clone_dir = self.temp_dir / f"{repo['task']}_{repo['round']}"
+        static_checker = StaticChecker(repo["repo_url"], repo["commit_sha"], clone_dir)
+
+        try:
+            static_checker.clone_repo()
+
+            # Run static checks
+            logger.info("Running static checks...")
+            static_results = await self._run_static_checks(
+                static_checker, task_dict["timestamp"]
+            )
+
+            # Run LLM checks
+            logger.info("Running LLM checks...")
+            llm_results = self._run_llm_checks(clone_dir)
+
+            # Run dynamic checks
+            logger.info("Running dynamic checks...")
+            dynamic_results = await self._run_dynamic_checks(repo["pages_url"], checks_list)
+
+            # Save all results
+            all_results = static_results + llm_results + dynamic_results
+            for result in all_results:
+                self.db.add_result(
+                    {
+                        "timestamp": datetime.utcnow(),
+                        "email": repo["email"],
+                        "task": repo["task"],
+                        "round": repo["round"],
+                        "repo_url": repo["repo_url"],
+                        "commit_sha": repo["commit_sha"],
+                        "pages_url": repo["pages_url"],
+                        "check": result["check"],
+                        "score": result["score"],
+                        "reason": result["reason"],
+                        "logs": result.get("logs", ""),
+                    }
+                )
+
+            logger.info(
+                f"Completed evaluation for {repo['email']}/{repo['task']}: "
+                f"{len(all_results)} checks"
+            )
+
+        except Exception as e:
+            logger.error(f"Error evaluating {repo['task']}: {e}", exc_info=True)
+
+            # Save error result
+            self.db.add_result(
+                {
+                    "timestamp": datetime.utcnow(),
+                    "email": repo["email"],
+                    "task": repo["task"],
+                    "round": repo["round"],
+                    "repo_url": repo["repo_url"],
+                    "commit_sha": repo["commit_sha"],
+                    "pages_url": repo["pages_url"],
+                    "check": "Evaluation",
+                    "score": 0.0,
+                    "reason": f"Evaluation failed: {e}",
+                    "logs": str(e),
+                }
+            )
+
+        finally:
+            # Cleanup clone
+            if clone_dir.exists():
+                shutil.rmtree(clone_dir)
+
+    async def _run_static_checks(
+        self, checker: StaticChecker, task_timestamp: str
+    ) -> list[dict]:
+        """Run static checks.
+
+        Args:
+            checker: Static checker instance
+            task_timestamp: When task was sent
+
+        Returns:
+            List of check results
+        """
+        results = []
+
+        # Parse timestamp
+        if isinstance(task_timestamp, str):
+            task_time = datetime.fromisoformat(task_timestamp.replace("Z", "+00:00"))
+        else:
+            task_time = task_timestamp
+
+        # Repo creation time
+        result = checker.check_created_after_task(task_time)
+        results.append({"check": "Repo created after task", **result})
+
+        # MIT License
+        result = checker.check_mit_license()
+        results.append({"check": "MIT LICENSE exists", **result})
+
+        # README exists and basic quality
+        result = checker.check_readme()
+        results.append({"check": "README.md basic quality", **result})
+
+        # No secrets
+        result = checker.check_no_secrets()
+        results.append({"check": "No secrets in code", **result})
+
+        return results
+
+    def _run_llm_checks(self, code_dir: Path) -> list[dict]:
+        """Run LLM-based checks.
+
+        Args:
+            code_dir: Directory containing code
+
+        Returns:
+            List of check results
+        """
+        results = []
+
+        # README quality
+        readme_path = code_dir / "README.md"
+        if readme_path.exists():
+            result = self.llm_checker.check_readme_quality(readme_path)
+            results.append({"check": "LLM: README.md quality", **result})
+
+        # Code quality
+        result = self.llm_checker.check_code_quality(code_dir)
+        results.append({"check": "LLM: Code quality", **result})
+
+        return results
+
+    async def _run_dynamic_checks(
+        self, pages_url: str, checks: list[str]
+    ) -> list[dict]:
+        """Run dynamic Playwright checks.
+
+        Args:
+            pages_url: GitHub Pages URL
+            checks: List of checks to run
+
+        Returns:
+            List of check results
+        """
+        dynamic_checker = DynamicChecker(pages_url, checks)
+        results = await dynamic_checker.run_checks()
+        return results
+
+    async def run(self) -> None:
+        """Run evaluation on all pending repos."""
+        logger.info("Starting evaluation process")
+
+        pending_repos = self.get_pending_repos()
+
+        if not pending_repos:
+            logger.info("No pending repos to evaluate")
+            return
+
+        # Evaluate each repo
+        for repo in pending_repos:
+            await self.evaluate_repo(repo)
+
+        logger.info("Evaluation process complete")
+
+
+async def main():
+    """Main entry point."""
+    evaluator = Evaluator()
+    await evaluator.run()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

instructor/round1.py

@@ -0,0 +1,223 @@
+"""Round 1 task generation and distribution script."""
+import csv
+import json
+from datetime import datetime
+
+import httpx
+
+from instructor.database import Database
+from instructor.task_templates import TaskTemplateManager
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.models import Attachment, TaskRequest
+from shared.utils import generate_nonce
+
+logger = setup_logger(__name__)
+
+
+class Round1TaskGenerator:
+    """Generate and send round 1 tasks to students."""
+
+    def __init__(self) -> None:
+        """Initialize task generator."""
+        self.db = Database()
+        self.template_manager = TaskTemplateManager()
+        self.db.create_tables()
+
+    def load_submissions(self) -> list[dict]:
+        """Load submissions from CSV file.
+
+        Returns:
+            List of submission dictionaries
+        """
+        submissions = []
+
+        if not settings.submissions_csv.exists():
+            logger.error(f"Submissions file not found: {settings.submissions_csv}")
+            return submissions
+
+        with open(settings.submissions_csv, "r") as f:
+            reader = csv.DictReader(f)
+            for row in reader:
+                submissions.append(
+                    {
+                        "timestamp": row["timestamp"],
+                        "email": row["email"],
+                        "endpoint": row["endpoint"],
+                        "secret": row["secret"],
+                    }
+                )
+
+        logger.info(f"Loaded {len(submissions)} submissions")
+        return submissions
+
+    def generate_task_request(self, submission: dict) -> TaskRequest:
+        """Generate task request for a submission.
+
+        Args:
+            submission: Submission data
+
+        Returns:
+            Task request
+        """
+        email = submission["email"]
+
+        # Check if round 1 task already exists
+        # We check if there's a successful task (with statuscode 200)
+        session = self.db.get_session()
+        try:
+            existing = (
+                session.query(self.db.Task)
+                .filter_by(email=email, round=1, statuscode=200)
+                .first()
+            )
+            if existing:
+                logger.info(f"Round 1 task already exists for {email}, skipping")
+                return None
+        finally:
+            session.close()
+
+        # Generate task from random template
+        task_data = self.template_manager.generate_task(email, round_num=1)
+
+        # Create task request
+        nonce = generate_nonce()
+        attachments = [Attachment(**att) for att in task_data["attachments"]]
+
+        task_request = TaskRequest(
+            email=email,
+            secret=submission["secret"],
+            task=task_data["task_id"],
+            round=1,
+            nonce=nonce,
+            brief=task_data["brief"],
+            checks=task_data["checks"],
+            evaluation_url=settings.evaluation_api_url,
+            attachments=attachments,
+        )
+
+        logger.info(f"Generated task request for {email}: {task_request.task}")
+        return task_request, submission
+
+    async def send_task_request(
+        self, task_request: TaskRequest, submission: dict
+    ) -> int:
+        """Send task request to student endpoint.
+
+        Args:
+            task_request: Task request to send
+            submission: Submission data with endpoint
+
+        Returns:
+            HTTP status code
+        """
+        endpoint = submission["endpoint"]
+        logger.info(f"Sending task to {endpoint}")
+
+        try:
+            async with httpx.AsyncClient(timeout=30.0) as client:
+                response = await client.post(
+                    endpoint,
+                    json=task_request.model_dump(),
+                    headers={"Content-Type": "application/json"},
+                )
+
+                status_code = response.status_code
+                logger.info(
+                    f"Task sent to {endpoint}: "
+                    f"status {status_code}, response: {response.text[:200]}"
+                )
+
+                return status_code
+
+        except Exception as e:
+            logger.error(f"Failed to send task to {endpoint}: {e}")
+            return 0  # Use 0 to indicate failure
+
+    def save_task_record(
+        self, task_request: TaskRequest, submission: dict, status_code: int
+    ) -> None:
+        """Save task record to database.
+
+        Args:
+            task_request: Task request
+            submission: Submission data
+            status_code: HTTP status code from response
+        """
+        task_data = {
+            "timestamp": datetime.utcnow(),
+            "email": task_request.email,
+            "task": task_request.task,
+            "round": task_request.round,
+            "nonce": task_request.nonce,
+            "brief": task_request.brief,
+            "attachments": json.dumps([att.model_dump() for att in task_request.attachments]),
+            "checks": json.dumps(task_request.checks),
+            "evaluation_url": task_request.evaluation_url,
+            "endpoint": submission["endpoint"],
+            "statuscode": status_code,
+            "secret": submission["secret"],
+        }
+
+        self.db.add_task(task_data)
+        logger.info(f"Saved task record: {task_request.task}")
+
+    async def process_submission(self, submission: dict) -> None:
+        """Process a single submission.
+
+        Args:
+            submission: Submission data
+        """
+        try:
+            # Generate task
+            result = self.generate_task_request(submission)
+            if result is None:
+                return  # Already processed
+
+            task_request, submission = result
+
+            # Send task
+            status_code = await self.send_task_request(task_request, submission)
+
+            # Save record
+            self.save_task_record(task_request, submission, status_code)
+
+            if status_code == 200:
+                logger.info(f"Successfully sent round 1 task to {submission['email']}")
+            else:
+                logger.warning(
+                    f"Failed to send round 1 task to {submission['email']}: "
+                    f"status {status_code}"
+                )
+
+        except Exception as e:
+            logger.error(f"Error processing submission {submission['email']}: {e}", exc_info=True)
+
+    async def run(self) -> None:
+        """Run round 1 task generation."""
+        logger.info("Starting round 1 task generation")
+
+        # Load submissions
+        submissions = self.load_submissions()
+
+        if not submissions:
+            logger.error("No submissions to process")
+            return
+
+        # Process each submission
+        for submission in submissions:
+            await self.process_submission(submission)
+
+        logger.info("Round 1 task generation complete")
+
+
+async def main():
+    """Main entry point."""
+    generator = Round1TaskGenerator()
+    await generator.run()
+
+
+if __name__ == "__main__":
+    import asyncio
+
+    asyncio.run(main())

instructor/round2.py

@@ -0,0 +1,227 @@
+"""Round 2 task generation and distribution script."""
+import json
+from datetime import datetime
+
+import httpx
+
+from instructor.database import Database
+from instructor.task_templates import TaskTemplateManager
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.models import Attachment, TaskRequest
+from shared.utils import generate_nonce
+
+logger = setup_logger(__name__)
+
+
+class Round2TaskGenerator:
+    """Generate and send round 2 tasks to students."""
+
+    def __init__(self) -> None:
+        """Initialize task generator."""
+        self.db = Database()
+        self.template_manager = TaskTemplateManager()
+
+    def get_round1_repos(self) -> list[dict]:
+        """Get all round 1 repository submissions.
+
+        Returns:
+            List of repo dictionaries
+        """
+        repos = []
+        session = self.db.get_session()
+        try:
+            round1_repos = session.query(self.db.Repo).filter_by(round=1).all()
+            repos = [repo.to_dict() for repo in round1_repos]
+        finally:
+            session.close()
+
+        logger.info(f"Found {len(repos)} round 1 repos")
+        return repos
+
+    def generate_round2_task(self, repo: dict) -> tuple[TaskRequest, dict]:
+        """Generate round 2 task for a repo.
+
+        Args:
+            repo: Round 1 repo submission
+
+        Returns:
+            Tuple of (task_request, submission_data) or None if skipped
+        """
+        email = repo["email"]
+        task_id = repo["task"]
+
+        # Check if round 2 task already exists
+        if self.db.task_exists(email, task_id, round=2):
+            logger.info(f"Round 2 task already exists for {email}/{task_id}, skipping")
+            return None
+
+        # Get the round 1 task to find the template
+        session = self.db.get_session()
+        try:
+            round1_task = (
+                session.query(self.db.Task)
+                .filter_by(email=email, task=task_id, round=1)
+                .first()
+            )
+
+            if not round1_task:
+                logger.warning(f"No round 1 task found for {email}/{task_id}")
+                return None
+
+            # Extract template ID from task ID
+            template_id = task_id.rsplit("-", 1)[0]
+
+            # Generate round 2 task from same template
+            task_data = self.template_manager.generate_task(
+                email, template_id=template_id, round_num=2
+            )
+
+            # Create task request
+            nonce = generate_nonce()
+            attachments = [Attachment(**att) for att in task_data["attachments"]]
+
+            task_request = TaskRequest(
+                email=email,
+                secret=round1_task.secret,
+                task=task_id,  # Keep same task ID
+                round=2,
+                nonce=nonce,
+                brief=task_data["brief"],
+                checks=task_data["checks"],
+                evaluation_url=settings.evaluation_api_url,
+                attachments=attachments,
+            )
+
+            submission_data = {"endpoint": round1_task.endpoint, "secret": round1_task.secret}
+
+            logger.info(f"Generated round 2 task for {email}: {task_id}")
+            return task_request, submission_data
+
+        except Exception as e:
+            logger.error(f"Error generating round 2 task for {email}/{task_id}: {e}")
+            return None
+        finally:
+            session.close()
+
+    async def send_task_request(
+        self, task_request: TaskRequest, submission: dict
+    ) -> int:
+        """Send task request to student endpoint.
+
+        Args:
+            task_request: Task request to send
+            submission: Submission data with endpoint
+
+        Returns:
+            HTTP status code
+        """
+        endpoint = submission["endpoint"]
+        logger.info(f"Sending round 2 task to {endpoint}")
+
+        try:
+            async with httpx.AsyncClient(timeout=30.0) as client:
+                response = await client.post(
+                    endpoint,
+                    json=task_request.model_dump(),
+                    headers={"Content-Type": "application/json"},
+                )
+
+                status_code = response.status_code
+                logger.info(
+                    f"Round 2 task sent to {endpoint}: "
+                    f"status {status_code}, response: {response.text[:200]}"
+                )
+
+                return status_code
+
+        except Exception as e:
+            logger.error(f"Failed to send round 2 task to {endpoint}: {e}")
+            return 0
+
+    def save_task_record(
+        self, task_request: TaskRequest, submission: dict, status_code: int
+    ) -> None:
+        """Save task record to database.
+
+        Args:
+            task_request: Task request
+            submission: Submission data
+            status_code: HTTP status code from response
+        """
+        task_data = {
+            "timestamp": datetime.utcnow(),
+            "email": task_request.email,
+            "task": task_request.task,
+            "round": task_request.round,
+            "nonce": task_request.nonce,
+            "brief": task_request.brief,
+            "attachments": json.dumps([att.model_dump() for att in task_request.attachments]),
+            "checks": json.dumps(task_request.checks),
+            "evaluation_url": task_request.evaluation_url,
+            "endpoint": submission["endpoint"],
+            "statuscode": status_code,
+            "secret": submission["secret"],
+        }
+
+        self.db.add_task(task_data)
+        logger.info(f"Saved round 2 task record: {task_request.task}")
+
+    async def process_repo(self, repo: dict) -> None:
+        """Process a single repo for round 2.
+
+        Args:
+            repo: Repo submission data
+        """
+        try:
+            # Generate task
+            result = self.generate_round2_task(repo)
+            if result is None:
+                return  # Already processed or error
+
+            task_request, submission = result
+
+            # Send task
+            status_code = await self.send_task_request(task_request, submission)
+
+            # Save record
+            self.save_task_record(task_request, submission, status_code)
+
+            if status_code == 200:
+                logger.info(f"Successfully sent round 2 task to {repo['email']}")
+            else:
+                logger.warning(
+                    f"Failed to send round 2 task to {repo['email']}: status {status_code}"
+                )
+
+        except Exception as e:
+            logger.error(f"Error processing repo {repo['task']}: {e}", exc_info=True)
+
+    async def run(self) -> None:
+        """Run round 2 task generation."""
+        logger.info("Starting round 2 task generation")
+
+        # Get round 1 repos
+        repos = self.get_round1_repos()
+
+        if not repos:
+            logger.error("No round 1 repos to process")
+            return
+
+        # Process each repo
+        for repo in repos:
+            await self.process_repo(repo)
+
+        logger.info("Round 2 task generation complete")
+
+
+async def main():
+    """Main entry point."""
+    generator = Round2TaskGenerator()
+    await generator.run()
+
+
+if __name__ == "__main__":
+    import asyncio
+
+    asyncio.run(main())

instructor/task_templates.py

@@ -0,0 +1,312 @@
+"""Task template management and parametrization."""
+import csv
+import hashlib
+import random
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import yaml
+from jinja2 import Template
+
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.utils import encode_data_uri, generate_task_id
+
+logger = setup_logger(__name__)
+
+
+class TaskTemplate:
+    """Single task template with parametrization."""
+
+    def __init__(self, data: dict[str, Any]) -> None:
+        """Initialize task template.
+
+        Args:
+            data: Template data from YAML
+        """
+        self.id = data["id"]
+        self.brief = data["brief"]
+        self.attachments = data.get("attachments", [])
+        self.checks = data.get("checks", [])
+        self.round2 = data.get("round2", [])
+
+    def parametrize(self, seed: str) -> dict[str, Any]:
+        """Parametrize template with seed.
+
+        Args:
+            seed: Seed string for randomization
+
+        Returns:
+            Parametrized task data
+        """
+        # Set random seed for reproducibility
+        random.seed(seed)
+
+        # Generate data based on template
+        context = self._generate_context(seed)
+
+        # Parametrize brief
+        brief = Template(self.brief).render(**context)
+
+        # Parametrize attachments
+        attachments = []
+        for att_template in self.attachments:
+            att_name = Template(att_template["name"]).render(**context)
+            att_url_template = att_template["url"]
+
+            # Generate attachment content
+            if "data:text/csv" in att_url_template:
+                content = self._generate_csv_data(seed)
+                att_url = encode_data_uri("text/csv", content.encode())
+            elif "data:text/markdown" in att_url_template:
+                content = self._generate_markdown_data(seed)
+                att_url = encode_data_uri("text/markdown", content.encode())
+            elif "data:application/json" in att_url_template:
+                content = self._generate_json_data(seed)
+                att_url = encode_data_uri("application/json", content.encode())
+            else:
+                att_url = Template(att_url_template).render(**context)
+
+            attachments.append({"name": att_name, "url": att_url})
+
+        # Parametrize checks
+        checks = [Template(check).render(**context) for check in self.checks]
+
+        return {
+            "brief": brief,
+            "attachments": attachments,
+            "checks": checks,
+            "context": context,
+        }
+
+    def parametrize_round2(self, seed: str, round2_index: int = 0) -> dict[str, Any]:
+        """Parametrize round 2 task.
+
+        Args:
+            seed: Seed string
+            round2_index: Index of round2 option to use
+
+        Returns:
+            Parametrized round 2 task data
+        """
+        if not self.round2 or round2_index >= len(self.round2):
+            raise ValueError(f"No round2 option at index {round2_index}")
+
+        round2_task = self.round2[round2_index]
+        random.seed(seed + f"-round2-{round2_index}")
+
+        context = self._generate_context(seed)
+
+        brief = Template(round2_task["brief"]).render(**context)
+        checks = [Template(check).render(**context) for check in round2_task.get("checks", [])]
+
+        attachments = []
+        for att_template in round2_task.get("attachments", []):
+            att_name = Template(att_template["name"]).render(**context)
+            att_url = Template(att_template["url"]).render(**context)
+            attachments.append({"name": att_name, "url": att_url})
+
+        return {
+            "brief": brief,
+            "attachments": attachments,
+            "checks": checks,
+            "context": context,
+        }
+
+    def _generate_context(self, seed: str) -> dict[str, Any]:
+        """Generate context variables for template.
+
+        Args:
+            seed: Seed string
+
+        Returns:
+            Context dictionary
+        """
+        # Generate deterministic values based on seed
+        hash_val = int(hashlib.md5(seed.encode()).hexdigest(), 16)
+
+        return {
+            "seed": seed,
+            "hash": hash_val % 10000,
+            "result": (hash_val % 9000) + 1000,  # Result between 1000-9999
+        }
+
+    def _generate_csv_data(self, seed: str) -> str:
+        """Generate sample CSV data.
+
+        Args:
+            seed: Seed for randomization
+
+        Returns:
+            CSV string
+        """
+        random.seed(seed)
+
+        products = ["Widget", "Gadget", "Doohickey", "Thingamajig", "Gizmo"]
+        regions = ["North", "South", "East", "West", "Central"]
+
+        lines = ["product,region,sales"]
+        for _ in range(random.randint(10, 20)):
+            product = random.choice(products)
+            region = random.choice(regions)
+            sales = round(random.uniform(100, 1000), 2)
+            lines.append(f"{product},{region},{sales}")
+
+        return "\n".join(lines)
+
+    def _generate_markdown_data(self, seed: str) -> str:
+        """Generate sample Markdown data.
+
+        Args:
+            seed: Seed for randomization
+
+        Returns:
+            Markdown string
+        """
+        random.seed(seed)
+
+        content = f"""# Sample Document {seed[:8]}
+
+## Introduction
+
+This is a sample markdown document generated for testing.
+
+## Code Example
+
+```python
+def hello():
+    print("Hello, world!")
+```
+
+## List
+
+- Item 1
+- Item 2
+- Item 3
+
+## Conclusion
+
+This document contains {random.randint(50, 150)} words.
+"""
+        return content
+
+    def _generate_json_data(self, seed: str) -> str:
+        """Generate sample JSON data.
+
+        Args:
+            seed: Seed for randomization
+
+        Returns:
+            JSON string
+        """
+        import json
+
+        random.seed(seed)
+
+        data = {
+            "USD": 1.0,
+            "EUR": round(random.uniform(0.8, 0.95), 2),
+            "GBP": round(random.uniform(0.7, 0.85), 2),
+            "JPY": round(random.uniform(110, 140), 2),
+        }
+
+        return json.dumps(data, indent=2)
+
+
+class TaskTemplateManager:
+    """Manage task templates."""
+
+    def __init__(self, templates_dir: Path | None = None) -> None:
+        """Initialize template manager.
+
+        Args:
+            templates_dir: Directory containing template YAML files
+        """
+        self.templates_dir = templates_dir or settings.task_templates_dir
+        self.templates: dict[str, TaskTemplate] = {}
+        self.load_templates()
+
+    def load_templates(self) -> None:
+        """Load all templates from directory."""
+        if not self.templates_dir.exists():
+            logger.warning(f"Templates directory not found: {self.templates_dir}")
+            return
+
+        for yaml_file in self.templates_dir.glob("*.yaml"):
+            try:
+                with open(yaml_file) as f:
+                    data = yaml.safe_load(f)
+                    template = TaskTemplate(data)
+                    self.templates[template.id] = template
+                    logger.info(f"Loaded template: {template.id}")
+            except Exception as e:
+                logger.error(f"Failed to load template {yaml_file}: {e}")
+
+        logger.info(f"Loaded {len(self.templates)} templates")
+
+    def get_template(self, template_id: str) -> TaskTemplate:
+        """Get template by ID.
+
+        Args:
+            template_id: Template identifier
+
+        Returns:
+            Task template
+
+        Raises:
+            KeyError: If template not found
+        """
+        if template_id not in self.templates:
+            raise KeyError(f"Template not found: {template_id}")
+        return self.templates[template_id]
+
+    def get_random_template(self) -> TaskTemplate:
+        """Get random template.
+
+        Returns:
+            Random task template
+        """
+        if not self.templates:
+            raise ValueError("No templates available")
+        return random.choice(list(self.templates.values()))
+
+    def generate_task(
+        self, email: str, template_id: str | None = None, round_num: int = 1
+    ) -> dict[str, Any]:
+        """Generate a complete task from template.
+
+        Args:
+            email: Student email
+            template_id: Template ID (random if None)
+            round_num: Round number
+
+        Returns:
+            Complete task data
+        """
+        # Get template
+        if template_id:
+            template = self.get_template(template_id)
+        else:
+            template = self.get_random_template()
+
+        # Generate seed
+        now = datetime.utcnow()
+        seed = f"{email}-{now.strftime('%Y-%m-%d-%H')}"
+
+        # Parametrize
+        if round_num == 1:
+            task_data = template.parametrize(seed)
+        else:
+            # For round 2, pick random round2 option
+            round2_index = random.randint(0, len(template.round2) - 1) if template.round2 else 0
+            task_data = template.parametrize_round2(seed, round2_index)
+
+        # Generate task ID
+        task_id = generate_task_id(template.id, task_data["brief"], task_data["attachments"])
+
+        return {
+            "template_id": template.id,
+            "task_id": task_id,
+            **task_data,
+        }

main.py

@@ -0,0 +1,95 @@
+"""Main entry point for LLM Code Deployment System."""
+import argparse
+import asyncio
+import sys
+
+
+def main():
+    """Main CLI entry point."""
+    parser = argparse.ArgumentParser(
+        description="LLM Code Deployment System",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  # Start student API
+  python main.py student-api
+  
+  # Start instructor API
+  python main.py instructor-api
+  
+  # Run round 1 task generation
+  python main.py round1
+  
+  # Run evaluation
+  python main.py evaluate
+  
+  # Run round 2 task generation
+  python main.py round2
+  
+  # Initialize database
+  python main.py init-db
+        """,
+    )
+
+    parser.add_argument(
+        "command",
+        choices=[
+            "student-api",
+            "instructor-api",
+            "round1",
+            "round2",
+            "evaluate",
+            "init-db",
+        ],
+        help="Command to run",
+    )
+
+    args = parser.parse_args()
+
+    if args.command == "student-api":
+        from student.api import app
+        import uvicorn
+        from shared.config import settings
+
+        settings.ensure_directories()
+        print(f"Starting Student API on port {settings.student_api_port}...")
+        uvicorn.run(app, host="0.0.0.0", port=settings.student_api_port)
+
+    elif args.command == "instructor-api":
+        from instructor.api import app
+        import uvicorn
+        from shared.config import settings
+
+        settings.ensure_directories()
+        print(f"Starting Instructor API on port {settings.instructor_api_port}...")
+        uvicorn.run(app, host="0.0.0.0", port=settings.instructor_api_port)
+
+    elif args.command == "round1":
+        from instructor.round1 import main as round1_main
+
+        print("Running Round 1 task generation...")
+        asyncio.run(round1_main())
+
+    elif args.command == "round2":
+        from instructor.round2 import main as round2_main
+
+        print("Running Round 2 task generation...")
+        asyncio.run(round2_main())
+
+    elif args.command == "evaluate":
+        from instructor.evaluate import main as evaluate_main
+
+        print("Running evaluation...")
+        asyncio.run(evaluate_main())
+
+    elif args.command == "init-db":
+        from instructor.database import Database
+
+        print("Initializing database...")
+        db = Database()
+        db.create_tables()
+        print("Database initialized successfully!")
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -0,0 +1,52 @@
+[project]
+name = "llm-code-deployment"
+version = "1.0.0"
+description = "LLM-powered code deployment and evaluation system for automated app building and testing"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+    "fastapi>=0.104.1",
+    "uvicorn[standard]>=0.24.0",
+    "anthropic>=0.34.0",
+    "openai>=1.3.0",
+    "pydantic>=2.5.0",
+    "pydantic-settings>=2.1.0",
+    "requests>=2.31.0",
+    "httpx>=0.25.0",
+    "PyGithub>=2.1.1",
+    "gitpython>=3.1.40",
+    "python-dotenv>=1.0.0",
+    "sqlalchemy>=2.0.23",
+    "alembic>=1.12.1",
+    "psycopg2-binary>=2.9.9",
+    "pyyaml>=6.0.1",
+    "jinja2>=3.1.2",
+    "playwright>=1.40.0",
+    "pandas>=2.1.3",
+    "beautifulsoup4>=4.12.2",
+    "tenacity>=8.2.3",
+    "uuid-utils>=0.9.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.4.3",
+    "pytest-asyncio>=0.21.1",
+    "black>=23.11.0",
+    "ruff>=0.1.6",
+    "mypy>=1.7.0",
+]
+
+[tool.black]
+line-length = 100
+target-version = ['py310']
+
+[tool.ruff]
+line-length = 100
+select = ["E", "F", "I", "N", "W"]
+ignore = ["E501"]
+
+[tool.mypy]
+python_version = "3.10"
+warn_return_any = true
+warn_unused_configs = true

requirements.txt

@@ -0,0 +1,21 @@
+fastapi>=0.104.1
+uvicorn[standard]>=0.24.0
+anthropic>=0.34.0
+openai>=1.3.0
+pydantic>=2.5.0
+pydantic-settings>=2.1.0
+requests>=2.31.0
+httpx>=0.25.0
+PyGithub>=2.1.1
+gitpython>=3.1.40
+python-dotenv>=1.0.0
+sqlalchemy>=2.0.23
+alembic>=1.12.1
+psycopg2-binary>=2.9.9
+pyyaml>=6.0.1
+jinja2>=3.1.2
+playwright>=1.40.0
+pandas>=2.1.3
+beautifulsoup4>=4.12.2
+tenacity>=8.2.3
+uuid-utils>=0.9.0

shared/__init__.py

@@ -0,0 +1 @@
+"""Shared modules for LLM Code Deployment system."""

shared/config.py

@@ -0,0 +1,82 @@
+"""Configuration management for LLM Code Deployment system."""
+import os
+from pathlib import Path
+from typing import Literal
+
+from pydantic import Field
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class Settings(BaseSettings):
+    """Application settings loaded from environment variables."""
+
+    model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
+
+    # Student Configuration
+    student_secret: str = Field(default="", description="Secret key for authentication")
+    student_email: str = Field(default="", description="Student email address")
+    student_api_port: int = Field(default=8000, description="Student API port")
+
+    # GitHub Configuration
+    github_token: str = Field(default="", description="GitHub personal access token")
+    github_username: str = Field(default="", description="GitHub username")
+
+    # LLM Configuration
+    anthropic_api_key: str = Field(default="", description="Anthropic API key")
+    openai_api_key: str = Field(default="", description="OpenAI API key")
+    llm_provider: Literal["anthropic", "openai", "aipipe"] = Field(
+        default="aipipe", description="LLM provider to use"
+    )
+    llm_model: str = Field(
+        default="google/gemini-2.0-flash-lite-001", description="LLM model to use"
+    )
+
+    # AIPipe Configuration
+    aipipe_token: str = Field(default="", description="AIPipe API token")
+    aipipe_base_url: str = Field(
+        default="https://aipipe.org/openrouter/v1",
+        description="AIPipe base URL"
+    )
+
+    # Instructor Configuration
+    instructor_api_port: int = Field(default=8001, description="Instructor API port")
+    database_url: str = Field(
+        default="postgresql://localhost:5432/llm_deployment", description="Database URL"
+    )
+    evaluation_api_url: str = Field(
+        default="http://localhost:8001/api/evaluate", description="Evaluation API URL"
+    )
+
+    # Task Configuration
+    task_timeout_minutes: int = Field(default=10, description="Task timeout in minutes")
+    max_retry_attempts: int = Field(default=3, description="Maximum retry attempts")
+    retry_delays: str = Field(default="1,2,4,8", description="Retry delays in seconds")
+
+    # Paths
+    generated_repos_dir: Path = Field(
+        default=Path("./generated_repos"), description="Directory for generated repos"
+    )
+    task_templates_dir: Path = Field(
+        default=Path("./templates"), description="Directory for task templates"
+    )
+    submissions_csv: Path = Field(
+        default=Path("./submissions.csv"), description="Path to submissions CSV"
+    )
+
+    # Logging
+    log_level: str = Field(default="INFO", description="Logging level")
+    log_file: Path = Field(default=Path("./logs/app.log"), description="Log file path")
+
+    def get_retry_delays(self) -> list[int]:
+        """Parse retry delays from comma-separated string."""
+        return [int(d.strip()) for d in self.retry_delays.split(",")]
+
+    def ensure_directories(self) -> None:
+        """Ensure all required directories exist."""
+        self.generated_repos_dir.mkdir(parents=True, exist_ok=True)
+        self.task_templates_dir.mkdir(parents=True, exist_ok=True)
+        self.log_file.parent.mkdir(parents=True, exist_ok=True)
+
+
+# Global settings instance
+settings = Settings()

shared/logger.py

@@ -0,0 +1,39 @@
+"""Logging configuration for LLM Code Deployment system."""
+import logging
+import sys
+from pathlib import Path
+
+from .config import settings
+
+
+def setup_logger(name: str) -> logging.Logger:
+    """Set up logger with file and console handlers."""
+    logger = logging.getLogger(name)
+    logger.setLevel(getattr(logging, settings.log_level.upper()))
+
+    # Avoid adding handlers multiple times
+    if logger.handlers:
+        return logger
+
+    # Console handler
+    console_handler = logging.StreamHandler(sys.stdout)
+    console_handler.setLevel(logging.INFO)
+    console_formatter = logging.Formatter(
+        "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+    console_handler.setFormatter(console_formatter)
+    logger.addHandler(console_handler)
+
+    # File handler
+    settings.log_file.parent.mkdir(parents=True, exist_ok=True)
+    file_handler = logging.FileHandler(settings.log_file)
+    file_handler.setLevel(logging.DEBUG)
+    file_formatter = logging.Formatter(
+        "%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+    file_handler.setFormatter(file_formatter)
+    logger.addHandler(file_handler)
+
+    return logger

shared/models.py

@@ -0,0 +1,93 @@
+"""Shared data models for LLM Code Deployment system."""
+from datetime import datetime
+from typing import Any
+
+from pydantic import BaseModel, Field, HttpUrl
+
+
+class Attachment(BaseModel):
+    """File attachment with data URI."""
+
+    name: str = Field(..., description="Attachment filename")
+    url: str = Field(..., description="Data URI or URL of the attachment")
+
+
+class TaskRequest(BaseModel):
+    """Task request sent to student endpoint."""
+
+    email: str = Field(..., description="Student email ID")
+    secret: str = Field(..., description="Student-provided secret")
+    task: str = Field(..., description="Unique task ID")
+    round: int = Field(..., description="Round index", ge=1)
+    nonce: str = Field(..., description="Nonce to pass back to evaluation URL")
+    brief: str = Field(..., description="App brief description")
+    checks: list[str] = Field(..., description="Evaluation checks")
+    evaluation_url: str = Field(..., description="URL to send repo details")
+    attachments: list[Attachment] = Field(default_factory=list, description="File attachments")
+
+
+class RepoSubmission(BaseModel):
+    """Repository submission sent to evaluation endpoint."""
+
+    email: str = Field(..., description="Student email ID")
+    task: str = Field(..., description="Task ID")
+    round: int = Field(..., description="Round index", ge=1)
+    nonce: str = Field(..., description="Nonce from request")
+    repo_url: str = Field(..., description="GitHub repository URL")
+    commit_sha: str = Field(..., description="Git commit SHA")
+    pages_url: str = Field(..., description="GitHub Pages URL")
+
+
+class TaskRecord(BaseModel):
+    """Task record in database."""
+
+    timestamp: datetime
+    email: str
+    task: str
+    round: int
+    nonce: str
+    brief: str
+    attachments: str  # JSON serialized
+    checks: str  # JSON serialized
+    evaluation_url: str
+    endpoint: str
+    statuscode: int | None = None
+    secret: str
+
+
+class RepoRecord(BaseModel):
+    """Repository record in database."""
+
+    timestamp: datetime
+    email: str
+    task: str
+    round: int
+    nonce: str
+    repo_url: str
+    commit_sha: str
+    pages_url: str
+
+
+class EvaluationResult(BaseModel):
+    """Evaluation result record."""
+
+    timestamp: datetime
+    email: str
+    task: str
+    round: int
+    repo_url: str
+    commit_sha: str
+    pages_url: str
+    check: str
+    score: float
+    reason: str
+    logs: str
+
+
+class Submission(BaseModel):
+    """Student submission from CSV."""
+
+    timestamp: datetime
+    email: str
+    endpoint: str
+    secret: str

shared/utils.py

@@ -0,0 +1,57 @@
+"""Utility functions for LLM Code Deployment system."""
+import base64
+import hashlib
+import re
+from datetime import datetime
+from typing import Any
+
+from uuid_utils import uuid7
+
+
+def generate_nonce() -> str:
+    """Generate a unique nonce using UUID7."""
+    return str(uuid7())
+
+
+def generate_task_id(template_id: str, brief: str, attachments: list[dict[str, Any]]) -> str:
+    """Generate a unique task ID based on template and content."""
+    content = f"{brief}{str(attachments)}"
+    hash_value = hashlib.sha256(content.encode()).hexdigest()[:5]
+    return f"{template_id}-{hash_value}"
+
+
+def decode_data_uri(data_uri: str) -> tuple[str, bytes]:
+    """Decode a data URI into mime type and content."""
+    # Format: data:mime/type;base64,content
+    match = re.match(r"data:([^;]+);base64,(.+)", data_uri)
+    if not match:
+        raise ValueError(f"Invalid data URI format: {data_uri[:50]}...")
+
+    mime_type = match.group(1)
+    encoded_content = match.group(2)
+    content = base64.b64decode(encoded_content)
+
+    return mime_type, content
+
+
+def encode_data_uri(mime_type: str, content: bytes) -> str:
+    """Encode content as a data URI."""
+    encoded = base64.b64encode(content).decode("utf-8")
+    return f"data:{mime_type};base64,{encoded}"
+
+
+def sanitize_repo_name(name: str) -> str:
+    """Sanitize a string for use as a GitHub repository name."""
+    # Replace spaces and special chars with hyphens
+    sanitized = re.sub(r"[^a-zA-Z0-9-_.]", "-", name)
+    # Remove leading/trailing hyphens
+    sanitized = sanitized.strip("-")
+    # Collapse multiple hyphens
+    sanitized = re.sub(r"-+", "-", sanitized)
+    # Ensure lowercase
+    return sanitized.lower()
+
+
+def get_timestamp() -> datetime:
+    """Get current UTC timestamp."""
+    return datetime.utcnow()

student/__init__.py

@@ -0,0 +1 @@
+"""Student-side modules for receiving tasks and deploying code."""

student/api.py

@@ -0,0 +1,276 @@
+"""Student API endpoint for receiving build/update requests."""
+import asyncio
+from datetime import datetime
+from pathlib import Path
+
+from fastapi import FastAPI, HTTPException, BackgroundTasks
+from fastapi.responses import JSONResponse
+
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.models import RepoSubmission, TaskRequest
+from student.code_generator import CodeGenerator
+from student.github_manager import GitHubManager
+from student.notification_client import NotificationClient
+
+logger = setup_logger(__name__)
+
+app = FastAPI(
+    title="LLM Code Deployment - Student API",
+    description="API endpoint for receiving build and update requests",
+    version="1.0.0",
+)
+
+# Initialize services
+code_generator = CodeGenerator()
+github_manager = GitHubManager()
+notification_client = NotificationClient()
+
+# Track ongoing tasks
+active_tasks: dict[str, dict] = {}
+
+
+@app.post("/api/build")
+async def build_app(request: TaskRequest, background_tasks: BackgroundTasks) -> JSONResponse:
+    """Receive build request and process in background.
+
+    Args:
+        request: Task request with app brief and requirements
+        background_tasks: FastAPI background tasks
+
+    Returns:
+        HTTP 200 JSON response
+    """
+    logger.info(f"Received build request for task {request.task}, round {request.round}")
+
+    # Verify secret
+    if request.secret != settings.student_secret:
+        logger.error(f"Invalid secret for task {request.task}")
+        raise HTTPException(status_code=401, detail="Invalid secret")
+
+    # Verify email
+    if request.email != settings.student_email:
+        logger.error(f"Email mismatch: expected {settings.student_email}, got {request.email}")
+        raise HTTPException(status_code=401, detail="Email mismatch")
+
+    # Check if task is already in progress
+    task_key = f"{request.task}-{request.round}"
+    if task_key in active_tasks:
+        logger.warning(f"Task {task_key} already in progress")
+        return JSONResponse(
+            status_code=200,
+            content={
+                "status": "in_progress",
+                "message": f"Task {task_key} is already being processed",
+            },
+        )
+
+    # Mark task as active
+    active_tasks[task_key] = {
+        "status": "accepted",
+        "started_at": datetime.utcnow().isoformat(),
+    }
+
+    # Process in background
+    if request.round == 1:
+        background_tasks.add_task(process_build_request, request)
+    else:
+        background_tasks.add_task(process_update_request, request)
+
+    logger.info(f"Accepted task {task_key} for background processing")
+
+    return JSONResponse(
+        status_code=200,
+        content={
+            "status": "accepted",
+            "message": f"Task {task_key} accepted for processing",
+            "task": request.task,
+            "round": request.round,
+        },
+    )
+
+
+async def process_build_request(request: TaskRequest) -> None:
+    """Process build request in background.
+
+    Args:
+        request: Task request
+    """
+    task_key = f"{request.task}-{request.round}"
+
+    try:
+        logger.info(f"Processing build request for {task_key}")
+
+        # Create output directory
+        output_dir = settings.generated_repos_dir / request.task
+        output_dir.mkdir(parents=True, exist_ok=True)
+
+        # Generate code
+        logger.info(f"Generating code for {task_key}")
+        active_tasks[task_key]["status"] = "generating"
+        code_generator.generate_app(request, output_dir)
+
+        # Create repo and deploy
+        logger.info(f"Deploying to GitHub for {task_key}")
+        active_tasks[task_key]["status"] = "deploying"
+        repo_url, commit_sha, pages_url = github_manager.create_and_deploy(
+            request.task, output_dir
+        )
+
+        # Prepare submission
+        submission = RepoSubmission(
+            email=request.email,
+            task=request.task,
+            round=request.round,
+            nonce=request.nonce,
+            repo_url=repo_url,
+            commit_sha=commit_sha,
+            pages_url=pages_url,
+        )
+
+        # Notify evaluation endpoint
+        logger.info(f"Notifying evaluation endpoint for {task_key}")
+        active_tasks[task_key]["status"] = "notifying"
+        success = await notification_client.notify_with_timeout(
+            request.evaluation_url,
+            submission,
+            timeout_minutes=settings.task_timeout_minutes,
+        )
+
+        if success:
+            logger.info(f"Successfully completed task {task_key}")
+            active_tasks[task_key]["status"] = "completed"
+            active_tasks[task_key]["completed_at"] = datetime.utcnow().isoformat()
+        else:
+            logger.error(f"Failed to notify evaluation endpoint for {task_key}")
+            active_tasks[task_key]["status"] = "notification_failed"
+
+    except Exception as e:
+        logger.error(f"Error processing build request {task_key}: {e}", exc_info=True)
+        active_tasks[task_key]["status"] = "failed"
+        active_tasks[task_key]["error"] = str(e)
+
+
+async def process_update_request(request: TaskRequest) -> None:
+    """Process update request in background.
+
+    Args:
+        request: Task request
+    """
+    task_key = f"{request.task}-{request.round}"
+
+    try:
+        logger.info(f"Processing update request for {task_key}")
+
+        # Use existing output directory
+        output_dir = settings.generated_repos_dir / request.task
+
+        if not output_dir.exists():
+            # If directory doesn't exist, treat as new build
+            logger.warning(f"No existing code for {request.task}, creating new")
+            output_dir.mkdir(parents=True, exist_ok=True)
+            code_generator.generate_app(request, output_dir)
+        else:
+            # Generate updated code
+            logger.info(f"Updating code for {task_key}")
+            active_tasks[task_key]["status"] = "generating"
+            code_generator.generate_app(request, output_dir)
+
+        # Update repo and redeploy
+        logger.info(f"Redeploying to GitHub for {task_key}")
+        active_tasks[task_key]["status"] = "deploying"
+
+        from shared.utils import sanitize_repo_name
+
+        repo_name = sanitize_repo_name(request.task)
+
+        try:
+            repo_url, commit_sha = github_manager.update_and_redeploy(repo_name, output_dir)
+            pages_url = f"https://{settings.github_username}.github.io/{repo_name}/"
+        except Exception as e:
+            logger.warning(f"Update failed, creating new repo: {e}")
+            repo_url, commit_sha, pages_url = github_manager.create_and_deploy(
+                request.task, output_dir
+            )
+
+        # Prepare submission
+        submission = RepoSubmission(
+            email=request.email,
+            task=request.task,
+            round=request.round,
+            nonce=request.nonce,
+            repo_url=repo_url,
+            commit_sha=commit_sha,
+            pages_url=pages_url,
+        )
+
+        # Notify evaluation endpoint
+        logger.info(f"Notifying evaluation endpoint for {task_key}")
+        active_tasks[task_key]["status"] = "notifying"
+        success = await notification_client.notify_with_timeout(
+            request.evaluation_url,
+            submission,
+            timeout_minutes=settings.task_timeout_minutes,
+        )
+
+        if success:
+            logger.info(f"Successfully completed task {task_key}")
+            active_tasks[task_key]["status"] = "completed"
+            active_tasks[task_key]["completed_at"] = datetime.utcnow().isoformat()
+        else:
+            logger.error(f"Failed to notify evaluation endpoint for {task_key}")
+            active_tasks[task_key]["status"] = "notification_failed"
+
+    except Exception as e:
+        logger.error(f"Error processing update request {task_key}: {e}", exc_info=True)
+        active_tasks[task_key]["status"] = "failed"
+        active_tasks[task_key]["error"] = str(e)
+
+
+@app.get("/api/status/{task_id}")
+async def get_task_status(task_id: str) -> JSONResponse:
+    """Get status of a task.
+
+    Args:
+        task_id: Task identifier
+
+    Returns:
+        Task status information
+    """
+    matching_tasks = {k: v for k, v in active_tasks.items() if k.startswith(task_id)}
+
+    if not matching_tasks:
+        raise HTTPException(status_code=404, detail="Task not found")
+
+    return JSONResponse(content=matching_tasks)
+
+
+@app.get("/health")
+async def health_check() -> JSONResponse:
+    """Health check endpoint.
+
+    Returns:
+        Health status
+    """
+    return JSONResponse(
+        content={
+            "status": "healthy",
+            "active_tasks": len(active_tasks),
+            "timestamp": datetime.utcnow().isoformat(),
+        }
+    )
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    # Ensure directories exist
+    settings.ensure_directories()
+
+    logger.info(f"Starting Student API on port {settings.student_api_port}")
+    uvicorn.run(
+        "student.api:app",
+        host="0.0.0.0",
+        port=settings.student_api_port,
+        reload=True,
+    )

student/code_generator.py

@@ -0,0 +1,276 @@
+"""LLM-based code generator for creating web applications."""
+import json
+from pathlib import Path
+from typing import Any
+
+import anthropic
+import openai
+
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.models import Attachment, TaskRequest
+from shared.utils import decode_data_uri
+
+logger = setup_logger(__name__)
+
+
+class CodeGenerator:
+    """Generate code using LLM based on task requirements."""
+
+    def __init__(self) -> None:
+        """Initialize code generator with LLM client."""
+        self.provider = settings.llm_provider
+        self.model = settings.llm_model
+
+        if self.provider == "anthropic":
+            self.client = anthropic.Anthropic(api_key=settings.anthropic_api_key)
+        elif self.provider == "openai":
+            self.client = openai.OpenAI(api_key=settings.openai_api_key)
+        elif self.provider == "aipipe":
+            # Use OpenAI client with AIPipe endpoints
+            self.client = openai.OpenAI(
+                api_key=settings.aipipe_token,
+                base_url=settings.aipipe_base_url
+            )
+        else:
+            raise ValueError(f"Unsupported LLM provider: {self.provider}")
+
+        logger.info(f"Initialized CodeGenerator with {self.provider}/{self.model}")
+
+    def generate_app(self, task: TaskRequest, output_dir: Path) -> dict[str, str]:
+        """Generate application code based on task requirements.
+
+        Args:
+            task: Task request containing brief and requirements
+            output_dir: Directory to save generated files
+
+        Returns:
+            Dictionary mapping filenames to their content
+        """
+        logger.info(f"Generating app for task {task.task}")
+
+        # Prepare context with attachments
+        attachment_info = self._prepare_attachments(task.attachments, output_dir)
+
+        # Build prompt
+        prompt = self._build_generation_prompt(task, attachment_info)
+
+        # Generate code
+        generated_files = self._generate_with_llm(prompt)
+
+        # Save files
+        self._save_files(generated_files, output_dir)
+
+        logger.info(f"Generated {len(generated_files)} files for task {task.task}")
+        return generated_files
+
+    def _prepare_attachments(
+        self, attachments: list[Attachment], output_dir: Path
+    ) -> list[dict[str, str]]:
+        """Decode and save attachments, return metadata.
+
+        Args:
+            attachments: List of attachments with data URIs
+            output_dir: Directory to save attachments
+
+        Returns:
+            List of attachment metadata
+        """
+        attachment_info = []
+
+        for att in attachments:
+            try:
+                mime_type, content = decode_data_uri(att.url)
+                file_path = output_dir / att.name
+                file_path.write_bytes(content)
+
+                attachment_info.append(
+                    {
+                        "name": att.name,
+                        "mime_type": mime_type,
+                        "size": len(content),
+                        "preview": content[:200].decode("utf-8", errors="ignore")
+                        if mime_type.startswith("text/")
+                        else "[binary data]",
+                    }
+                )
+                logger.debug(f"Saved attachment {att.name} ({mime_type}, {len(content)} bytes)")
+            except Exception as e:
+                logger.error(f"Failed to process attachment {att.name}: {e}")
+                attachment_info.append({"name": att.name, "error": str(e)})
+
+        return attachment_info
+
+    def _build_generation_prompt(
+        self, task: TaskRequest, attachment_info: list[dict[str, str]]
+    ) -> str:
+        """Build prompt for LLM code generation.
+
+        Args:
+            task: Task request
+            attachment_info: Attachment metadata
+
+        Returns:
+            Formatted prompt
+        """
+        attachments_section = ""
+        if attachment_info:
+            attachments_section = "\n\n**Attachments:**\n" + "\n".join(
+                f"- {att['name']}: {att.get('mime_type', 'unknown')}"
+                for att in attachment_info
+            )
+
+        checks_section = "\n\n**Requirements (will be tested):**\n" + "\n".join(
+            f"- {check}" for check in task.checks
+        )
+
+        prompt = f"""You are an expert web developer. Create a complete, production-ready single-page web application based on the following requirements.
+
+**Task:** {task.task}
+
+**Brief:** {task.brief}{attachments_section}{checks_section}
+
+**Instructions:**
+1. Create a minimal, functional web application that meets ALL requirements
+2. Use only vanilla HTML, CSS, and JavaScript (no build tools required)
+3. Include all necessary CDN links for external libraries (Bootstrap, marked, highlight.js, etc.)
+4. Ensure the app is self-contained in a single index.html file or minimal files
+5. Follow best practices for code quality, accessibility, and user experience
+6. Include helpful comments explaining key functionality
+7. Make the UI clean and professional using Bootstrap 5 or similar
+
+**Output Format:**
+Provide the complete code for each file in JSON format:
+```json
+{{
+  "index.html": "<!DOCTYPE html>...",
+  "style.css": "/* optional styles */",
+  "script.js": "// optional separate JS",
+  "README.md": "# Project Title\\n\\n..."
+}}
+```
+
+Generate ONLY the JSON output, no other text. Ensure all code is complete and functional.
+"""
+        return prompt
+
+    def _generate_with_llm(self, prompt: str) -> dict[str, str]:
+        """Call LLM API to generate code.
+
+        Args:
+            prompt: Generation prompt
+
+        Returns:
+            Dictionary of filename -> content
+        """
+        logger.info(f"Calling {self.provider} API for code generation")
+
+        try:
+            if self.provider == "anthropic":
+                response = self.client.messages.create(
+                    model=self.model,
+                    max_tokens=4096,
+                    temperature=0.3,
+                    messages=[{"role": "user", "content": prompt}],
+                )
+                content = response.content[0].text
+
+            elif self.provider in ["openai", "aipipe"]:
+                # Both OpenAI and AIPipe use the same API format
+                response = self.client.chat.completions.create(
+                    model=self.model,
+                    messages=[{"role": "user", "content": prompt}],
+                    temperature=0.3,
+                    max_tokens=4096,
+                )
+                content = response.choices[0].message.content
+
+            else:
+                raise ValueError(f"Unsupported provider: {self.provider}")
+
+            # Extract JSON from response
+            files = self._extract_json(content)
+            return files
+
+        except Exception as e:
+            logger.error(f"LLM generation failed: {e}")
+            # Fallback to minimal template
+            return self._get_fallback_template()
+
+    def _extract_json(self, content: str) -> dict[str, str]:
+        """Extract JSON from LLM response.
+
+        Args:
+            content: LLM response text
+
+        Returns:
+            Parsed JSON dictionary
+        """
+        # Try to find JSON in markdown code block
+        import re
+
+        json_match = re.search(r"```json\s*(\{.*?\})\s*```", content, re.DOTALL)
+        if json_match:
+            return json.loads(json_match.group(1))
+
+        # Try to parse the whole content as JSON
+        try:
+            return json.loads(content)
+        except json.JSONDecodeError:
+            # Try to find any JSON object
+            json_match = re.search(r"\{.*\}", content, re.DOTALL)
+            if json_match:
+                return json.loads(json_match.group(0))
+
+        raise ValueError("Could not extract JSON from LLM response")
+
+    def _get_fallback_template(self) -> dict[str, str]:
+        """Get fallback template when LLM generation fails.
+
+        Returns:
+            Basic HTML template
+        """
+        return {
+            "index.html": """<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Generated App</title>
+    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet">
+</head>
+<body>
+    <div class="container mt-5">
+        <h1>Application</h1>
+        <p>This is a minimal fallback template.</p>
+    </div>
+    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script>
+</body>
+</html>""",
+            "README.md": """# Generated Application
+
+This is an automatically generated web application.
+
+## Setup
+
+Simply open `index.html` in a web browser.
+
+## License
+
+MIT License
+""",
+        }
+
+    def _save_files(self, files: dict[str, str], output_dir: Path) -> None:
+        """Save generated files to output directory.
+
+        Args:
+            files: Dictionary of filename -> content
+            output_dir: Directory to save files
+        """
+        output_dir.mkdir(parents=True, exist_ok=True)
+
+        for filename, content in files.items():
+            file_path = output_dir / filename
+            file_path.write_text(content, encoding="utf-8")
+            logger.debug(f"Saved {filename} ({len(content)} bytes)")

student/github_manager.py

@@ -0,0 +1,319 @@
+"""GitHub repository management and Pages deployment."""
+import shutil
+import time
+from pathlib import Path
+
+import git
+from github import Github, GithubException
+
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.utils import sanitize_repo_name
+
+logger = setup_logger(__name__)
+
+
+class GitHubManager:
+    """Manage GitHub repository creation, pushing, and Pages deployment."""
+
+    def __init__(self) -> None:
+        """Initialize GitHub manager."""
+        self.github = Github(settings.github_token)
+        self.user = self.github.get_user()
+        logger.info(f"Initialized GitHubManager for user {self.user.login}")
+
+    def create_and_deploy(
+        self, task_id: str, code_dir: Path
+    ) -> tuple[str, str, str]:
+        """Create repo, push code, enable Pages, and return URLs.
+
+        Args:
+            task_id: Unique task identifier
+            code_dir: Directory containing generated code
+
+        Returns:
+            Tuple of (repo_url, commit_sha, pages_url)
+        """
+        repo_name = sanitize_repo_name(task_id)
+        logger.info(f"Creating and deploying repository: {repo_name}")
+
+        # Create GitHub repository
+        repo_url = self._create_repo(repo_name)
+
+        # Initialize git and push
+        commit_sha = self._push_code(repo_name, code_dir)
+
+        # Enable GitHub Pages
+        pages_url = self._enable_pages(repo_name)
+
+        # Wait for Pages to be ready
+        self._wait_for_pages(pages_url)
+
+        logger.info(
+            f"Successfully deployed {repo_name}: "
+            f"repo={repo_url}, commit={commit_sha}, pages={pages_url}"
+        )
+
+        return repo_url, commit_sha, pages_url
+
+    def update_and_redeploy(
+        self, repo_name: str, code_dir: Path
+    ) -> tuple[str, str]:
+        """Update existing repo with new code and return new commit.
+
+        Args:
+            repo_name: Repository name
+            code_dir: Directory containing updated code
+
+        Returns:
+            Tuple of (repo_url, commit_sha)
+        """
+        logger.info(f"Updating repository: {repo_name}")
+
+        try:
+            repo = self.user.get_repo(repo_name)
+            repo_url = repo.html_url
+
+            # Clone, update, and push
+            commit_sha = self._update_code(repo_name, code_dir)
+
+            logger.info(f"Successfully updated {repo_name}: commit={commit_sha}")
+            return repo_url, commit_sha
+
+        except GithubException as e:
+            logger.error(f"Failed to update repo {repo_name}: {e}")
+            raise
+
+    def _create_repo(self, repo_name: str) -> str:
+        """Create a new GitHub repository.
+
+        Args:
+            repo_name: Repository name
+
+        Returns:
+            Repository URL
+        """
+        try:
+            # Check if repo already exists
+            try:
+                existing_repo = self.user.get_repo(repo_name)
+                logger.warning(f"Repository {repo_name} already exists, using it")
+                return existing_repo.html_url
+            except GithubException:
+                pass  # Repo doesn't exist, create it
+
+            # Create new repo
+            repo = self.user.create_repo(
+                name=repo_name,
+                description=f"Auto-generated repository for task {repo_name}",
+                private=False,  # Must be public for Pages
+                auto_init=False,
+            )
+
+            logger.info(f"Created repository: {repo.html_url}")
+            return repo.html_url
+
+        except GithubException as e:
+            logger.error(f"Failed to create repository {repo_name}: {e}")
+            raise
+
+    def _push_code(self, repo_name: str, code_dir: Path) -> str:
+        """Initialize git repo and push code.
+
+        Args:
+            repo_name: Repository name
+            code_dir: Directory containing code
+
+        Returns:
+            Commit SHA
+        """
+        try:
+            # Ensure LICENSE file exists
+            self._ensure_license(code_dir)
+
+            # Initialize git repo
+            repo = git.Repo.init(code_dir)
+
+            # Configure git
+            with repo.config_writer() as config:
+                if not config.has_option("user", "name"):
+                    config.set_value("user", "name", settings.github_username)
+                if not config.has_option("user", "email"):
+                    config.set_value("user", "email", settings.student_email)
+
+            # Add all files
+            repo.index.add("*")
+
+            # Create .gitignore if needed
+            gitignore_path = code_dir / ".gitignore"
+            if not gitignore_path.exists():
+                gitignore_path.write_text("*.pyc\n__pycache__/\n.DS_Store\n")
+                repo.index.add([".gitignore"])
+
+            # Commit
+            commit = repo.index.commit("Initial commit: Generated application")
+            commit_sha = commit.hexsha
+
+            # Add remote
+            remote_url = f"https://{settings.github_token}@github.com/{settings.github_username}/{repo_name}.git"
+            try:
+                origin = repo.remote("origin")
+                origin.set_url(remote_url)
+            except ValueError:
+                origin = repo.create_remote("origin", remote_url)
+
+            # Push
+            origin.push(refspec="main:main", force=True)
+
+            logger.info(f"Pushed code to {repo_name}: commit {commit_sha}")
+            return commit_sha
+
+        except Exception as e:
+            logger.error(f"Failed to push code to {repo_name}: {e}")
+            raise
+
+    def _update_code(self, repo_name: str, code_dir: Path) -> str:
+        """Update existing repository with new code.
+
+        Args:
+            repo_name: Repository name
+            code_dir: Directory containing updated code
+
+        Returns:
+            New commit SHA
+        """
+        try:
+            # Clone the repo
+            temp_dir = settings.generated_repos_dir / f"{repo_name}_temp"
+            if temp_dir.exists():
+                shutil.rmtree(temp_dir)
+
+            clone_url = f"https://{settings.github_token}@github.com/{settings.github_username}/{repo_name}.git"
+            repo = git.Repo.clone_from(clone_url, temp_dir)
+
+            # Copy new files (except .git)
+            for item in code_dir.iterdir():
+                if item.name == ".git":
+                    continue
+
+                dest = temp_dir / item.name
+                if item.is_file():
+                    shutil.copy2(item, dest)
+                elif item.is_dir():
+                    if dest.exists():
+                        shutil.rmtree(dest)
+                    shutil.copytree(item, dest)
+
+            # Ensure LICENSE
+            self._ensure_license(temp_dir)
+
+            # Commit and push
+            repo.index.add("*")
+            commit = repo.index.commit("Update: Revised application")
+            commit_sha = commit.hexsha
+
+            origin = repo.remote("origin")
+            origin.push()
+
+            # Cleanup
+            shutil.rmtree(temp_dir)
+
+            logger.info(f"Updated code in {repo_name}: commit {commit_sha}")
+            return commit_sha
+
+        except Exception as e:
+            logger.error(f"Failed to update code in {repo_name}: {e}")
+            raise
+
+    def _ensure_license(self, code_dir: Path) -> None:
+        """Ensure MIT LICENSE file exists in code directory.
+
+        Args:
+            code_dir: Directory to add license to
+        """
+        license_path = code_dir / "LICENSE"
+        if not license_path.exists():
+            year = time.strftime("%Y")
+            license_text = f"""MIT License
+
+Copyright (c) {year} {settings.github_username}
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+"""
+            license_path.write_text(license_text)
+            logger.debug(f"Created MIT LICENSE in {code_dir}")
+
+    def _enable_pages(self, repo_name: str) -> str:
+        """Enable GitHub Pages for repository.
+
+        Args:
+            repo_name: Repository name
+
+        Returns:
+            Pages URL
+        """
+        try:
+            repo = self.user.get_repo(repo_name)
+
+            # Enable Pages on main branch
+            try:
+                repo.create_pages_site(source={"branch": "main", "path": "/"})
+                logger.info(f"Enabled GitHub Pages for {repo_name}")
+            except GithubException as e:
+                if "already enabled" in str(e).lower():
+                    logger.info(f"GitHub Pages already enabled for {repo_name}")
+                else:
+                    raise
+
+            # Construct Pages URL
+            pages_url = f"https://{settings.github_username}.github.io/{repo_name}/"
+            return pages_url
+
+        except GithubException as e:
+            logger.error(f"Failed to enable Pages for {repo_name}: {e}")
+            # Return expected URL even if enabling fails
+            return f"https://{settings.github_username}.github.io/{repo_name}/"
+
+    def _wait_for_pages(self, pages_url: str, max_wait: int = 60) -> bool:
+        """Wait for GitHub Pages to be ready.
+
+        Args:
+            pages_url: Pages URL to check
+            max_wait: Maximum wait time in seconds
+
+        Returns:
+            True if ready, False if timeout
+        """
+        import requests
+
+        logger.info(f"Waiting for Pages to be ready: {pages_url}")
+
+        for i in range(max_wait):
+            try:
+                response = requests.get(pages_url, timeout=5)
+                if response.status_code == 200:
+                    logger.info(f"Pages ready after {i+1} seconds")
+                    return True
+            except requests.RequestException:
+                pass
+
+            time.sleep(1)
+
+        logger.warning(f"Pages not ready after {max_wait} seconds")
+        return False

student/notification_client.py

@@ -0,0 +1,86 @@
+"""Client for sending notifications to evaluation URL."""
+import asyncio
+from typing import Any
+
+import httpx
+from tenacity import retry, stop_after_attempt, wait_exponential
+
+from shared.config import settings
+from shared.logger import setup_logger
+from shared.models import RepoSubmission
+
+logger = setup_logger(__name__)
+
+
+class NotificationClient:
+    """Client for notifying evaluation endpoint about repo submissions."""
+
+    def __init__(self) -> None:
+        """Initialize notification client."""
+        self.retry_delays = settings.get_retry_delays()
+        self.max_retries = settings.max_retry_attempts
+
+    @retry(
+        stop=stop_after_attempt(4),
+        wait=wait_exponential(multiplier=1, min=1, max=8),
+        reraise=True,
+    )
+    async def notify(self, evaluation_url: str, submission: RepoSubmission) -> bool:
+        """Send repo submission to evaluation URL with retry logic.
+
+        Args:
+            evaluation_url: URL to send notification to
+            submission: Repository submission data
+
+        Returns:
+            True if successful, False otherwise
+        """
+        logger.info(f"Sending notification to {evaluation_url}")
+        logger.debug(f"Submission data: {submission.model_dump()}")
+
+        try:
+            async with httpx.AsyncClient(timeout=30.0) as client:
+                response = await client.post(
+                    evaluation_url,
+                    json=submission.model_dump(),
+                    headers={"Content-Type": "application/json"},
+                )
+
+                if response.status_code == 200:
+                    logger.info(f"Successfully notified {evaluation_url}")
+                    return True
+                else:
+                    logger.error(
+                        f"Failed to notify {evaluation_url}: "
+                        f"Status {response.status_code}, Response: {response.text}"
+                    )
+                    raise Exception(f"HTTP {response.status_code}: {response.text}")
+
+        except httpx.TimeoutException:
+            logger.error(f"Timeout while notifying {evaluation_url}")
+            raise
+        except Exception as e:
+            logger.error(f"Error notifying {evaluation_url}: {e}")
+            raise
+
+    async def notify_with_timeout(
+        self, evaluation_url: str, submission: RepoSubmission, timeout_minutes: int = 10
+    ) -> bool:
+        """Send notification with overall timeout.
+
+        Args:
+            evaluation_url: URL to send notification to
+            submission: Repository submission data
+            timeout_minutes: Overall timeout in minutes
+
+        Returns:
+            True if successful, False otherwise
+        """
+        try:
+            return await asyncio.wait_for(
+                self.notify(evaluation_url, submission),
+                timeout=timeout_minutes * 60,
+            )
+        except asyncio.TimeoutError:
+            logger.error(f"Overall timeout of {timeout_minutes} minutes exceeded")
+            return False

submissions.csv.example

@@ -0,0 +1,3 @@
+timestamp,email,endpoint,secret
+2025-01-15T10:00:00,student1@example.com,http://localhost:8000/api/build,secret1
+2025-01-15T10:05:00,student2@example.com,http://localhost:8000/api/build,secret2

templates/github-user-created.yaml

@@ -0,0 +1,25 @@
+id: github-user-created
+brief: Publish a Bootstrap page with form id="github-user-{{ seed[:8] }}" that fetches a GitHub username, optionally uses ?token=, and displays the account creation date in YYYY-MM-DD UTC inside #github-created-at.
+checks:
+  - "Form with id 'github-user-{{ seed[:8] }}' exists"
+  - "Element #github-created-at displays date in YYYY-MM-DD format"
+  - "Code fetches from GitHub API (https://api.github.com/users/)"
+  - "Bootstrap CSS is loaded"
+round2:
+  - brief: Show an aria-live alert #github-status that reports when a lookup starts, succeeds, or fails.
+    checks:
+      - "Element #github-status has aria-live='polite' attribute"
+      - "Code updates #github-status with status messages"
+      - "Status changes are properly announced"
+  
+  - brief: Display the account age in whole years inside #github-account-age alongside the creation date.
+    checks:
+      - "Element #github-account-age displays number of years"
+      - "Text includes 'years' or 'year'"
+      - "Age calculation is accurate"
+  
+  - brief: Cache the last successful lookup in localStorage under "github-user-{{ seed[:8] }}" and repopulate the form on load.
+    checks:
+      - "Code uses localStorage.setItem with 'github-user-{{ seed[:8] }}'"
+      - "Code uses localStorage.getItem with 'github-user-{{ seed[:8] }}'"
+      - "Form repopulates on page reload with cached data"

templates/markdown-to-html.yaml

@@ -0,0 +1,28 @@
+id: markdown-to-html
+brief: Publish a static page that converts input.md from attachments to HTML with marked, renders it inside #markdown-output, and loads highlight.js for code blocks.
+attachments:
+  - name: input.md
+    url: data:text/markdown;base64,placeholder
+checks:
+  - "Script tag loading marked library exists"
+  - "Script tag loading highlight.js exists"
+  - "Element #markdown-output contains rendered HTML with headings"
+  - "Code blocks are properly highlighted"
+round2:
+  - brief: Add tabs #markdown-tabs that switch between rendered HTML in #markdown-output and the original Markdown in #markdown-source while keeping content in sync.
+    checks:
+      - "Element #markdown-tabs has at least 2 buttons"
+      - "Element #markdown-source displays original markdown"
+      - "Tab switching works correctly"
+  
+  - brief: Support loading Markdown from a ?url= parameter when present and fall back to the attachment otherwise, showing the active source in #markdown-source-label.
+    checks:
+      - "Element #markdown-source-label exists and shows source"
+      - "Code includes fetch() for loading external URLs"
+      - "Fallback to attachment works when no URL parameter"
+  
+  - brief: Display a live word count badge #markdown-word-count that updates after every render and formats numbers with Intl.NumberFormat.
+    checks:
+      - "Element #markdown-word-count displays formatted number with comma"
+      - "Code uses Intl.NumberFormat for formatting"
+      - "Word count updates on content change"

templates/sum-of-sales.yaml

@@ -0,0 +1,31 @@
+id: sum-of-sales
+brief: Publish a single-page site that fetches data.csv from attachments, sums its sales column, sets the title to "Sales Summary {{ seed }}", displays the total inside #total-sales, and loads Bootstrap 5 from jsdelivr.
+attachments:
+  - name: data.csv
+    url: data:text/csv;base64,placeholder
+checks:
+  - "Page title equals 'Sales Summary {{ seed }}'"
+  - "Bootstrap 5 CSS loaded from jsdelivr"
+  - "Element #total-sales exists and displays numeric value"
+  - "Total sales calculation is accurate"
+round2:
+  - brief: Add a Bootstrap table #product-sales that lists each product with its total sales and keeps #total-sales accurate after render.
+    checks:
+      - "Table #product-sales exists with proper Bootstrap classes"
+      - "Table has at least one row in tbody"
+      - "Sum of product sales matches #total-sales value"
+  
+  - brief: Introduce a currency select #currency-picker that converts the computed total using rates.json from attachments and mirrors the active currency inside #total-currency.
+    attachments:
+      - name: rates.json
+        url: data:application/json;base64,placeholder
+    checks:
+      - "Select #currency-picker exists with USD option"
+      - "Element #total-currency displays current currency"
+      - "Currency conversion works correctly"
+  
+  - brief: Allow filtering by region via #region-filter, update #total-sales with the filtered sum, and set data-region on that element to the active choice.
+    checks:
+      - "Select #region-filter exists"
+      - "Element #total-sales has data-region attribute"
+      - "Filtering updates the total correctly"