feat: complete tiffin packing OpenEnv environment with 3 tasks, VLM, grader, and inference

.dockerignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.git/
+.gitignore
+.env
+outputs/
+*.md
+!README.md

.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.venv/
+.env
+outputs/logs/
+outputs/evals/
+test_*.py
+*.egg
+.eggs/

Dockerfile

@@ -0,0 +1,39 @@
+# Dockerfile for Tiffin Packer — HF Spaces Compatible
+# Read the doc: https://huggingface.co/docs/hub/spaces-sdks-docker
+
+FROM python:3.10-slim
+
+# Create non-root user (HF Spaces requirement)
+RUN useradd -m -u 1000 user
+ENV PATH="/home/user/.local/bin:$PATH"
+
+# System dependencies for PyBullet (headless OpenGL)
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    libgl1-mesa-glx \
+    libglib2.0-0 \
+    libsm6 \
+    libxrender1 \
+    libxext6 \
+    && rm -rf /var/lib/apt/lists/*
+
+# Switch to non-root user
+USER user
+WORKDIR /app
+
+# Install Python dependencies
+COPY --chown=user requirements.txt .
+RUN pip install --no-cache-dir --upgrade pip && \
+    pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY --chown=user . /app
+
+# Expose port (HF Spaces default for Docker)
+EXPOSE 7860
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \
+    CMD python -c "import requests; r=requests.get('http://localhost:7860/health'); r.raise_for_status()" || exit 1
+
+# Run the server
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -0,0 +1,148 @@
+# Smart Tiffin Packing Environment 🍱🤖
+
+> **Semantic-aware constrained packing under real-world constraints**
+>
+> An OpenEnv-compliant RL environment where an LLM agent controls a robotic arm
+> to pack an Indian tiffin meal. The agent uses VLM-derived food classification
+> to reason about container compatibility, volume constraints, temperature zones,
+> and fragility — then physically executes packing decisions.
+
+## 🎯 What is this?
+
+This environment simulates the real-world task of **packing an Indian meal into tiffin containers**. An AI agent must:
+
+1. **Identify** food items using a Vision-Language Model (VLM)
+2. **Reason** about which container each item should go into
+3. **Execute** packing commands via a robotic arm
+4. **Satisfy** multiple constraints simultaneously
+
+### Why Tiffin Packing?
+
+Every day, millions of people in India pack tiffin boxes for lunch. It's a genuine spatial-reasoning task with real constraints:
+- Liquids (sambar, dal) must go in sealed containers
+- Fragile items (papad, chapati) shouldn't be crushed
+- Hot and cold foods should be separated
+- Volume limits mean you can't just stuff everything in one box
+
+## 🏗️ Architecture
+
+```
+LLM Agent (via OpenAI API)
+    │
+    ├── observe → See scene description
+    ├── identify → VLM classifies food item
+    ├── pick → Robotic arm picks up food
+    ├── place → Place item in container
+    └── pour → Pour liquid into container
+    │
+    ▼
+OpenEnv Server (FastAPI)
+    │
+    ├── Simulation Engine (logic + PyBullet physics)
+    ├── VLM Classifier (cached food_db.json)
+    ├── Task Manager (easy/medium/hard)
+    └── Deterministic Grader (0.0-1.0)
+```
+
+## 🎮 Tasks
+
+| Task | Items | Containers | Constraints | Difficulty |
+|------|-------|-----------|-------------|------------|
+| 🟢 Easy | rice, sambar (2) | sealed, flat (2) | Type matching | Straightforward |
+| 🟡 Medium | rice, sambar, chapati, pickle (4) | sealed, flat, deep (3) | Types + overflow + temperature | Requires reasoning |
+| 🔴 Hard | rice, sambar, curd, chapati, papad, curry (6) | sealed, flat, deep, small_sealed (4) | All constraints active | Genuinely challenging |
+
+## 📊 Scoring (0.0 – 1.0)
+
+| Component | Weight | Description |
+|-----------|--------|-------------|
+| Validity | 40% | Food placed in type-compatible container? |
+| Efficiency | 30% | Space utilization vs capacity used |
+| Constraints | 20% | Temperature, fragility, flavor isolation |
+| Neatness | 10% | All items packed? Nothing dropped? |
+
+## 🚀 Quick Start
+
+### Run locally
+```bash
+pip install -r requirements.txt
+uvicorn server.app:app --host 0.0.0.0 --port 7860
+```
+
+### Run inference
+```bash
+export API_BASE_URL=https://api.openai.com/v1
+export MODEL_NAME=gpt-4o
+export HF_TOKEN=your-api-key
+export ENV_URL=http://localhost:7860
+python inference.py
+```
+
+### Docker
+```bash
+docker build -t tiffin-packer .
+docker run -p 7860:7860 tiffin-packer
+```
+
+## 🔧 Action Space
+
+```json
+{
+  "command": "identify | pick | place | pour | observe",
+  "target_id": 1
+}
+```
+
+## 👁️ Observation Space
+
+```json
+{
+  "scene_description": "Natural language scene state",
+  "food_items": [{"id": 1, "name": "rice", "status": "on_table", ...}],
+  "containers": [{"id": 1, "type": "sealed_round", "capacity_ml": 300, ...}],
+  "held_item": null,
+  "vlm_result": {"type": "solid", "fragility": 0.1, ...},
+  "available_commands": ["observe", "identify", "pick"],
+  "step_feedback": "Successfully picked up rice"
+}
+```
+
+## 📁 Project Structure
+
+```
+tiffen-packer/
+├── openenv.yaml          # OpenEnv manifest
+├── inference.py           # LLM inference script (OpenAI Client)
+├── Dockerfile             # HF Spaces deployment
+├── tiffin_packer/         # Core package
+│   ├── models.py          # Pydantic Action/Observation/State
+│   ├── simulation/
+│   │   ├── engine.py      # Logic simulation engine
+│   │   └── pybullet_renderer.py  # Physics visualization
+│   ├── vlm/
+│   │   ├── classifier.py  # VLM food classifier
+│   │   └── food_db.json   # 15 Indian food items
+│   ├── tasks.py           # Easy/Medium/Hard task configs
+│   └── grader.py          # Deterministic scoring
+└── server/
+    ├── tiffin_environment.py  # OpenEnv Environment
+    └── app.py                 # FastAPI server
+```
+
+## 🏆 OpenEnv Compliance
+
+- ✅ Typed Pydantic models (Action, Observation, State)
+- ✅ `step()` / `reset()` / `state()` API
+- ✅ `openenv.yaml` manifest
+- ✅ 3 tasks with deterministic graders (0.0–1.0)
+- ✅ Dense reward function with partial progress signals
+- ✅ Baseline inference script using OpenAI Client
+- ✅ Docker deployment for HF Spaces
+
+## 👥 Team
+
+**CtrlAltWin** — Meta PyTorch OpenEnv Hackathon 2026
+
+## 📄 License
+
+MIT

inference.py

@@ -0,0 +1,292 @@
+#!/usr/bin/env python3
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Tiffin Packer — OpenEnv Inference Script.
+
+Runs an LLM agent against the tiffin packing environment using the
+OpenAI Client API with environment variables:
+    API_BASE_URL  — The API endpoint for the LLM
+    MODEL_NAME    — The model identifier for inference
+    HF_TOKEN      — Hugging Face / API key
+
+Usage:
+    API_BASE_URL=https://api.openai.com/v1 \
+    MODEL_NAME=gpt-4o \
+    HF_TOKEN=your-key \
+    python inference.py
+"""
+
+import json
+import os
+import sys
+import time
+
+import requests
+from openai import OpenAI
+
+# ---------------------------------------------------------------------------
+# Required environment variables
+# ---------------------------------------------------------------------------
+API_BASE_URL = os.environ.get("API_BASE_URL", "https://api.openai.com/v1")
+MODEL_NAME = os.environ.get("MODEL_NAME", "gpt-4o")
+HF_TOKEN = os.environ.get("HF_TOKEN", "")
+ENV_URL = os.environ.get("ENV_URL", "http://localhost:7860")
+
+if not HF_TOKEN:
+    print("WARNING: HF_TOKEN not set. LLM calls will fail.")
+
+client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
+
+# ---------------------------------------------------------------------------
+# System prompt
+# ---------------------------------------------------------------------------
+SYSTEM_PROMPT = """You are a tiffin packing assistant that controls a robotic arm.
+Your goal: pack Indian meal items into the correct tiffin containers.
+
+COMMANDS — respond with ONLY a JSON object, no other text:
+  {"command": "observe"}                    — See the full scene
+  {"command": "identify", "target_id": N}   — Classify food item N using VLM
+  {"command": "pick", "target_id": N}       — Pick up food item N
+  {"command": "place", "target_id": N}      — Place held item into container N
+  {"command": "pour", "target_id": N}       — Pour held liquid into container N
+
+PACKING RULES:
+1. ALWAYS identify items before packing (you cannot see food properties otherwise)
+2. Liquids (sambar, dal, rasam, curry) → sealed containers only
+3. Solids (rice, chapati, idli) → any container type
+4. Semi-solids (curd, pickle, chutney) → sealed containers preferred
+5. FRAGILE items (papad=0.9, chapati=0.7) → don't crush under heavy items
+6. HOT and COLD food must NOT share a container
+7. Don't overflow containers — check volume math!
+8. Strong-flavor items (pickle, chutney) should be isolated
+
+STRATEGY:
+1. First: observe the scene
+2. Then: identify ALL food items (one by one)
+3. Then: plan which food goes where based on constraints
+4. Finally: pick and place/pour each item
+
+Respond with ONLY valid JSON. No explanation, no markdown, no extra text."""
+
+
+def parse_action(text: str) -> dict:
+    """Parse LLM output into an action dict."""
+    text = text.strip()
+
+    # Try to extract JSON from the text
+    if text.startswith("```"):
+        # Handle markdown code blocks
+        lines = text.split("\n")
+        json_lines = [l for l in lines if not l.startswith("```")]
+        text = "\n".join(json_lines).strip()
+
+    # Try direct JSON parse
+    try:
+        action = json.loads(text)
+        if "command" in action:
+            return action
+    except json.JSONDecodeError:
+        pass
+
+    # Try to find JSON in the text
+    for i in range(len(text)):
+        if text[i] == "{":
+            for j in range(len(text) - 1, i, -1):
+                if text[j] == "}":
+                    try:
+                        action = json.loads(text[i : j + 1])
+                        if "command" in action:
+                            return action
+                    except json.JSONDecodeError:
+                        continue
+
+    # Fallback
+    print(f"  [WARN] Could not parse action: {text[:100]}")
+    return {"command": "observe"}
+
+
+def run_episode(task_id: str) -> dict:
+    """Run one episode of the tiffin packing task."""
+    print(f"\n{'='*60}")
+    print(f"  TASK: {task_id.upper()}")
+    print(f"{'='*60}")
+
+    # Reset the environment
+    try:
+        resp = requests.post(
+            f"{ENV_URL}/reset",
+            json={"task_id": task_id, "seed": 42},
+            timeout=30,
+        )
+        resp.raise_for_status()
+        result = resp.json()
+        obs = result.get("observation", result)
+    except Exception as e:
+        print(f"  ERROR: Failed to reset environment: {e}")
+        return {"task_id": task_id, "reward": 0.0, "score": 0.0, "error": str(e)}
+
+    # Initialize conversation
+    init_scene = obs.get("scene_description", "")
+    init_feedback = obs.get("step_feedback", "")
+    messages = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {
+            "role": "user",
+            "content": (
+                f"Task: {task_id}\n\n"
+                f"{init_feedback}\n\n"
+                f"Scene:\n{init_scene}\n\n"
+                f"Available commands: {obs.get('available_commands', [])}\n\n"
+                f"What is your first action? Respond with JSON only."
+            ),
+        },
+    ]
+
+    total_reward = 0.0
+    step = 0
+    max_steps = 35  # safety limit
+
+    while not obs.get("done", False) and step < max_steps:
+        step += 1
+
+        # Get LLM decision
+        try:
+            response = client.chat.completions.create(
+                model=MODEL_NAME,
+                messages=messages,
+                temperature=0.0,
+                max_tokens=200,
+            )
+            action_text = response.choices[0].message.content.strip()
+        except Exception as e:
+            print(f"  [Step {step}] LLM error: {e}")
+            action_text = '{"command": "observe"}'
+
+        action = parse_action(action_text)
+        print(f"  [Step {step}] Action: {json.dumps(action)}")
+
+        # Execute step
+        try:
+            resp = requests.post(
+                f"{ENV_URL}/step",
+                json={"action": action},
+                timeout=30,
+            )
+            resp.raise_for_status()
+            result = resp.json()
+            obs = result.get("observation", result)
+            reward = result.get("reward", obs.get("reward", 0.0))
+            total_reward += reward or 0
+        except Exception as e:
+            print(f"  [Step {step}] Step error: {e}")
+            break
+
+        # Print feedback
+        feedback = obs.get("step_feedback", "")[:200]
+        print(f"           Reward: {reward:+.2f} | Feedback: {feedback}")
+
+        # Update conversation with assistant response and new observation
+        messages.append({"role": "assistant", "content": action_text})
+
+        # Build concise next observation for LLM
+        held = obs.get("held_item")
+        held_str = (
+            f"Holding: {held.get('name', 'unknown')}" if held else "Arm: idle"
+        )
+        items_status = [
+            f"[{i['id']}] {i.get('name', '?')} ({i['status']})"
+            for i in obs.get("food_items", [])
+        ]
+        containers_status = [
+            f"[{c['id']}] {c['name']} {c.get('fill_percentage',0):.0f}% full"
+            for c in obs.get("containers", [])
+        ]
+
+        messages.append(
+            {
+                "role": "user",
+                "content": (
+                    f"Step {step} result (reward={reward:+.2f}):\n"
+                    f"Feedback: {obs.get('step_feedback', '')}\n\n"
+                    f"{held_str}\n"
+                    f"Items: {', '.join(items_status)}\n"
+                    f"Containers: {', '.join(containers_status)}\n"
+                    f"Available: {obs.get('available_commands', [])}\n\n"
+                    f"{'VLM Result: ' + json.dumps(obs.get('vlm_result')) if obs.get('vlm_result') else ''}\n\n"
+                    f"Next action? JSON only."
+                ),
+            },
+        )
+
+    # Extract final score
+    final_score = obs.get("metadata", {}).get("final_score", 0.0)
+    grade_breakdown = obs.get("metadata", {}).get("grade_breakdown", {})
+
+    print(f"\n  {'─'*40}")
+    print(f"  Steps taken:  {step}")
+    print(f"  Total reward: {total_reward:+.2f}")
+    print(f"  Final score:  {final_score:.4f}")
+    if grade_breakdown:
+        print(f"  Breakdown:")
+        print(f"    Validity:    {grade_breakdown.get('validity', 0):.4f} (x0.4)")
+        print(f"    Efficiency:  {grade_breakdown.get('efficiency', 0):.4f} (x0.3)")
+        print(f"    Constraints: {grade_breakdown.get('constraints', 0):.4f} (x0.2)")
+        print(f"    Neatness:    {grade_breakdown.get('neatness', 0):.4f} (x0.1)")
+
+    return {
+        "task_id": task_id,
+        "steps": step,
+        "total_reward": round(total_reward, 4),
+        "score": final_score,
+        "grade_breakdown": grade_breakdown,
+    }
+
+
+def main():
+    """Run all 3 tasks and report results."""
+    print("=" * 60)
+    print("  TIFFIN PACKER — INFERENCE SCRIPT")
+    print(f"  Model: {MODEL_NAME}")
+    print(f"  API:   {API_BASE_URL}")
+    print(f"  Env:   {ENV_URL}")
+    print("=" * 60)
+
+    start_time = time.time()
+    results = {}
+
+    for task_id in ["easy", "medium", "hard"]:
+        result = run_episode(task_id)
+        results[task_id] = result
+
+    elapsed = time.time() - start_time
+
+    # Summary
+    print("\n" + "=" * 60)
+    print("  FINAL RESULTS")
+    print("=" * 60)
+    for task_id, r in results.items():
+        print(f"  {task_id:8s}: score={r['score']:.4f}  reward={r['total_reward']:+.2f}  steps={r.get('steps', '?')}")
+
+    avg_score = sum(r["score"] for r in results.values()) / max(len(results), 1)
+    print(f"\n  Average score: {avg_score:.4f}")
+    print(f"  Total time:    {elapsed:.1f}s")
+
+    # Save results
+    os.makedirs("outputs/evals", exist_ok=True)
+    with open("outputs/evals/results.json", "w") as f:
+        json.dump(
+            {
+                "model": MODEL_NAME,
+                "api_base_url": API_BASE_URL,
+                "results": results,
+                "average_score": avg_score,
+                "elapsed_seconds": round(elapsed, 1),
+            },
+            f,
+            indent=2,
+        )
+    print(f"\n  Results saved to outputs/evals/results.json")
+
+
+if __name__ == "__main__":
+    main()

openenv.yaml

@@ -0,0 +1,6 @@
+spec_version: 1
+name: tiffin_packer
+type: space
+runtime: fastapi
+app: server.app:app
+port: 7860

pyproject.toml

@@ -0,0 +1,28 @@
+[project]
+name = "tiffin-packer"
+version = "1.0.0"
+description = "Smart Tiffin Packing Environment — OpenEnv compliant RL environment for semantic-aware constrained packing"
+authors = [{name = "CtrlAltWin", email = "team@ctrlaltwin.dev"}]
+license = {text = "MIT"}
+readme = "README.md"
+requires-python = ">=3.9"
+dependencies = [
+    "openenv-core>=0.1.0",
+    "fastapi>=0.104.0",
+    "uvicorn[standard]>=0.24.0",
+    "pydantic>=2.0.0",
+    "numpy>=1.24.0",
+    "requests>=2.28.0",
+    "openai>=1.0.0",
+    "pybullet>=3.2.5",
+]
+
+[project.scripts]
+tiffin-packer = "server.app:main"
+
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+include = ["tiffin_packer*", "server*"]

requirements.txt

@@ -0,0 +1,8 @@
+openenv-core>=0.1.0
+fastapi>=0.104.0
+uvicorn[standard]>=0.24.0
+pydantic>=2.0.0
+numpy>=1.24.0
+requests>=2.28.0
+openai>=1.0.0
+pybullet>=3.2.5

server/__init__.py

@@ -0,0 +1 @@
+# Server package

server/app.py

@@ -0,0 +1,38 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+FastAPI application for the Tiffin Packing Environment.
+
+Creates an HTTP + WebSocket server exposing the TiffinPackingEnvironment
+via the OpenEnv interface.
+
+Usage:
+    uvicorn server.app:app --host 0.0.0.0 --port 7860
+"""
+
+try:
+    from openenv.core.env_server.http_server import create_app
+except ImportError:
+    from openenv.core.env_server import create_app
+
+from tiffin_packer.models import TiffinAction, TiffinObservation
+from server.tiffin_environment import TiffinPackingEnvironment
+
+# Create the FastAPI app
+# Pass the class (factory) for WebSocket session support
+app = create_app(
+    TiffinPackingEnvironment,
+    TiffinAction,
+    TiffinObservation,
+    env_name="tiffin_packer",
+)
+
+
+def main():
+    """Entry point for direct execution."""
+    import uvicorn
+
+    uvicorn.run(app, host="0.0.0.0", port=7860)
+
+
+if __name__ == "__main__":
+    main()

server/tiffin_environment.py

@@ -0,0 +1,268 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Tiffin Packing Environment — OpenEnv Server Implementation.
+
+Wraps the packing simulation into the OpenEnv Environment base class,
+exposing step(), reset(), and state() for LLM agent interaction.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Optional
+from uuid import uuid4
+
+try:
+    from openenv.core.env_server import Environment
+except ImportError:
+    # Fallback for local testing without openenv installed
+    class Environment:
+        def __init__(self, **kwargs): pass
+        def reset(self, **kwargs): raise NotImplementedError
+        def step(self, action, **kwargs): raise NotImplementedError
+        @property
+        def state(self): raise NotImplementedError
+
+from tiffin_packer.models import TiffinAction, TiffinObservation, TiffinState
+from tiffin_packer.simulation.engine import PackingSimulation
+from tiffin_packer.vlm.classifier import FoodClassifier
+from tiffin_packer.tasks import get_task_config, list_tasks
+from tiffin_packer.grader import grade, grade_detailed
+
+
+class TiffinPackingEnvironment(Environment):
+    """
+    OpenEnv-compliant tiffin packing environment.
+
+    An LLM agent controls a robotic arm to identify food items using VLM
+    and pack them into the correct tiffin containers under real-world
+    constraints (type compatibility, volume, temperature, fragility).
+
+    Supports 3 tasks: easy, medium, hard.
+    """
+
+    def __init__(self):
+        super().__init__()
+        self.sim = PackingSimulation()
+        self.vlm = FoodClassifier()
+        self._state = TiffinState()
+        self._identified_items: set = set()
+        self._task_config = None
+
+    def reset(
+        self,
+        seed: Optional[int] = None,
+        episode_id: Optional[str] = None,
+        **kwargs: Any,
+    ) -> TiffinObservation:
+        """
+        Reset the environment for a new episode.
+
+        Args:
+            seed: Optional random seed for reproducibility.
+            episode_id: Optional custom episode ID.
+            **kwargs: Must include 'task_id' (easy/medium/hard).
+
+        Returns:
+            Initial TiffinObservation with scene description.
+        """
+        task_id = kwargs.get("task_id", "easy")
+
+        # Load task configuration
+        self._task_config = get_task_config(task_id, seed=seed)
+
+        # Reset simulation
+        self.sim.reset(
+            food_items=self._task_config.food_items,
+            containers=self._task_config.containers,
+            seed=seed,
+        )
+
+        # Reset state
+        self._state = TiffinState(
+            episode_id=episode_id or str(uuid4()),
+            step_count=0,
+            task_id=task_id,
+            items_packed=0,
+            total_items=len(self._task_config.food_items),
+            items_identified=0,
+            packing_log=[],
+            constraints_violated=[],
+        )
+        self._identified_items = set()
+
+        # Build initial observation
+        return self._build_observation(
+            reward=0.0,
+            done=False,
+            feedback=(
+                f"Episode started! Task: {task_id.upper()}\n\n"
+                f"{self._task_config.description}\n\n"
+                f"You have {self._task_config.max_steps} steps to pack "
+                f"{len(self._task_config.food_items)} food items into "
+                f"{len(self._task_config.containers)} containers.\n\n"
+                f"Start by using 'observe' to see the scene, then 'identify' "
+                f"each food item before packing."
+            ),
+        )
+
+    def step(
+        self,
+        action: TiffinAction,
+        timeout_s: Optional[float] = None,
+        **kwargs: Any,
+    ) -> TiffinObservation:
+        """
+        Execute one step in the environment.
+
+        Args:
+            action: TiffinAction with command and optional target_id.
+            timeout_s: Optional timeout (unused).
+
+        Returns:
+            TiffinObservation with updated scene state.
+        """
+        self._state.step_count += 1
+        reward = 0.0
+        done = False
+        vlm_result = None
+        feedback = ""
+
+        command = action.command.lower().strip()
+        target_id = action.target_id
+
+        # --- Dispatch command ---
+        if command == "observe":
+            _, feedback, reward = self.sim.observe()
+
+        elif command == "identify":
+            if target_id is None:
+                feedback = "Error: 'identify' requires a target_id (food item ID)."
+                reward = -0.1
+            else:
+                success, feedback, reward, vlm_result = self.sim.identify(target_id)
+                if success and vlm_result and vlm_result.get("name"):
+                    self._identified_items.add(target_id)
+                    self._state.items_identified = len(self._identified_items)
+
+        elif command == "pick":
+            if target_id is None:
+                feedback = "Error: 'pick' requires a target_id (food item ID)."
+                reward = -0.1
+            else:
+                success, feedback, reward = self.sim.pick(target_id)
+
+        elif command == "place":
+            if target_id is None:
+                feedback = "Error: 'place' requires a target_id (container ID)."
+                reward = -0.1
+            else:
+                success, feedback, reward = self.sim.place(target_id)
+                if success:
+                    self._state.items_packed = sum(
+                        1
+                        for i in self.sim.food_items
+                        if i.status == "packed"
+                    )
+                    self._state.packing_log = list(self.sim.packing_log)
+
+        elif command == "pour":
+            if target_id is None:
+                feedback = "Error: 'pour' requires a target_id (container ID)."
+                reward = -0.1
+            else:
+                success, feedback, reward = self.sim.pour(target_id)
+                if success:
+                    self._state.items_packed = sum(
+                        1
+                        for i in self.sim.food_items
+                        if i.status == "packed"
+                    )
+                    self._state.packing_log = list(self.sim.packing_log)
+
+        else:
+            feedback = (
+                f"Unknown command: '{command}'. "
+                f"Available commands: {self.sim.get_available_commands()}"
+            )
+            reward = -0.1
+
+        # --- Time penalty ---
+        reward -= 0.02
+
+        # --- Check termination ---
+        done = (
+            self.sim.all_packed
+            or self._state.step_count >= self._task_config.max_steps
+        )
+
+        # --- Final grading ---
+        final_score = None
+        grade_breakdown = None
+        if done:
+            grade_breakdown = grade_detailed(
+                self._state.packing_log, self._task_config
+            )
+            final_score = grade_breakdown["final_score"]
+            reward += final_score  # bonus = final grade
+
+            if self.sim.all_packed:
+                feedback += f"\n\n🎉 All items packed! Final score: {final_score:.4f}"
+            else:
+                feedback += (
+                    f"\n\n⏰ Time's up! {self.sim.unpacked_count} items remaining. "
+                    f"Final score: {final_score:.4f}"
+                )
+
+        return self._build_observation(
+            reward=reward,
+            done=done,
+            feedback=feedback,
+            vlm_result=vlm_result,
+            final_score=final_score,
+            grade_breakdown=grade_breakdown,
+        )
+
+    @property
+    def state(self) -> TiffinState:
+        """Return the current episode state."""
+        return self._state
+
+    # -------------------------------------------------------------------
+    # Helpers
+    # -------------------------------------------------------------------
+
+    def _build_observation(
+        self,
+        reward: float = 0.0,
+        done: bool = False,
+        feedback: str = "",
+        vlm_result: dict = None,
+        final_score: float = None,
+        grade_breakdown: dict = None,
+    ) -> TiffinObservation:
+        """Build a TiffinObservation from current state."""
+        metadata = {}
+        if final_score is not None:
+            metadata["final_score"] = final_score
+        if grade_breakdown is not None:
+            metadata["grade_breakdown"] = grade_breakdown
+
+        return TiffinObservation(
+            done=done,
+            reward=round(reward, 4),
+            metadata=metadata,
+            scene_description=self.sim.get_scene_description(),
+            food_items=[
+                item.to_dict(hide_unidentified=True)
+                for item in self.sim.food_items
+            ],
+            containers=[c.to_dict() for c in self.sim.containers],
+            held_item=(
+                self.sim.held_item.to_dict(hide_unidentified=False)
+                if self.sim.held_item
+                else None
+            ),
+            vlm_result=vlm_result,
+            available_commands=self.sim.get_available_commands(),
+            step_feedback=feedback,
+        )

tiffin_packer/__init__.py

@@ -0,0 +1,20 @@
+# Copyright (c) 2026 CtrlAltWin Team
+# Smart Tiffin Packing Environment for OpenEnv
+
+"""
+Tiffin Packer — A multimodal RL environment for semantic-aware
+constrained packing tasks inspired by real-world Indian meal organization.
+
+An LLM agent controls a robotic arm to identify food items (via VLM),
+reason about container compatibility, and pack a complete Indian meal
+into tiffin containers.
+"""
+
+from .models import TiffinAction, TiffinObservation, TiffinState
+
+try:
+    from .client import TiffinEnv
+except ImportError:
+    TiffinEnv = None  # Client requires openenv-core
+
+__all__ = ["TiffinAction", "TiffinObservation", "TiffinState", "TiffinEnv"]

tiffin_packer/client.py

@@ -0,0 +1,28 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Tiffin Packer Environment Client.
+
+Provides the client for connecting to a running TiffinPackingEnvironment server.
+"""
+
+try:
+    from openenv.core.env_client import EnvClient
+except ImportError:
+    # Fallback if openenv not installed
+    EnvClient = object
+
+from .models import TiffinAction, TiffinObservation
+
+
+class TiffinEnv(EnvClient):
+    """
+    Client for the Tiffin Packing Environment.
+
+    Example:
+        >>> with TiffinEnv(base_url="http://localhost:7860").sync() as env:
+        ...     obs = env.reset(task_id="easy")
+        ...     obs = env.step(TiffinAction(command="observe"))
+        ...     print(obs.scene_description)
+    """
+
+    pass  # EnvClient provides all needed functionality

tiffin_packer/grader.py

@@ -0,0 +1,237 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Deterministic Grader — Scores packing quality from 0.0 to 1.0.
+
+Scoring formula:
+    score = 0.4 * validity + 0.3 * efficiency + 0.2 * constraints + 0.1 * neatness
+
+Each component:
+    validity  — food placed in type-compatible container?
+    efficiency — space utilization vs total capacity used
+    constraints — temperature separation, fragility, flavor isolation
+    neatness  — all items packed? nothing dropped?
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from .tasks import TaskConfig
+from .simulation.engine import is_type_compatible
+
+
+def grade(
+    packing_log: List[Dict[str, Any]],
+    task_config: TaskConfig,
+) -> float:
+    """
+    Grade a packing episode. Returns score between 0.0 and 1.0.
+
+    Args:
+        packing_log: List of placement records from the simulation.
+        task_config: The task configuration used for this episode.
+
+    Returns:
+        Final score (0.0 to 1.0), rounded to 4 decimal places.
+    """
+    total_items = len(task_config.food_items)
+
+    if total_items == 0:
+        return 0.0
+
+    # ---- Validity (40%) ----
+    validity = _score_validity(packing_log, total_items)
+
+    # ---- Efficiency (30%) ----
+    efficiency = _score_efficiency(packing_log, task_config)
+
+    # ---- Constraint Satisfaction (20%) ----
+    constraints = _score_constraints(packing_log, task_config)
+
+    # ---- Neatness (10%) ----
+    neatness = _score_neatness(packing_log, total_items)
+
+    # ---- Final score ----
+    score = 0.4 * validity + 0.3 * efficiency + 0.2 * constraints + 0.1 * neatness
+    return round(max(0.0, min(1.0, score)), 4)
+
+
+def grade_detailed(
+    packing_log: List[Dict[str, Any]],
+    task_config: TaskConfig,
+) -> Dict[str, Any]:
+    """Grade with full breakdown for debugging."""
+    total_items = len(task_config.food_items)
+
+    validity = _score_validity(packing_log, total_items)
+    efficiency = _score_efficiency(packing_log, task_config)
+    constraints = _score_constraints(packing_log, task_config)
+    neatness = _score_neatness(packing_log, total_items)
+
+    score = 0.4 * validity + 0.3 * efficiency + 0.2 * constraints + 0.1 * neatness
+    score = round(max(0.0, min(1.0, score)), 4)
+
+    return {
+        "final_score": score,
+        "validity": round(validity, 4),
+        "efficiency": round(efficiency, 4),
+        "constraints": round(constraints, 4),
+        "neatness": round(neatness, 4),
+        "items_packed": len(packing_log),
+        "total_items": total_items,
+        "weights": {
+            "validity": 0.4,
+            "efficiency": 0.3,
+            "constraints": 0.2,
+            "neatness": 0.1,
+        },
+    }
+
+
+# -----------------------------------------------------------------------
+# Component scorers
+# -----------------------------------------------------------------------
+
+
+def _score_validity(packing_log: List[Dict], total_items: int) -> float:
+    """Score: food placed in type-compatible container? (0-1)"""
+    if not packing_log:
+        return 0.0
+
+    correct = sum(1 for entry in packing_log if entry.get("type_compatible", False))
+    return correct / max(total_items, 1)
+
+
+def _score_efficiency(packing_log: List[Dict], task_config: TaskConfig) -> float:
+    """Score: how well is container space utilized? (0-1)"""
+    if not packing_log:
+        return 0.0
+
+    total_food_vol = sum(entry.get("food_volume", 0) for entry in packing_log)
+
+    # Find which containers were used
+    used_container_ids = set(entry.get("container_id") for entry in packing_log)
+    total_capacity = sum(
+        c.capacity_ml
+        for c in task_config.containers
+        if c.id in used_container_ids
+    )
+
+    if total_capacity == 0:
+        return 0.0
+
+    utilization = total_food_vol / total_capacity
+
+    # Penalize overflow
+    overflow_count = sum(1 for entry in packing_log if entry.get("overflow", False))
+    if overflow_count > 0:
+        utilization *= max(0.3, 1.0 - 0.2 * overflow_count)
+
+    return min(1.0, utilization)
+
+
+def _score_constraints(packing_log: List[Dict], task_config: TaskConfig) -> float:
+    """Score: task-specific constraints satisfied? (0-1)"""
+    if not packing_log:
+        return 0.0
+
+    scores = []
+    active = set(task_config.constraints)
+
+    if "temperature_separation" in active:
+        scores.append(_check_temperature(packing_log))
+
+    if "fragility_ordering" in active:
+        scores.append(_check_fragility(packing_log))
+
+    if "flavor_isolation" in active:
+        scores.append(_check_flavor_isolation(packing_log))
+
+    if "no_overflow" in active:
+        overflow_count = sum(1 for e in packing_log if e.get("overflow", False))
+        scores.append(1.0 if overflow_count == 0 else max(0.0, 1.0 - 0.3 * overflow_count))
+
+    if "type_match" in active:
+        correct = sum(1 for e in packing_log if e.get("type_compatible", False))
+        scores.append(correct / max(len(packing_log), 1))
+
+    if not scores:
+        return 1.0  # no constraints to violate
+
+    return sum(scores) / len(scores)
+
+
+def _check_temperature(packing_log: List[Dict]) -> float:
+    """Check if hot and cold items are kept separate."""
+    # Group items by container
+    container_temps: Dict[int, List[str]] = {}
+    for entry in packing_log:
+        cid = entry.get("container_id")
+        temp = entry.get("food_temperature", "room")
+        container_temps.setdefault(cid, []).append(temp)
+
+    violations = 0
+    total_containers = len(container_temps)
+    for temps in container_temps.values():
+        if "hot" in temps and "cold" in temps:
+            violations += 1
+
+    if total_containers == 0:
+        return 1.0
+    return max(0.0, 1.0 - violations / total_containers)
+
+
+def _check_fragility(packing_log: List[Dict]) -> float:
+    """Check if fragile items are not crushed by heavy items placed after them."""
+    # Group by container, check placement order
+    container_order: Dict[int, List[float]] = {}
+    for entry in packing_log:
+        cid = entry.get("container_id")
+        frag = entry.get("food_fragility", 0.5)
+        container_order.setdefault(cid, []).append(frag)
+
+    violations = 0
+    checks = 0
+    for fragilites in container_order.values():
+        for i in range(1, len(fragilites)):
+            checks += 1
+            # If a less fragile (heavy) item is placed AFTER a more fragile item
+            if fragilites[i] < 0.4 and fragilites[i - 1] > 0.6:
+                violations += 1
+
+    if checks == 0:
+        return 1.0
+    return max(0.0, 1.0 - violations / max(checks, 1))
+
+
+def _check_flavor_isolation(packing_log: List[Dict]) -> float:
+    """Check that strong-flavor items (pickle, chutney) are isolated."""
+    strong_flavors = {"pickle", "chutney"}
+    # Group by container
+    container_contents: Dict[int, List[str]] = {}
+    for entry in packing_log:
+        cid = entry.get("container_id")
+        name = entry.get("food_name", "")
+        container_contents.setdefault(cid, []).append(name)
+
+    violations = 0
+    total = 0
+    for contents in container_contents.values():
+        has_strong = any(c in strong_flavors for c in contents)
+        has_others = any(c not in strong_flavors for c in contents)
+        if has_strong and has_others and len(contents) > 1:
+            violations += 1
+            total += 1
+        elif has_strong:
+            total += 1
+
+    if total == 0:
+        return 1.0
+    return max(0.0, 1.0 - violations / max(total, 1))
+
+
+def _score_neatness(packing_log: List[Dict], total_items: int) -> float:
+    """Score: fraction of items successfully packed. (0-1)"""
+    if total_items == 0:
+        return 0.0
+    return len(packing_log) / total_items

tiffin_packer/models.py

@@ -0,0 +1,132 @@
+# Copyright (c) 2026 CtrlAltWin Team
+# Smart Tiffin Packing Environment — Pydantic Models
+
+"""
+Typed data models for the Tiffin Packing OpenEnv environment.
+Follows the OpenEnv specification with Action, Observation, and State base classes.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from pydantic import Field
+
+try:
+    from openenv.core.env_server import Action, Observation, State
+except ImportError:
+    try:
+        from openenv.core.env_server.types import Action, Observation, State
+    except ImportError:
+        # Fallback: define compatible base classes when openenv is not installed
+        from pydantic import BaseModel, ConfigDict
+
+        class Action(BaseModel):
+            model_config = ConfigDict(extra="forbid", validate_assignment=True, arbitrary_types_allowed=True)
+            metadata: Dict[str, Any] = Field(default_factory=dict)
+
+        class Observation(BaseModel):
+            model_config = ConfigDict(extra="forbid", validate_assignment=True, arbitrary_types_allowed=True)
+            done: bool = Field(default=False)
+            reward: Optional[float] = Field(default=None)
+            metadata: Dict[str, Any] = Field(default_factory=dict)
+
+        class State(BaseModel):
+            model_config = ConfigDict(extra="allow", validate_assignment=True, arbitrary_types_allowed=True)
+            episode_id: Optional[str] = Field(default=None)
+            step_count: int = Field(default=0)
+
+
+class TiffinAction(Action):
+    """
+    High-level command the LLM agent issues to the robotic arm.
+
+    Available commands:
+        - "observe"  : Get a full scene description (no target_id needed)
+        - "identify" : Use VLM to classify a food item (target_id = food item ID)
+        - "pick"     : Pick up a food item with the robotic arm (target_id = food item ID)
+        - "place"    : Place the currently held item into a container (target_id = container ID)
+        - "pour"     : Pour liquid from held bowl into a container (target_id = container ID)
+
+    Attributes:
+        command: The action command string.
+        target_id: The ID of the food item or container to act on.
+    """
+
+    command: str = Field(
+        description="One of: 'observe', 'identify', 'pick', 'place', 'pour'"
+    )
+    target_id: Optional[int] = Field(
+        default=None,
+        description="ID of food item (for identify/pick) or container (for place/pour)",
+    )
+
+
+class TiffinObservation(Observation):
+    """
+    Observation returned after each action.
+
+    Contains a natural-language scene description, structured data about
+    food items and containers, and feedback on the last action.
+
+    Attributes:
+        scene_description: Human-readable text describing the current scene.
+        food_items: List of food item dicts with id, name, status, etc.
+        containers: List of container dicts with id, type, capacity, contents.
+        held_item: The food item currently held by the robotic arm, if any.
+        vlm_result: VLM classification result after an 'identify' command.
+        available_commands: Commands the agent can issue right now.
+        step_feedback: Text feedback on the outcome of the last action.
+    """
+
+    scene_description: str = Field(
+        default="", description="Natural language description of current scene state"
+    )
+    food_items: List[Dict[str, Any]] = Field(
+        default_factory=list,
+        description="List of food items: [{id, name, status, position}]",
+    )
+    containers: List[Dict[str, Any]] = Field(
+        default_factory=list,
+        description="List of containers: [{id, type, capacity_ml, filled_ml, contents}]",
+    )
+    held_item: Optional[Dict[str, Any]] = Field(
+        default=None,
+        description="Currently held food item, or None if gripper is empty",
+    )
+    vlm_result: Optional[Dict[str, Any]] = Field(
+        default=None,
+        description="VLM classification result after 'identify' command",
+    )
+    available_commands: List[str] = Field(
+        default_factory=list,
+        description="Valid commands the agent can issue right now",
+    )
+    step_feedback: str = Field(
+        default="", description="Feedback on the last action (success/failure reason)"
+    )
+
+
+class TiffinState(State):
+    """
+    Internal episode state for tracking progress.
+
+    Attributes:
+        task_id: Which task is active (easy/medium/hard).
+        items_packed: Number of items successfully packed.
+        total_items: Total items that need to be packed.
+        items_identified: Number of items that have been VLM-classified.
+        packing_log: Record of each placement decision.
+        constraints_violated: List of constraint violations.
+    """
+
+    task_id: str = Field(default="easy", description="Active task ID")
+    items_packed: int = Field(default=0, description="Items successfully packed")
+    total_items: int = Field(default=0, description="Total items to pack")
+    items_identified: int = Field(default=0, description="Items VLM-classified")
+    packing_log: List[Dict[str, Any]] = Field(
+        default_factory=list, description="Record of placement decisions"
+    )
+    constraints_violated: List[str] = Field(
+        default_factory=list, description="Constraint violations"
+    )

tiffin_packer/simulation/__init__.py

@@ -0,0 +1,3 @@
+from .engine import PackingSimulation
+
+__all__ = ["PackingSimulation"]

tiffin_packer/simulation/engine.py

@@ -0,0 +1,538 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Tiffin Packing Simulation Engine — Pure Logic + PyBullet Physics.
+
+This module implements the core packing simulation. It operates in two modes:
+
+1. **Logic mode** (default): Pure Python state tracking — fast, lightweight,
+   guaranteed to run on 2 vCPU / 8 GB RAM. Used for all OpenEnv interactions.
+
+2. **Physics mode** (optional): PyBullet simulation with real URDF models
+   (Kuka arm, table, containers, food cubes/spheres). Used for rendering
+   and visual validation.
+
+The LLM agent issues high-level commands (pick, place, pour, identify),
+and this engine validates and executes them.
+"""
+
+from __future__ import annotations
+
+import math
+import random
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+
+# ---------------------------------------------------------------------------
+# Data structures
+# ---------------------------------------------------------------------------
+
+@dataclass
+class FoodItem:
+    """Represents a food item on the table."""
+
+    id: int
+    name: str
+    food_type: str  # "solid" | "liquid" | "semi-solid"
+    volume_ml: float
+    temperature: str  # "hot" | "cold" | "room"
+    fragility: float  # 0.0 (sturdy) to 1.0 (very fragile)
+    preferred_container: str  # "sealed" | "flat" | "deep"
+    color: str = "unknown"
+    special_notes: str = ""
+    status: str = "on_table"  # "on_table" | "held" | "packed" | "dropped"
+    identified: bool = False
+    position: Tuple[float, float, float] = (0.0, 0.0, 0.0)
+
+    def to_dict(self, hide_unidentified: bool = True) -> Dict[str, Any]:
+        """Convert to observation dict. Hides properties if not yet identified."""
+        base = {"id": self.id, "status": self.status}
+        if self.identified or not hide_unidentified:
+            base.update(
+                {
+                    "name": self.name,
+                    "food_type": self.food_type,
+                    "volume_ml": self.volume_ml,
+                    "temperature": self.temperature,
+                    "fragility": self.fragility,
+                    "preferred_container": self.preferred_container,
+                    "color": self.color,
+                }
+            )
+        else:
+            base["name"] = f"Unknown food item #{self.id}"
+            base["food_type"] = "unknown"
+            base["hint"] = "Use 'identify' command to classify this item"
+        return base
+
+
+@dataclass
+class Container:
+    """Represents a tiffin container."""
+
+    id: int
+    name: str
+    container_type: str  # "sealed_round" | "flat_open" | "deep_box" | "small_sealed"
+    capacity_ml: float
+    filled_ml: float = 0.0
+    contents: List[str] = field(default_factory=list)
+    content_types: List[str] = field(default_factory=list)  # food types inside
+    content_temperatures: List[str] = field(default_factory=list)
+    content_fragilites: List[float] = field(default_factory=list)
+    position: Tuple[float, float, float] = (0.0, 0.0, 0.0)
+
+    @property
+    def remaining_ml(self) -> float:
+        return max(0, self.capacity_ml - self.filled_ml)
+
+    @property
+    def fill_percentage(self) -> float:
+        return (self.filled_ml / self.capacity_ml) * 100 if self.capacity_ml > 0 else 0
+
+    @property
+    def accepts_liquid(self) -> bool:
+        """Sealed containers can hold liquids."""
+        return "sealed" in self.container_type
+
+    @property
+    def is_flat(self) -> bool:
+        return "flat" in self.container_type
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "id": self.id,
+            "name": self.name,
+            "type": self.container_type,
+            "capacity_ml": self.capacity_ml,
+            "filled_ml": round(self.filled_ml, 1),
+            "remaining_ml": round(self.remaining_ml, 1),
+            "fill_percentage": round(self.fill_percentage, 1),
+            "contents": self.contents,
+            "accepts_liquid": self.accepts_liquid,
+        }
+
+
+# ---------------------------------------------------------------------------
+# Compatibility rules
+# ---------------------------------------------------------------------------
+
+CONTAINER_TYPE_COMPATIBILITY = {
+    # food_type -> set of compatible container_types
+    "liquid": {"sealed_round", "small_sealed"},
+    "semi-solid": {"sealed_round", "small_sealed", "deep_box"},
+    "solid": {"sealed_round", "flat_open", "deep_box", "small_sealed"},
+}
+
+
+def is_type_compatible(food_type: str, container_type: str) -> bool:
+    """Check if a food type is compatible with a container type."""
+    compatible = CONTAINER_TYPE_COMPATIBILITY.get(food_type, set())
+    return container_type in compatible
+
+
+# ---------------------------------------------------------------------------
+# Main simulation engine
+# ---------------------------------------------------------------------------
+
+class PackingSimulation:
+    """
+    Pure-logic tiffin packing simulation.
+
+    Models a robotic arm, food items, and containers as data structures.
+    Validates all actions against physical constraints (volume, type
+    compatibility, temperature zones, fragility ordering).
+    """
+
+    def __init__(self):
+        self.arm_state: str = "idle"  # "idle" | "holding"
+        self.held_item: Optional[FoodItem] = None
+        self.food_items: List[FoodItem] = []
+        self.containers: List[Container] = []
+        self.packing_log: List[Dict[str, Any]] = []
+        self._step_count: int = 0
+
+    def reset(
+        self,
+        food_items: List[FoodItem],
+        containers: List[Container],
+        seed: Optional[int] = None,
+    ):
+        """Initialize simulation with food items and containers."""
+        if seed is not None:
+            random.seed(seed)
+
+        self.arm_state = "idle"
+        self.held_item = None
+        self.food_items = food_items
+        self.containers = containers
+        self.packing_log = []
+        self._step_count = 0
+
+        # Randomize positions on table
+        for i, item in enumerate(self.food_items):
+            angle = (2 * math.pi * i) / max(len(self.food_items), 1)
+            item.position = (
+                0.3 * math.cos(angle),
+                0.3 * math.sin(angle),
+                0.65,  # table height
+            )
+
+        for i, container in enumerate(self.containers):
+            container.position = (
+                -0.4 + 0.25 * i,
+                0.5,
+                0.65,
+            )
+
+    # -------------------------------------------------------------------
+    # Actions
+    # -------------------------------------------------------------------
+
+    def observe(self) -> Tuple[bool, str, float]:
+        """Get detailed scene description. Returns (success, feedback, reward)."""
+        desc = self.get_scene_description()
+        return True, desc, 0.05  # small reward for observing
+
+    def identify(self, item_id: int) -> Tuple[bool, str, float, Optional[Dict]]:
+        """
+        Classify a food item using VLM.
+        Returns (success, feedback, reward, vlm_result_or_None).
+        """
+        item = self._find_item(item_id)
+        if item is None:
+            return False, f"No food item with ID {item_id} found.", -0.1, None
+
+        if item.status == "packed":
+            return (
+                False,
+                f"Item #{item_id} ({item.name}) is already packed.",
+                -0.05,
+                None,
+            )
+
+        if item.identified:
+            # Re-identifying is allowed but gives no reward
+            vlm_result = {
+                "name": item.name,
+                "type": item.food_type,
+                "fragility": item.fragility,
+                "preferred_container": item.preferred_container,
+                "volume_ml": item.volume_ml,
+                "temperature": item.temperature,
+                "color": item.color,
+                "special_notes": item.special_notes,
+            }
+            return (
+                True,
+                f"Item #{item_id} already identified as '{item.name}'. {item.special_notes}",
+                0.0,
+                vlm_result,
+            )
+
+        # First-time identification
+        item.identified = True
+        vlm_result = {
+            "name": item.name,
+            "type": item.food_type,
+            "fragility": item.fragility,
+            "preferred_container": item.preferred_container,
+            "volume_ml": item.volume_ml,
+            "temperature": item.temperature,
+            "color": item.color,
+            "special_notes": item.special_notes,
+        }
+        return (
+            True,
+            f"VLM identified item #{item_id}: '{item.name}' — "
+            f"type={item.food_type}, volume={item.volume_ml}ml, "
+            f"temperature={item.temperature}, fragility={item.fragility:.1f}, "
+            f"preferred container={item.preferred_container}. "
+            f"Note: {item.special_notes}",
+            0.1,  # reward for gathering information
+            vlm_result,
+        )
+
+    def pick(self, item_id: int) -> Tuple[bool, str, float]:
+        """Pick up a food item. Returns (success, feedback, reward)."""
+        if self.arm_state == "holding":
+            return (
+                False,
+                f"Arm is already holding '{self.held_item.name}'. "
+                f"Place or pour it first before picking another item.",
+                -0.1,
+            )
+
+        item = self._find_item(item_id)
+        if item is None:
+            return False, f"No food item with ID {item_id} found.", -0.1
+
+        if item.status != "on_table":
+            return (
+                False,
+                f"Item #{item_id} ({item.name}) cannot be picked — status is '{item.status}'.",
+                -0.1,
+            )
+
+        # Success — pick up the item
+        item.status = "held"
+        self.held_item = item
+        self.arm_state = "holding"
+        return (
+            True,
+            f"Successfully picked up item #{item_id} "
+            f"({'identified as ' + item.name if item.identified else 'unidentified'}).",
+            0.3,
+        )
+
+    def place(self, container_id: int) -> Tuple[bool, str, float]:
+        """Place held item into container. Returns (success, feedback, reward)."""
+        if self.arm_state != "holding" or self.held_item is None:
+            return (
+                False,
+                "Arm is not holding any item. Use 'pick' first.",
+                -0.1,
+            )
+
+        container = self._find_container(container_id)
+        if container is None:
+            return False, f"No container with ID {container_id} found.", -0.1
+
+        item = self.held_item
+        reward = 0.0
+        feedback_parts = []
+
+        # --- Check type compatibility ---
+        type_ok = is_type_compatible(item.food_type, container.container_type)
+        if not type_ok:
+            reward -= 1.5
+            feedback_parts.append(
+                f"WARNING: {item.food_type} food in {container.container_type} "
+                f"container is incompatible! (e.g. liquid will spill from open container)"
+            )
+
+        # --- Check volume overflow ---
+        if item.volume_ml > container.remaining_ml:
+            overflow = item.volume_ml - container.remaining_ml
+            reward -= 1.0
+            feedback_parts.append(
+                f"WARNING: Overflow! Item needs {item.volume_ml}ml but container "
+                f"only has {container.remaining_ml:.0f}ml remaining. "
+                f"Overflow of {overflow:.0f}ml!"
+            )
+
+        # --- Check temperature mixing ---
+        if container.content_temperatures:
+            existing_temps = set(container.content_temperatures)
+            if item.temperature == "hot" and "cold" in existing_temps:
+                reward -= 0.5
+                feedback_parts.append(
+                    "WARNING: Placing hot food with cold items! "
+                    "Temperature contamination will occur."
+                )
+            elif item.temperature == "cold" and "hot" in existing_temps:
+                reward -= 0.5
+                feedback_parts.append(
+                    "WARNING: Placing cold food with hot items! "
+                    "Temperature contamination will occur."
+                )
+
+        # --- Check fragility ---
+        if container.content_fragilites and item.fragility < 0.5:
+            # Placing heavy/sturdy item — check if fragile items are under
+            max_existing_fragility = max(container.content_fragilites)
+            if max_existing_fragility > 0.6:
+                reward -= 0.3
+                feedback_parts.append(
+                    f"WARNING: Placing sturdy item on top of fragile item "
+                    f"(fragility {max_existing_fragility:.1f}) — may crush it!"
+                )
+
+        # --- Positive rewards ---
+        if type_ok:
+            reward += 1.5  # correct container type
+            if container.container_type == item.preferred_container or (
+                item.preferred_container in container.container_type
+            ):
+                reward += 0.5  # preferred container bonus
+                feedback_parts.append("Great choice — matches preferred container type!")
+
+        if item.volume_ml <= container.remaining_ml:
+            # Good volume fit
+            utilization = item.volume_ml / container.capacity_ml
+            reward += 0.3 * utilization  # reward proportional to space usage
+
+        # --- Execute placement ---
+        container.filled_ml += item.volume_ml
+        container.contents.append(item.name)
+        container.content_types.append(item.food_type)
+        container.content_temperatures.append(item.temperature)
+        container.content_fragilites.append(item.fragility)
+        item.status = "packed"
+        self.held_item = None
+        self.arm_state = "idle"
+
+        # Log the placement
+        self.packing_log.append(
+            {
+                "food_name": item.name,
+                "food_id": item.id,
+                "food_type": item.food_type,
+                "food_volume": item.volume_ml,
+                "food_temperature": item.temperature,
+                "food_fragility": item.fragility,
+                "food_preferred_container": item.preferred_container,
+                "container_id": container.id,
+                "container_type": container.container_type,
+                "container_name": container.name,
+                "type_compatible": type_ok,
+                "overflow": item.volume_ml > container.remaining_ml + item.volume_ml,
+            }
+        )
+
+        if feedback_parts:
+            feedback = f"Placed '{item.name}' in '{container.name}'. " + " ".join(
+                feedback_parts
+            )
+        else:
+            feedback = (
+                f"Placed '{item.name}' in '{container.name}'. "
+                f"Container now {container.fill_percentage:.0f}% full."
+            )
+
+        return True, feedback, round(reward, 2)
+
+    def pour(self, container_id: int) -> Tuple[bool, str, float]:
+        """Pour liquid from held item into container. Returns (success, feedback, reward)."""
+        if self.arm_state != "holding" or self.held_item is None:
+            return False, "Arm is not holding any item. Use 'pick' first.", -0.1
+
+        item = self.held_item
+
+        # Only liquids or semi-solids can be poured
+        if item.food_type not in ("liquid", "semi-solid"):
+            return (
+                False,
+                f"Cannot pour '{item.name}' — it is '{item.food_type}', not a pourable item. "
+                f"Use 'place' instead.",
+                -0.1,
+            )
+
+        # Pour is functionally same as place but gives extra reward for liquids
+        success, feedback, reward = self.place(container_id)
+        if success:
+            reward += 0.2  # bonus for correctly using pour for liquids
+            feedback = feedback.replace("Placed", "Poured")
+        return success, feedback, reward
+
+    # -------------------------------------------------------------------
+    # Scene description
+    # -------------------------------------------------------------------
+
+    def get_scene_description(self) -> str:
+        """Generate natural language description of the current scene."""
+        lines = []
+        lines.append("=" * 60)
+        lines.append("TIFFIN PACKING SCENE")
+        lines.append("=" * 60)
+
+        # Arm state
+        lines.append("")
+        lines.append("🤖 ROBOTIC ARM STATUS:")
+        if self.arm_state == "holding" and self.held_item:
+            item = self.held_item
+            if item.identified:
+                lines.append(
+                    f"   Currently holding: {item.name} "
+                    f"(type={item.food_type}, volume={item.volume_ml}ml)"
+                )
+            else:
+                lines.append(f"   Currently holding: Unknown food item #{item.id}")
+        else:
+            lines.append("   Arm is idle — ready to pick up an item")
+
+        # Food items on table
+        on_table = [i for i in self.food_items if i.status == "on_table"]
+        packed = [i for i in self.food_items if i.status == "packed"]
+        lines.append("")
+        lines.append(f"🍛 FOOD ITEMS ON TABLE ({len(on_table)} remaining, {len(packed)} packed):")
+        for item in self.food_items:
+            status_icon = {"on_table": "⬜", "held": "🤏", "packed": "✅", "dropped": "❌"}.get(
+                item.status, "?"
+            )
+            if item.identified:
+                lines.append(
+                    f"   {status_icon} [{item.id}] {item.name} — "
+                    f"type={item.food_type}, volume={item.volume_ml}ml, "
+                    f"temp={item.temperature}, fragility={item.fragility:.1f}, "
+                    f"preferred={item.preferred_container}"
+                )
+            else:
+                lines.append(
+                    f"   {status_icon} [{item.id}] Unknown food item "
+                    f"(use 'identify' to classify)"
+                )
+
+        # Containers
+        lines.append("")
+        lines.append("🍱 TIFFIN CONTAINERS:")
+        for c in self.containers:
+            bar_len = 20
+            filled_bars = int((c.fill_percentage / 100) * bar_len)
+            bar = "█" * filled_bars + "░" * (bar_len - filled_bars)
+            lines.append(
+                f"   [{c.id}] {c.name} ({c.container_type}) — "
+                f"[{bar}] {c.fill_percentage:.0f}% "
+                f"({c.filled_ml:.0f}/{c.capacity_ml:.0f}ml)"
+            )
+            if c.contents:
+                lines.append(f"       Contains: {', '.join(c.contents)}")
+            liquid_note = "✅ Can hold liquids" if c.accepts_liquid else "⚠️ Open — no liquids"
+            lines.append(f"       {liquid_note}")
+
+        lines.append("")
+        lines.append("=" * 60)
+        return "\n".join(lines)
+
+    def get_available_commands(self) -> List[str]:
+        """Return list of valid commands given current state."""
+        commands = ["observe"]
+
+        unpacked = [i for i in self.food_items if i.status == "on_table"]
+        unidentified = [i for i in self.food_items if not i.identified and i.status != "packed"]
+
+        if unidentified:
+            commands.append("identify")
+
+        if self.arm_state == "idle" and unpacked:
+            commands.append("pick")
+
+        if self.arm_state == "holding" and self.held_item:
+            commands.append("place")
+            if self.held_item.food_type in ("liquid", "semi-solid"):
+                commands.append("pour")
+
+        return commands
+
+    # -------------------------------------------------------------------
+    # Helpers
+    # -------------------------------------------------------------------
+
+    def _find_item(self, item_id: int) -> Optional[FoodItem]:
+        for item in self.food_items:
+            if item.id == item_id:
+                return item
+        return None
+
+    def _find_container(self, container_id: int) -> Optional[Container]:
+        for c in self.containers:
+            if c.id == container_id:
+                return c
+        return None
+
+    @property
+    def all_packed(self) -> bool:
+        return all(i.status == "packed" for i in self.food_items)
+
+    @property
+    def unpacked_count(self) -> int:
+        return sum(1 for i in self.food_items if i.status != "packed")

tiffin_packer/simulation/pybullet_renderer.py

@@ -0,0 +1,354 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+PyBullet Rendering Module — Real physics visualization using URDF models.
+
+Provides an optional physics-backed renderer that loads real URDF models
+(Kuka robot arm, table, containers, food items) and renders frames.
+
+This module is used for:
+1. Generating visual frames for the frontend viewer
+2. Physics validation of placements
+3. Demo/presentation screenshots
+
+The simulation engine (engine.py) handles all logic — this module only
+provides visualization and optional physics validation.
+"""
+
+from __future__ import annotations
+
+import base64
+import io
+import math
+import os
+from typing import Any, Dict, List, Optional, Tuple
+
+import numpy as np
+
+# PyBullet may not be available in all environments
+try:
+    import pybullet as p
+    import pybullet_data
+
+    PYBULLET_AVAILABLE = True
+except ImportError:
+    PYBULLET_AVAILABLE = False
+
+
+# Color presets for food items
+FOOD_COLORS = {
+    "rice": [1.0, 1.0, 0.9, 1.0],  # white
+    "sambar": [0.9, 0.5, 0.1, 1.0],  # orange
+    "curd": [1.0, 1.0, 0.95, 1.0],  # off-white
+    "chapati": [0.8, 0.6, 0.3, 1.0],  # brown
+    "pickle": [0.8, 0.1, 0.1, 1.0],  # red
+    "dal": [0.9, 0.8, 0.2, 1.0],  # yellow
+    "rasam": [0.6, 0.1, 0.05, 1.0],  # dark red
+    "poriyal": [0.2, 0.7, 0.2, 1.0],  # green
+    "papad": [0.9, 0.8, 0.4, 1.0],  # golden
+    "raita": [0.8, 0.9, 0.8, 1.0],  # pale green
+    "idli": [1.0, 1.0, 0.95, 1.0],  # white
+    "chutney": [0.1, 0.6, 0.1, 1.0],  # green
+    "biryani": [0.9, 0.7, 0.2, 1.0],  # saffron
+    "curry": [0.6, 0.3, 0.1, 1.0],  # brown
+    "salad": [0.3, 0.8, 0.3, 1.0],  # mixed green
+}
+
+# Container colors
+CONTAINER_COLORS = {
+    "sealed_round": [0.7, 0.7, 0.8, 0.7],  # steel blue
+    "flat_open": [0.8, 0.6, 0.3, 0.8],  # bronze
+    "deep_box": [0.6, 0.6, 0.7, 0.7],  # grey steel
+    "small_sealed": [0.9, 0.9, 0.95, 0.7],  # silver
+}
+
+
+class PyBulletRenderer:
+    """
+    Optional PyBullet-based renderer for the tiffin packing scene.
+
+    Creates a physics simulation with:
+    - Kuka IIWA robot arm (from pybullet_data)
+    - Table (box primitive)
+    - Food items (colored cubes/spheres on table)
+    - Tiffin containers (open-top box composites)
+    """
+
+    def __init__(self, gui: bool = False):
+        if not PYBULLET_AVAILABLE:
+            raise ImportError(
+                "pybullet is not installed. Install with: pip install pybullet"
+            )
+
+        self._gui = gui
+        self._physics_client = None
+        self._robot_id = None
+        self._table_id = None
+        self._food_ids: Dict[int, int] = {}  # food_item_id -> bullet_body_id
+        self._container_ids: Dict[int, int] = {}  # container_id -> bullet_body_id
+        self._initialized = False
+
+    def initialize(self):
+        """Start the PyBullet physics server."""
+        if self._initialized:
+            return
+
+        if self._gui:
+            self._physics_client = p.connect(p.GUI)
+        else:
+            self._physics_client = p.connect(p.DIRECT)
+
+        p.setAdditionalSearchPath(pybullet_data.getDataPath())
+        p.setGravity(0, 0, -9.81)
+
+        # Load ground plane
+        p.loadURDF("plane.urdf")
+
+        self._initialized = True
+
+    def setup_scene(
+        self,
+        food_items: list,
+        containers: list,
+    ):
+        """
+        Set up the full PyBullet scene with robot, table, food, containers.
+
+        Args:
+            food_items: List of FoodItem dataclasses
+            containers: List of Container dataclasses
+        """
+        self.initialize()
+
+        # Clear previous objects
+        self._clear_objects()
+
+        # --- Table ---
+        table_half_extents = [0.4, 0.6, 0.02]
+        table_col = p.createCollisionShape(p.GEOM_BOX, halfExtents=table_half_extents)
+        table_vis = p.createVisualShape(
+            p.GEOM_BOX,
+            halfExtents=table_half_extents,
+            rgbaColor=[0.6, 0.4, 0.2, 1.0],
+        )
+        self._table_id = p.createMultiBody(
+            baseMass=0,
+            baseCollisionShapeIndex=table_col,
+            baseVisualShapeIndex=table_vis,
+            basePosition=[0, 0, 0.6],
+        )
+
+        # Table legs
+        for lx, ly in [(-0.35, -0.55), (-0.35, 0.55), (0.35, -0.55), (0.35, 0.55)]:
+            leg_col = p.createCollisionShape(
+                p.GEOM_BOX, halfExtents=[0.02, 0.02, 0.3]
+            )
+            leg_vis = p.createVisualShape(
+                p.GEOM_BOX,
+                halfExtents=[0.02, 0.02, 0.3],
+                rgbaColor=[0.5, 0.3, 0.15, 1.0],
+            )
+            p.createMultiBody(
+                baseMass=0,
+                baseCollisionShapeIndex=leg_col,
+                baseVisualShapeIndex=leg_vis,
+                basePosition=[lx, ly, 0.3],
+            )
+
+        # --- Robot arm (Kuka IIWA) ---
+        self._robot_id = p.loadURDF(
+            "kuka_iiwa/model.urdf",
+            basePosition=[-0.5, 0, 0.62],
+            useFixedBase=True,
+        )
+
+        # --- Food items ---
+        for item in food_items:
+            color = FOOD_COLORS.get(item.name, [0.5, 0.5, 0.5, 1.0])
+
+            if item.food_type == "liquid":
+                # Sphere for liquids
+                shape_col = p.createCollisionShape(p.GEOM_SPHERE, radius=0.03)
+                shape_vis = p.createVisualShape(
+                    p.GEOM_SPHERE, radius=0.03, rgbaColor=color
+                )
+            elif item.fragility > 0.6:
+                # Flat disc for fragile items (papad, chapati)
+                shape_col = p.createCollisionShape(
+                    p.GEOM_CYLINDER, radius=0.04, height=0.01
+                )
+                shape_vis = p.createVisualShape(
+                    p.GEOM_CYLINDER,
+                    radius=0.04,
+                    length=0.01,
+                    rgbaColor=color,
+                )
+            else:
+                # Cube for solid foods
+                sz = 0.025
+                shape_col = p.createCollisionShape(
+                    p.GEOM_BOX, halfExtents=[sz, sz, sz]
+                )
+                shape_vis = p.createVisualShape(
+                    p.GEOM_BOX, halfExtents=[sz, sz, sz], rgbaColor=color
+                )
+
+            body_id = p.createMultiBody(
+                baseMass=0.1,
+                baseCollisionShapeIndex=shape_col,
+                baseVisualShapeIndex=shape_vis,
+                basePosition=[
+                    item.position[0],
+                    item.position[1],
+                    item.position[2] + 0.03,
+                ],
+            )
+            self._food_ids[item.id] = body_id
+
+        # --- Containers (open-top boxes) ---
+        for container in containers:
+            color = CONTAINER_COLORS.get(
+                container.container_type, [0.5, 0.5, 0.5, 0.7]
+            )
+            # Scale container size based on capacity
+            scale = (container.capacity_ml / 300) ** 0.33
+            w, d, h = 0.05 * scale, 0.05 * scale, 0.06 * scale
+
+            # Bottom
+            bottom_col = p.createCollisionShape(
+                p.GEOM_BOX, halfExtents=[w, d, 0.002]
+            )
+            bottom_vis = p.createVisualShape(
+                p.GEOM_BOX, halfExtents=[w, d, 0.002], rgbaColor=color
+            )
+            cx, cy, cz = container.position
+            body_id = p.createMultiBody(
+                baseMass=0,
+                baseCollisionShapeIndex=bottom_col,
+                baseVisualShapeIndex=bottom_vis,
+                basePosition=[cx, cy, cz],
+            )
+            self._container_ids[container.id] = body_id
+
+            # Walls (4 sides)
+            wall_thickness = 0.003
+            walls = [
+                ([w, wall_thickness, h / 2], [cx, cy + d, cz + h / 2]),
+                ([w, wall_thickness, h / 2], [cx, cy - d, cz + h / 2]),
+                ([wall_thickness, d, h / 2], [cx + w, cy, cz + h / 2]),
+                ([wall_thickness, d, h / 2], [cx - w, cy, cz + h / 2]),
+            ]
+            for wall_ext, wall_pos in walls:
+                wall_col = p.createCollisionShape(
+                    p.GEOM_BOX, halfExtents=wall_ext
+                )
+                wall_vis = p.createVisualShape(
+                    p.GEOM_BOX, halfExtents=wall_ext, rgbaColor=color
+                )
+                p.createMultiBody(
+                    baseMass=0,
+                    baseCollisionShapeIndex=wall_col,
+                    baseVisualShapeIndex=wall_vis,
+                    basePosition=wall_pos,
+                )
+
+        # Set up camera
+        p.resetDebugVisualizerCamera(
+            cameraDistance=1.2,
+            cameraYaw=45,
+            cameraPitch=-30,
+            cameraTargetPosition=[0, 0, 0.6],
+        )
+
+    def render(
+        self,
+        width: int = 640,
+        height: int = 480,
+        camera_distance: float = 1.2,
+        camera_yaw: float = 45,
+        camera_pitch: float = -30,
+    ) -> np.ndarray:
+        """
+        Render the current scene as an RGB image.
+
+        Returns:
+            numpy array of shape (height, width, 3) with RGB values.
+        """
+        if not self._initialized:
+            raise RuntimeError("Renderer not initialized. Call setup_scene() first.")
+
+        view_matrix = p.computeViewMatrixFromYawPitchRoll(
+            cameraTargetPosition=[0, 0, 0.6],
+            distance=camera_distance,
+            yaw=camera_yaw,
+            pitch=camera_pitch,
+            roll=0,
+            upAxisIndex=2,
+        )
+        proj_matrix = p.computeProjectionMatrixFOV(
+            fov=60,
+            aspect=width / height,
+            nearVal=0.1,
+            farVal=3.0,
+        )
+
+        _, _, rgba, _, _ = p.getCameraImage(
+            width=width,
+            height=height,
+            viewMatrix=view_matrix,
+            projectionMatrix=proj_matrix,
+            renderer=p.ER_TINY_RENDERER,
+        )
+
+        rgb = np.array(rgba, dtype=np.uint8).reshape(height, width, 4)[:, :, :3]
+        return rgb
+
+    def render_base64(self, **kwargs) -> str:
+        """Render scene and return as base64-encoded PNG string."""
+        rgb = self.render(**kwargs)
+
+        from PIL import Image
+
+        img = Image.fromarray(rgb)
+        buffer = io.BytesIO()
+        img.save(buffer, format="PNG")
+        return base64.b64encode(buffer.getvalue()).decode("utf-8")
+
+    def move_food_to_container(self, food_item_id: int, container_id: int):
+        """Visually move a food item into a container (for animation)."""
+        if food_item_id not in self._food_ids or container_id not in self._container_ids:
+            return
+
+        food_body = self._food_ids[food_item_id]
+        container_body = self._container_ids[container_id]
+
+        # Get container position
+        pos, _ = p.getBasePositionAndOrientation(container_body)
+        # Place food slightly above container center
+        new_pos = [pos[0], pos[1], pos[2] + 0.05]
+        p.resetBasePositionAndOrientation(
+            food_body, new_pos, [0, 0, 0, 1]
+        )
+
+    def close(self):
+        """Disconnect from PyBullet."""
+        if self._initialized:
+            p.disconnect(self._physics_client)
+            self._initialized = False
+
+    def _clear_objects(self):
+        """Remove all food and container objects."""
+        for body_id in self._food_ids.values():
+            try:
+                p.removeBody(body_id)
+            except Exception:
+                pass
+        for body_id in self._container_ids.values():
+            try:
+                p.removeBody(body_id)
+            except Exception:
+                pass
+        self._food_ids.clear()
+        self._container_ids.clear()
+
+    def __del__(self):
+        self.close()

tiffin_packer/tasks.py

@@ -0,0 +1,226 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+Task Definitions — Easy, Medium, Hard difficulty levels.
+
+Each task defines what food items are on the table, what containers are
+available, what constraints are active, and how many steps the agent gets.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import List, Optional
+
+from .simulation.engine import Container, FoodItem
+from .vlm.classifier import FoodClassifier
+
+
+@dataclass
+class TaskConfig:
+    """Configuration for a single task."""
+
+    task_id: str
+    description: str
+    food_items: List[FoodItem]
+    containers: List[Container]
+    constraints: List[str]
+    max_steps: int
+    seed: Optional[int] = None
+
+
+_vlm = FoodClassifier()
+
+
+def _make_food(id: int, name: str) -> FoodItem:
+    """Create a FoodItem from the VLM database."""
+    attrs = _vlm.classify(name)
+    return FoodItem(
+        id=id,
+        name=name,
+        food_type=attrs["type"],
+        volume_ml=attrs["volume_ml"],
+        temperature=attrs["temperature"],
+        fragility=attrs["fragility"],
+        preferred_container=attrs["preferred_container"],
+        color=attrs.get("color", "unknown"),
+        special_notes=attrs.get("special_notes", ""),
+    )
+
+
+def get_task_config(task_id: str, seed: Optional[int] = None) -> TaskConfig:
+    """Get task configuration by ID."""
+    tasks = {
+        "easy": _task_easy,
+        "medium": _task_medium,
+        "hard": _task_hard,
+    }
+    if task_id not in tasks:
+        raise ValueError(
+            f"Unknown task_id '{task_id}'. Available: {list(tasks.keys())}"
+        )
+    config = tasks[task_id](seed)
+    return config
+
+
+def _task_easy(seed: Optional[int] = None) -> TaskConfig:
+    """
+    Task 1 — Basic Packing (Easy)
+    
+    2 food items, 2 containers. Just match food type to container type.
+    Rice (solid) → open/deep container, Sambar (liquid) → sealed container.
+    """
+    return TaskConfig(
+        task_id="easy",
+        description=(
+            "Basic Packing: You have 2 food items (rice and sambar) and "
+            "2 containers (one sealed, one open). Place each food item in "
+            "a compatible container. Liquids must go in sealed containers."
+        ),
+        food_items=[
+            _make_food(1, "rice"),
+            _make_food(2, "sambar"),
+        ],
+        containers=[
+            Container(
+                id=1,
+                name="Sealed Round Container",
+                container_type="sealed_round",
+                capacity_ml=300,
+            ),
+            Container(
+                id=2,
+                name="Flat Open Container",
+                container_type="flat_open",
+                capacity_ml=400,
+            ),
+        ],
+        constraints=["type_match"],
+        max_steps=12,
+        seed=seed,
+    )
+
+
+def _task_medium(seed: Optional[int] = None) -> TaskConfig:
+    """
+    Task 2 — Efficient Packing (Medium)
+    
+    4 food items, 3 containers. Must match types AND avoid overflow.
+    Hot/cold separation matters.
+    """
+    return TaskConfig(
+        task_id="medium",
+        description=(
+            "Efficient Packing: You have 4 food items (rice, sambar, chapati, "
+            "pickle) and 3 containers. Place each item correctly:\n"
+            "- Match food type to container type (liquids → sealed)\n"
+            "- Don't overflow containers (check volumes!)\n"
+            "- Keep hot and cold items separate"
+        ),
+        food_items=[
+            _make_food(1, "rice"),
+            _make_food(2, "sambar"),
+            _make_food(3, "chapati"),
+            _make_food(4, "pickle"),
+        ],
+        containers=[
+            Container(
+                id=1,
+                name="Sealed Round Container",
+                container_type="sealed_round",
+                capacity_ml=200,
+            ),
+            Container(
+                id=2,
+                name="Flat Open Container",
+                container_type="flat_open",
+                capacity_ml=300,
+            ),
+            Container(
+                id=3,
+                name="Deep Box Container",
+                container_type="deep_box",
+                capacity_ml=350,
+            ),
+        ],
+        constraints=["type_match", "no_overflow", "temperature_separation"],
+        max_steps=20,
+        seed=seed,
+    )
+
+
+def _task_hard(seed: Optional[int] = None) -> TaskConfig:
+    """
+    Task 3 — Smart Packing (Hard)
+    
+    6 food items, 4 containers. Full constraint set:
+    type match, overflow, temperature, fragility, flavor mixing.
+    
+    Key challenges:
+    - Curd (cold) ≠ hot items in same container
+    - Papad (fragility=0.9) must not be crushed
+    - Curry + sambar both liquid+hot → total 300ml but sealed_round only 250ml!
+    - Must split liquids across containers
+    """
+    return TaskConfig(
+        task_id="hard",
+        description=(
+            "Smart Packing: You have 6 food items and 4 containers. This is a "
+            "complex meal with many constraints:\n"
+            "- Match food type to container type\n"
+            "- Don't overflow (watch the math!)\n"
+            "- Separate hot and cold items\n"
+            "- Don't crush fragile items (papad, chapati)\n"
+            "- Consider flavor isolation (pickle, chutney)\n"
+            "\nItems: rice, sambar, curd, chapati, papad, curry\n"
+            "Containers: sealed_round (250ml), flat_open (200ml), "
+            "deep_box (400ml), small_sealed (100ml)"
+        ),
+        food_items=[
+            _make_food(1, "rice"),
+            _make_food(2, "sambar"),
+            _make_food(3, "curd"),
+            _make_food(4, "chapati"),
+            _make_food(5, "papad"),
+            _make_food(6, "curry"),
+        ],
+        containers=[
+            Container(
+                id=1,
+                name="Sealed Round Container",
+                container_type="sealed_round",
+                capacity_ml=250,
+            ),
+            Container(
+                id=2,
+                name="Flat Open Container",
+                container_type="flat_open",
+                capacity_ml=200,
+            ),
+            Container(
+                id=3,
+                name="Deep Box Container",
+                container_type="deep_box",
+                capacity_ml=400,
+            ),
+            Container(
+                id=4,
+                name="Small Sealed Container",
+                container_type="small_sealed",
+                capacity_ml=100,
+            ),
+        ],
+        constraints=[
+            "type_match",
+            "no_overflow",
+            "temperature_separation",
+            "fragility_ordering",
+            "flavor_isolation",
+        ],
+        max_steps=30,
+        seed=seed,
+    )
+
+
+def list_tasks() -> List[str]:
+    """Return list of available task IDs."""
+    return ["easy", "medium", "hard"]

tiffin_packer/vlm/__init__.py

@@ -0,0 +1,3 @@
+from .classifier import FoodClassifier
+
+__all__ = ["FoodClassifier"]

tiffin_packer/vlm/classifier.py

@@ -0,0 +1,62 @@
+# Copyright (c) 2026 CtrlAltWin Team
+"""
+VLM Food Classifier — Simulates Vision-Language Model food classification.
+
+In production, this would call LLaVA / GPT-4V on a rendered PyBullet frame.
+For the hackathon, uses pre-computed attributes from food_db.json.
+
+The agent MUST call 'identify' before it knows a food item's properties.
+Without identification, items appear as generic "Unknown food item".
+"""
+
+import json
+import os
+from typing import Any, Dict, Optional
+
+
+class FoodClassifier:
+    """Cached VLM food classifier.
+
+    Loads pre-computed food attributes from food_db.json.
+    In a production system, replace ``classify()`` with a real
+    VLM API call (e.g. LLaVA, GPT-4V) on a rendered scene frame.
+    """
+
+    def __init__(self, db_path: Optional[str] = None):
+        if db_path is None:
+            db_path = os.path.join(os.path.dirname(__file__), "food_db.json")
+        with open(db_path, "r") as f:
+            self.food_db: Dict[str, Dict[str, Any]] = json.load(f)
+
+    def classify(self, food_name: str) -> Dict[str, Any]:
+        """Classify a food item and return its attributes.
+
+        Args:
+            food_name: Name of the food item (e.g. "sambar", "rice").
+
+        Returns:
+            Dict with keys: type, fragility, preferred_container,
+            volume_ml, temperature, color, special_notes.
+        """
+        key = food_name.lower().strip()
+        if key in self.food_db:
+            return {**self.food_db[key], "name": key, "classified": True}
+        return self._unknown_default(food_name)
+
+    def _unknown_default(self, food_name: str) -> Dict[str, Any]:
+        """Fallback for foods not in the database."""
+        return {
+            "name": food_name,
+            "type": "solid",
+            "fragility": 0.5,
+            "preferred_container": "deep",
+            "volume_ml": 100,
+            "temperature": "room",
+            "color": "unknown",
+            "special_notes": "Unknown food item — classification uncertain",
+            "classified": False,
+        }
+
+    def get_all_foods(self) -> list:
+        """Return list of all known food names."""
+        return list(self.food_db.keys())

tiffin_packer/vlm/food_db.json

@@ -0,0 +1,137 @@
+{
+  "rice": {
+    "type": "solid",
+    "fragility": 0.1,
+    "preferred_container": "deep",
+    "volume_ml": 200,
+    "temperature": "hot",
+    "color": "white",
+    "special_notes": "Staple grain, can be packed densely"
+  },
+  "sambar": {
+    "type": "liquid",
+    "fragility": 0.0,
+    "preferred_container": "sealed",
+    "volume_ml": 150,
+    "temperature": "hot",
+    "color": "orange",
+    "special_notes": "Lentil-based stew, will spill if container not sealed"
+  },
+  "curd": {
+    "type": "semi-solid",
+    "fragility": 0.3,
+    "preferred_container": "sealed",
+    "volume_ml": 100,
+    "temperature": "cold",
+    "color": "white",
+    "special_notes": "Dairy product, must be kept cold and away from hot items"
+  },
+  "chapati": {
+    "type": "solid",
+    "fragility": 0.7,
+    "preferred_container": "flat",
+    "volume_ml": 80,
+    "temperature": "room",
+    "color": "brown",
+    "special_notes": "Flatbread, fragile when stacked under heavy items"
+  },
+  "pickle": {
+    "type": "semi-solid",
+    "fragility": 0.2,
+    "preferred_container": "sealed",
+    "volume_ml": 30,
+    "temperature": "room",
+    "color": "red",
+    "special_notes": "Strong flavor, should not contaminate other items"
+  },
+  "dal": {
+    "type": "liquid",
+    "fragility": 0.0,
+    "preferred_container": "sealed",
+    "volume_ml": 120,
+    "temperature": "hot",
+    "color": "yellow",
+    "special_notes": "Lentil soup, needs sealed container"
+  },
+  "rasam": {
+    "type": "liquid",
+    "fragility": 0.0,
+    "preferred_container": "sealed",
+    "volume_ml": 100,
+    "temperature": "hot",
+    "color": "dark_red",
+    "special_notes": "Thin spicy soup, will leak easily"
+  },
+  "poriyal": {
+    "type": "solid",
+    "fragility": 0.5,
+    "preferred_container": "flat",
+    "volume_ml": 80,
+    "temperature": "hot",
+    "color": "green",
+    "special_notes": "Stir-fried vegetables, moderately fragile"
+  },
+  "papad": {
+    "type": "solid",
+    "fragility": 0.9,
+    "preferred_container": "flat",
+    "volume_ml": 20,
+    "temperature": "room",
+    "color": "golden",
+    "special_notes": "Very fragile crispy disc, breaks easily under pressure"
+  },
+  "raita": {
+    "type": "semi-solid",
+    "fragility": 0.2,
+    "preferred_container": "sealed",
+    "volume_ml": 80,
+    "temperature": "cold",
+    "color": "pale_green",
+    "special_notes": "Yogurt-based, must be kept cold"
+  },
+  "idli": {
+    "type": "solid",
+    "fragility": 0.4,
+    "preferred_container": "deep",
+    "volume_ml": 120,
+    "temperature": "hot",
+    "color": "white",
+    "special_notes": "Steamed rice cake, soft but holds shape"
+  },
+  "chutney": {
+    "type": "semi-solid",
+    "fragility": 0.1,
+    "preferred_container": "sealed",
+    "volume_ml": 50,
+    "temperature": "room",
+    "color": "green",
+    "special_notes": "Condiment, strong flavor, needs isolation"
+  },
+  "biryani": {
+    "type": "solid",
+    "fragility": 0.3,
+    "preferred_container": "deep",
+    "volume_ml": 250,
+    "temperature": "hot",
+    "color": "saffron",
+    "special_notes": "Fragrant rice dish, needs larger container"
+  },
+  "curry": {
+    "type": "liquid",
+    "fragility": 0.0,
+    "preferred_container": "sealed",
+    "volume_ml": 150,
+    "temperature": "hot",
+    "color": "brown",
+    "special_notes": "Gravy-based dish, will spill without sealed container"
+  },
+  "salad": {
+    "type": "solid",
+    "fragility": 0.6,
+    "preferred_container": "flat",
+    "volume_ml": 60,
+    "temperature": "cold",
+    "color": "mixed",
+    "special_notes": "Fresh vegetables, keep away from hot items"
+  }
+}