Phase 2 complete: Fixed HF API endpoint to router.huggingface.co and added environment setup

Downloads/hackathon/meta/.env

@@ -0,0 +1,16 @@
+# Environment Variables for Bug Report Structuring Inference
+# Fill in these values with your actual API credentials
+
+# LLM API Base URL - Hugging Face Router (updated endpoint)
+API_BASE_URL=https://router.huggingface.co/
+
+# Model identifier (must match your LLM provider)
+MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct
+
+# Your Hugging Face API Token
+# Get it from: https://huggingface.co/settings/tokens
+HF_TOKEN=hf_your_actual_token_here
+
+# Optional: Custom environment URL (default is provided)
+ENV_URL=https://rahul-13-bug-report-structuring-env.hf.space
+MUblGvkVFDVuWlXkVgfPulnfuDadWEGCWq

Downloads/hackathon/meta/.env.example

@@ -0,0 +1,22 @@
+# Environment Variables for Bug Report Structuring Inference
+# Copy this file to .env and fill in your actual values
+
+# LLM API Configuration
+# For Together AI, Hugging Face Inference, or vLLM
+API_BASE_URL=https://api.together.xyz/v1
+# or for local vLLM: http://localhost:8000/v1
+# or for HF Inference: https://api-inference.huggingface.co/v1
+
+# Model identifier
+MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct
+# Other options:
+# - meta-llama/Llama-3.2-70B-Instruct
+# - meta-llama/Llama-2-13b-chat
+# - mistralai/Mistral-7B-Instruct-v0.3
+
+# Hugging Face API Token
+# Get this from https://huggingface.co/settings/tokens
+HF_TOKEN=hf_your_token_here
+
+# Optional: Custom environment URL
+ENV_URL=https://rahul-13-bug-report-structuring-env.hf.space

Downloads/hackathon/meta/SETUP.md

@@ -0,0 +1,127 @@
+# 🔧 Fix: Missing Environment Variables Error
+
+## Problem
+Your `inference.py` is failing because it can't find these environment variables:
+- `API_BASE_URL` - URL to your LLM API
+- `MODEL_NAME` - Model identifier 
+- `HF_TOKEN` - Hugging Face authentication token
+
+## Solution
+
+### Step 1: Create/Edit the `.env` file
+A `.env` file has been created in your project root. Edit it with your actual credentials:
+
+```bash
+# Open .env in your editor and fill in:
+API_BASE_URL=https://api.together.xyz/v1     # Your LLM API endpoint
+MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct  # Model you want to use
+HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx            # Your HF token from https://huggingface.co/settings/tokens
+```
+
+### Step 2: Verify Your Setup
+Run the verification script to check if everything is configured:
+
+```bash
+python verify_env.py
+```
+
+Expected output:
+```
+✅ .env file found
+✅ API_BASE_URL: https://api.together.xyz/v1
+✅ MODEL_NAME: meta-llama/...
+✅ HF_TOKEN: hf_xxxx...xxxxx
+✅ All dependencies installed
+✅ All checks passed! Ready to run inference.py
+```
+
+### Step 3: Run Inference
+Once verification passes, run your inference:
+
+```bash
+python inference.py
+```
+
+## Environment Variable Options
+
+### API_BASE_URL (choose one):
+
+**Option 1: Together AI** (Recommended for hackathons)
+```
+API_BASE_URL=https://api.together.xyz/v1
+```
+- Free tier available
+- Sign up at: https://api.together.xyz
+
+**Option 2: Hugging Face Inference**
+```
+API_BASE_URL=https://api-inference.huggingface.co/v1
+```
+- Use your HF token for authentication
+- Limited free tier
+
+**Option 3: Local vLLM Server** (Advanced)
+```
+API_BASE_URL=http://localhost:8000/v1
+```
+- Requires running vLLM locally
+- Best for development
+
+### MODEL_NAME (examples):
+- `meta-llama/Llama-3.1-8B-Instruct` ✅ Recommended
+- `meta-llama/Llama-3.2-70B-Instruct` (more powerful)
+- `mistralai/Mistral-7B-Instruct-v0.3`
+- `NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO`
+
+### HF_TOKEN:
+1. Go to https://huggingface.co/settings/tokens
+2. Click "New token"
+3. Select "Read" permission
+4. Copy the token and paste in `.env`
+
+## Common Issues
+
+**❌ "curl: (6) Could not resolve host"**
+- Check your `API_BASE_URL` is correct and accessible
+- Test: `curl {API_BASE_URL}/models`
+
+**❌ "401 Unauthorized" or "invalid_api_key"**
+- Your `HF_TOKEN` is wrong or expired
+- Generate a new token at https://huggingface.co/settings/tokens
+
+**❌ "Model not found"**
+- Check your `MODEL_NAME` matches the provider's available models
+- Visit the provider's documentation
+
+## How It Works
+
+After the changes made to `inference.py`:
+1. The script automatically loads variables from `.env` file (if exists)
+2. Then it checks for required environment variables
+3. If any are missing, it shows a helpful error message
+4. If all are set, it proceeds with inference
+
+This means you can:
+- Use `.env` file (checked automatically)
+- Export variables manually: `export API_BASE_URL=...`
+- Mix both approaches (manual exports override `.env`)
+
+## Quick Commands (PowerShell)
+
+```powershell
+# Edit .env file
+code .env
+
+# Verify environment
+python verify_env.py
+
+# Run inference (after .env is filled in)
+python inference.py
+```
+
+## Next Steps for Phase 2
+
+1. ✅ Fill in `.env` with your credentials
+2. ✅ Run `python verify_env.py` to confirm setup
+3. ✅ Run `python inference.py` to complete phase 2
+4. ✅ Check `inference_results.txt` for logs an results

Downloads/hackathon/meta/inference.py

@@ -0,0 +1,373 @@
+#!/usr/bin/env python3
+"""
+Bug Report Structuring Environment - Inference Script
+
+This script runs the LLM agent against the Bug Report Structuring Environment.
+It connects to the deployed environment (HF Space), uses an LLM to structure
+messy bug reports, and logs results in the required OpenEnv format.
+
+Required environment variables:
+  API_BASE_URL  — Base URL for the LLM API (e.g., vLLM or HF Inference)
+  MODEL_NAME    — Model identifier (e.g., meta-llama/Llama-3.1-8B-Instruct)
+  HF_TOKEN      — Hugging Face authentication token
+
+Log format (STDOUT):
+  [START] task=<task> env=<env> model=<model>
+  [STEP]  step=<n> action=<summary> reward=<0.00> done=<bool> error=<msg|null>
+  [END]   success=<bool> steps=<n> score=<0.00> rewards=<r1,r2,...>
+"""
+
+import os
+import sys
+import json
+import time
+import requests
+from openai import OpenAI
+from pathlib import Path
+
+# ─── Load Environment Variables from .env if it exists ───────────
+env_file = Path(__file__).parent / ".env"
+if env_file.exists():
+    with open(env_file) as f:
+        for line in f:
+            line = line.strip()
+            if line and not line.startswith("#"):
+                key, _, value = line.partition("=")
+                key = key.strip()
+                value = value.strip()
+                if key and value:
+                    os.environ.setdefault(key, value)
+
+# ─── Configuration ────────────────────────────────────────────────
+
+API_BASE_URL = os.environ.get("API_BASE_URL", "")
+MODEL_NAME = os.environ.get("MODEL_NAME", "")
+HF_TOKEN = os.environ.get("HF_TOKEN", "")
+
+# Environment URL (the deployed HF Space)
+ENV_URL = os.environ.get(
+    "ENV_URL",
+    "https://rahul-13-bug-report-structuring-env.hf.space"
+)
+
+BENCHMARK_NAME = "bug_report_structuring"
+TASKS = ["easy", "medium", "hard"]
+MAX_RETRIES = 2
+
+# ─── LLM Client Setup ────────────────────────────────────────────
+
+client = OpenAI(
+    base_url=API_BASE_URL,
+    api_key=HF_TOKEN,
+)
+
+
+# ─── Prompt Templates ────────────────────────────────────────────
+
+SYSTEM_PROMPT = """You are an expert bug report analyst. Your job is to take messy, unstructured bug reports and convert them into well-organized, structured formats.
+
+You must output a valid JSON object with exactly these fields:
+- "title": A clear, concise title summarizing the bug
+- "steps_to_reproduce": Numbered step-by-step instructions to reproduce the bug
+- "expected_behavior": What should happen (correct behavior)
+- "actual_behavior": What actually happens (the bug symptoms)
+- "severity": One of "low", "medium", "high", or "critical"
+- "environment": OS, browser, version, platform details
+- "additional_notes": Any other relevant details
+
+Rules:
+1. Extract ALL information from the original report - don't miss details
+2. Use professional, clear language
+3. Steps should be specific and actionable
+4. Include version numbers, error messages, and technical details
+5. Severity should reflect the actual impact described
+6. Output ONLY the JSON object, no other text or markdown"""
+
+REFINEMENT_PROMPT = """You previously structured a bug report but the grading feedback indicates room for improvement.
+
+Original messy bug report:
+{raw_report}
+
+Your previous submission scored {score:.2f}/1.00.
+
+Feedback:
+{feedback}
+
+Previous field scores:
+{field_scores}
+
+Please submit an improved version. Focus on the fields with low scores.
+Output ONLY a valid JSON object with the same fields: title, steps_to_reproduce, expected_behavior, actual_behavior, severity, environment, additional_notes."""
+
+
+# ─── Helper Functions ─────────────────────────────────────────────
+
+def call_llm(messages: list) -> str:
+    """Call the LLM and return the response text."""
+    try:
+        response = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=messages,
+            temperature=0.3,
+            max_tokens=2048,
+        )
+        return response.choices[0].message.content.strip()
+    except Exception as e:
+        print(f"  [LLM ERROR] {e}", file=sys.stderr)
+        return ""
+
+
+def parse_json_response(text: str) -> dict:
+    """Parse JSON from LLM response, handling markdown code blocks."""
+    # Strip markdown code blocks if present
+    if "```json" in text:
+        text = text.split("```json")[1].split("```")[0].strip()
+    elif "```" in text:
+        text = text.split("```")[1].split("```")[0].strip()
+
+    try:
+        return json.loads(text)
+    except json.JSONDecodeError:
+        # Try to find JSON object in the text
+        start = text.find("{")
+        end = text.rfind("}") + 1
+        if start >= 0 and end > start:
+            try:
+                return json.loads(text[start:end])
+            except json.JSONDecodeError:
+                pass
+    return {}
+
+
+def env_reset(task_id: str) -> dict:
+    """Call the environment's reset endpoint."""
+    try:
+        resp = requests.post(
+            f"{ENV_URL}/reset",
+            json={"task_id": task_id},
+            timeout=30,
+        )
+        resp.raise_for_status()
+        return resp.json()
+    except Exception as e:
+        print(f"  [ENV ERROR] Reset failed: {e}", file=sys.stderr)
+        return {}
+
+
+def env_step(action: dict) -> dict:
+    """Call the environment's step endpoint."""
+    try:
+        resp = requests.post(
+            f"{ENV_URL}/step",
+            json={"action": action},
+            timeout=30,
+        )
+        resp.raise_for_status()
+        return resp.json()
+    except Exception as e:
+        print(f"  [ENV ERROR] Step failed: {e}", file=sys.stderr)
+        return {}
+
+
+def make_default_action() -> dict:
+    """Return a minimal valid action as fallback."""
+    return {
+        "title": "Bug Report",
+        "steps_to_reproduce": "1. See the bug report",
+        "expected_behavior": "Application works correctly",
+        "actual_behavior": "Application does not work as expected",
+        "severity": "medium",
+        "environment": "Not specified",
+        "additional_notes": "",
+    }
+
+
+# ─── Main Inference Loop ─────────────────────────────────────────
+
+def run_task(task_id: str) -> dict:
+    """
+    Run the agent on a single task.
+
+    Returns dict with: success, steps, score, rewards
+    """
+    # ── START ──
+    print(f"[START] task={task_id} env={BENCHMARK_NAME} model={MODEL_NAME}")
+
+    rewards = []
+    best_score = 0.0
+    step_count = 0
+    success = False
+
+    # Reset environment
+    obs = env_reset(task_id)
+    if not obs:
+        print(f"[STEP] step=1 action=reset_failed reward=0.00 done=true error=environment_reset_failed")
+        print(f"[END] success=false steps=1 score=0.00 rewards=0.00")
+        return {"success": False, "steps": 1, "score": 0.0, "rewards": [0.0]}
+
+    raw_report = obs.get("raw_report", "")
+    max_steps = obs.get("max_steps", 3)
+
+    # ── First submission ──
+    messages = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": f"Structure this bug report:\n\n{raw_report}"},
+    ]
+
+    llm_response = call_llm(messages)
+    action = parse_json_response(llm_response)
+
+    if not action or "title" not in action:
+        action = make_default_action()
+
+    # Ensure all fields exist
+    for field in ["title", "steps_to_reproduce", "expected_behavior",
+                  "actual_behavior", "severity", "environment", "additional_notes"]:
+        if field not in action:
+            action[field] = ""
+
+    step_count = 1
+    result = env_step(action)
+
+    if result:
+        score = result.get("score", 0.0)
+        reward = result.get("reward", 0.0)
+        done = result.get("done", False)
+        error = "null"
+    else:
+        score = 0.0
+        reward = 0.0
+        done = True
+        error = "step_request_failed"
+
+    rewards.append(reward)
+    best_score = max(best_score, score)
+    action_summary = action.get("title", "structured_report")[:50].replace(" ", "_")
+
+    print(
+        f"[STEP] step={step_count} action={action_summary} "
+        f"reward={reward:.2f} done={str(done).lower()} error={error}"
+    )
+
+    # ── Refinement steps ──
+    while not done and step_count < max_steps:
+        feedback = result.get("feedback", "")
+        field_scores = result.get("field_scores", {})
+
+        refinement_content = REFINEMENT_PROMPT.format(
+            raw_report=raw_report,
+            score=score,
+            feedback=feedback,
+            field_scores=json.dumps(field_scores, indent=2),
+        )
+
+        messages = [
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": refinement_content},
+        ]
+
+        llm_response = call_llm(messages)
+        action = parse_json_response(llm_response)
+
+        if not action or "title" not in action:
+            action = make_default_action()
+
+        for field in ["title", "steps_to_reproduce", "expected_behavior",
+                      "actual_behavior", "severity", "environment", "additional_notes"]:
+            if field not in action:
+                action[field] = ""
+
+        step_count += 1
+        result = env_step(action)
+
+        if result:
+            score = result.get("score", 0.0)
+            reward = result.get("reward", 0.0)
+            done = result.get("done", False)
+            error = "null"
+        else:
+            score = 0.0
+            reward = 0.0
+            done = True
+            error = "step_request_failed"
+
+        rewards.append(reward)
+        best_score = max(best_score, score)
+        action_summary = action.get("title", "refined_report")[:50].replace(" ", "_")
+
+        print(
+            f"[STEP] step={step_count} action={action_summary} "
+            f"reward={reward:.2f} done={str(done).lower()} error={error}"
+        )
+
+    # ── END ──
+    success = best_score >= 0.6
+    rewards_str = ",".join(f"{r:.2f}" for r in rewards)
+
+    print(
+        f"[END] success={str(success).lower()} steps={step_count} "
+        f"score={best_score:.2f} rewards={rewards_str}"
+    )
+
+    return {
+        "success": success,
+        "steps": step_count,
+        "score": best_score,
+        "rewards": rewards,
+    }
+
+
+def main():
+    """Run inference on all tasks."""
+    # Validate environment variables
+    missing = []
+    if not API_BASE_URL:
+        missing.append("API_BASE_URL")
+    if not MODEL_NAME:
+        missing.append("MODEL_NAME")
+    if not HF_TOKEN:
+        missing.append("HF_TOKEN")
+
+    if missing:
+        print(f"❌ Missing environment variables: {', '.join(missing)}", file=sys.stderr)
+        print("Set them before running:", file=sys.stderr)
+        print("  export API_BASE_URL=https://...", file=sys.stderr)
+        print("  export MODEL_NAME=meta-llama/...", file=sys.stderr)
+        print("  export HF_TOKEN=hf_...", file=sys.stderr)
+        sys.exit(1)
+
+    print(f"═══ Bug Report Structuring - Inference ═══", file=sys.stderr)
+    print(f"  Model: {MODEL_NAME}", file=sys.stderr)
+    print(f"  Env:   {ENV_URL}", file=sys.stderr)
+    print(f"  Tasks: {TASKS}", file=sys.stderr)
+    print(f"═══════════════════════════════════════════", file=sys.stderr)
+
+    results = {}
+    total_score = 0.0
+    start_time = time.time()
+
+    for task_id in TASKS:
+        print(f"\n--- Task: {task_id} ---", file=sys.stderr)
+        result = run_task(task_id)
+        results[task_id] = result
+        total_score += result["score"]
+        print(f"  Score: {result['score']:.2f}", file=sys.stderr)
+
+    elapsed = time.time() - start_time
+    avg_score = total_score / len(TASKS)
+
+    print(f"\n═══ Summary ═══", file=sys.stderr)
+    print(f"  Average Score: {avg_score:.2f}", file=sys.stderr)
+    print(f"  Time Elapsed:  {elapsed:.1f}s", file=sys.stderr)
+    for task_id, result in results.items():
+        status = "✅" if result["success"] else "❌"
+        print(
+            f"  {status} {task_id}: {result['score']:.2f} "
+            f"({result['steps']} steps)",
+            file=sys.stderr,
+        )
+    print(f"═══════════════", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

Downloads/hackathon/meta/pyproject.toml

@@ -24,11 +24,12 @@ classifiers = [
 ]
 
 dependencies = [
-    "fastapi==0.115.6",
-    "uvicorn==0.34.0",
-    "pydantic==2.10.4",
-    "requests==2.32.3",
-    "openai==1.58.1",
+    "fastapi>=0.115.0",
+    "uvicorn>=0.34.0",
+    "pydantic>=2.10.0",
+    "requests>=2.32.0",
+    "openai>=1.58.0",
+    "openenv-core>=0.2.0",
 ]
 
 [project.optional-dependencies]
@@ -37,12 +38,15 @@ dev = [
     "pytest-cov>=3.0",
 ]
 
+[project.scripts]
+server = "server.app:main"
+
 [project.urls]
 Repository = "https://github.com/SAI-RAHUL-ROKKAM/meta_hack"
 Documentation = "https://huggingface.co/spaces/RAHUL-13/bug-report-structuring-env"
 
 [tool.setuptools]
-packages = ["src"]
+packages = ["server"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

Downloads/hackathon/meta/verify_env.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""
+Verify that the environment is properly configured for running inference.
+Run this script before running inference.py to catch configuration issues early.
+"""
+
+import os
+import sys
+from pathlib import Path
+
+def check_env_file():
+    """Check if .env file exists and has required variables."""
+    env_file = Path(__file__).parent / ".env"
+    
+    if not env_file.exists():
+        print("❌ .env file not found")
+        print(f"   Create it at: {env_file}")
+        return False
+    
+    print("✅ .env file found")
+    return True
+
+def check_env_variables():
+    """Check if required environment variables are set."""
+    required_vars = ["API_BASE_URL", "MODEL_NAME", "HF_TOKEN"]
+    missing = []
+    
+    for var in required_vars:
+        value = os.environ.get(var, "").strip()
+        if not value:
+            missing.append(var)
+            print(f"❌ {var}: NOT SET")
+        else:
+            # Show masked value for security
+            masked = f"{value[:10]}...{value[-5:]}" if len(value) > 20 else value
+            print(f"✅ {var}: {masked}")
+    
+    return len(missing) == 0
+
+def load_env_file():
+    """Load .env file into environment."""
+    env_file = Path(__file__).parent / ".env"
+    if env_file.exists():
+        with open(env_file) as f:
+            for line in f:
+                line = line.strip()
+                if line and not line.startswith("#"):
+                    key, _, value = line.partition("=")
+                    key = key.strip()
+                    value = value.strip()
+                    if key and value:
+                        os.environ[key] = value
+
+def check_dependencies():
+    """Check if required packages are installed."""
+    print("\nChecking dependencies...")
+    required = ["openai", "requests", "fastapi", "uvicorn", "pydantic"]
+    missing = []
+    
+    for package in required:
+        try:
+            __import__(package)
+            print(f"✅ {package}: installed")
+        except ImportError:
+            missing.append(package)
+            print(f"❌ {package}: NOT INSTALLED")
+    
+    return len(missing) == 0
+
+def main():
+    print("═══ Environment Verification ═══\n")
+    
+    # Load .env file first
+    load_env_file()
+    
+    # Check .env file
+    has_env_file = check_env_file()
+    
+    print("\nChecking environment variables...")
+    has_all_vars = check_env_variables()
+    
+    # Check dependencies
+    has_deps = check_dependencies()
+    
+    print("\n" + "═" * 35)
+    
+    if has_all_vars and has_deps:
+        print("✅ All checks passed! Ready to run inference.py")
+        return 0
+    else:
+        print("❌ Some checks failed. Please fix the issues above.")
+        print("\nQuick fix:")
+        print("1. Edit .env file with your actual credentials")
+        print("2. Run: pip install -r requirements.txt")
+        return 1
+
+if __name__ == "__main__":
+    sys.exit(main())