Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/Architecture_diagram.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,255 +1,183 @@
----
-title: Rag Optimizer Environment Server
-emoji: 📸
-colorFrom: blue
-colorTo: gray
-sdk: docker
-pinned: false
-app_port: 8000
-base_path: /web
-tags:
-  - openenv
----
-
-# Rag Optimizer Environment
-
-A simple test environment that echoes back messages. Perfect for testing the env APIs as well as demonstrating environment usage patterns.
-
-## Quick Start
-
-The simplest way to use the Rag Optimizer environment is through the `RagOptimizerEnv` class:
-
-```python
-from rag_optimizer import RagOptimizerAction, RagOptimizerEnv
-
-try:
-    # Create environment from Docker image
-    rag_optimizerenv = RagOptimizerEnv.from_docker_image("rag_optimizer-env:latest")
-
-    # Reset
-    result = rag_optimizerenv.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-
-    # Send multiple messages
-    messages = ["Hello, World!", "Testing echo", "Final message"]
-
-    for msg in messages:
-        result = rag_optimizerenv.step(RagOptimizerAction(message=msg))
-        print(f"Sent: '{msg}'")
-        print(f"  → Echoed: '{result.observation.echoed_message}'")
-        print(f"  → Length: {result.observation.message_length}")
-        print(f"  → Reward: {result.reward}")
-
-finally:
-    # Always clean up
-    rag_optimizerenv.close()
-```
-
-That's it! The `RagOptimizerEnv.from_docker_image()` method handles:
-- Starting the Docker container
-- Waiting for the server to be ready
-- Connecting to the environment
-- Container cleanup when you call `close()`
-
-## Building the Docker Image
-
-Before using the environment, you need to build the Docker image:
-
-```bash
-# From project root
-docker build -t rag_optimizer-env:latest -f server/Dockerfile .
-```
-
-## Deploying to Hugging Face Spaces
-
-You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command:
-
-```bash
-# From the environment directory (where openenv.yaml is located)
-openenv push
-
-# Or specify options
-openenv push --namespace my-org --private
-```
-
-The `openenv push` command will:
-1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`)
-2. Prepare a custom build for Hugging Face Docker space (enables web interface)
-3. Upload to Hugging Face (ensuring you're logged in)
-
-### Prerequisites
-
-- Authenticate with Hugging Face: The command will prompt for login if not already authenticated
-
-### Options
-
-- `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory)
-- `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
-- `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM)
-- `--private`: Deploy the space as private (default: public)
-
-### Examples
-
-```bash
-# Push to your personal namespace (defaults to username/env-name from openenv.yaml)
-openenv push
-
-# Push to a specific repository
-openenv push --repo-id my-org/my-env
-
-# Push with a custom base image
-openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest
-
-# Push as a private space
-openenv push --private
-
-# Combine options
-openenv push --repo-id my-org/my-env --base-image custom-base:latest --private
-```
-
-After deployment, your space will be available at:
-`https://huggingface.co/spaces/<repo-id>`
-
-The deployed space includes:
-- **Web Interface** at `/web` - Interactive UI for exploring the environment
-- **API Documentation** at `/docs` - Full OpenAPI/Swagger interface
-- **Health Check** at `/health` - Container health monitoring
-- **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions
-
-## Environment Details
-
-### Action
-**RagOptimizerAction**: Contains a single field
-- `message` (str) - The message to echo back
-
-### Observation
-**RagOptimizerObservation**: Contains the echo response and metadata
-- `echoed_message` (str) - The message echoed back
-- `message_length` (int) - Length of the message
-- `reward` (float) - Reward based on message length (length × 0.1)
-- `done` (bool) - Always False for echo environment
-- `metadata` (dict) - Additional info like step count
-
-### Reward
-The reward is calculated as: `message_length × 0.1`
-- "Hi" → reward: 0.2
-- "Hello, World!" → reward: 1.3
-- Empty message → reward: 0.0
-
-## Advanced Usage
-
-### Connecting to an Existing Server
-
-If you already have a Rag Optimizer environment server running, you can connect directly:
-
-```python
-from rag_optimizer import RagOptimizerEnv
-
-# Connect to existing server
-rag_optimizerenv = RagOptimizerEnv(base_url="<ENV_HTTP_URL_HERE>")
-
-# Use as normal
-result = rag_optimizerenv.reset()
-result = rag_optimizerenv.step(RagOptimizerAction(message="Hello!"))
-```
-
-Note: When connecting to an existing server, `rag_optimizerenv.close()` will NOT stop the server.
-
-### Using the Context Manager
-
-The client supports context manager usage for automatic connection management:
-
-```python
-from rag_optimizer import RagOptimizerAction, RagOptimizerEnv
-
-# Connect with context manager (auto-connects and closes)
-with RagOptimizerEnv(base_url="http://localhost:8000") as env:
-    result = env.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-    # Multiple steps with low latency
-    for msg in ["Hello", "World", "!"]:
-        result = env.step(RagOptimizerAction(message=msg))
-        print(f"Echoed: {result.observation.echoed_message}")
-```
-
-The client uses WebSocket connections for:
-- **Lower latency**: No HTTP connection overhead per request
-- **Persistent session**: Server maintains your environment state
-- **Efficient for episodes**: Better for many sequential steps
-
-### Concurrent WebSocket Sessions
-
-The server supports multiple concurrent WebSocket connections. To enable this,
-modify `server/app.py` to use factory mode:
-
-```python
-# In server/app.py - use factory mode for concurrent sessions
-app = create_app(
-    RagOptimizerEnvironment,  # Pass class, not instance
-    RagOptimizerAction,
-    RagOptimizerObservation,
-    max_concurrent_envs=4,  # Allow 4 concurrent sessions
-)
-```
-
-Then multiple clients can connect simultaneously:
-
-```python
-from rag_optimizer import RagOptimizerAction, RagOptimizerEnv
-from concurrent.futures import ThreadPoolExecutor
-
-def run_episode(client_id: int):
-    with RagOptimizerEnv(base_url="http://localhost:8000") as env:
-        result = env.reset()
-        for i in range(10):
-            result = env.step(RagOptimizerAction(message=f"Client {client_id}, step {i}"))
-        return client_id, result.observation.message_length
-
-# Run 4 episodes concurrently
-with ThreadPoolExecutor(max_workers=4) as executor:
-    results = list(executor.map(run_episode, range(4)))
-```
-
-## Development & Testing
-
-### Direct Environment Testing
-
-Test the environment logic directly without starting the HTTP server:
-
-```bash
-# From the server directory
-python3 server/rag_optimizer_environment.py
-```
-
-This verifies that:
-- Environment resets correctly
-- Step executes actions properly
-- State tracking works
-- Rewards are calculated correctly
-
-### Running Locally
-
-Run the server locally for development:
-
-```bash
-uvicorn server.app:app --reload
-```
-
-## Project Structure
-
-```
-rag_optimizer/
-├── .dockerignore         # Docker build exclusions
-├── __init__.py            # Module exports
-├── README.md              # This file
-├── openenv.yaml           # OpenEnv manifest
-├── pyproject.toml         # Project metadata and dependencies
-├── uv.lock                # Locked dependencies (generated)
-├── client.py              # RagOptimizerEnv client
-├── models.py              # Action and Observation models
-└── server/
-    ├── __init__.py        # Server module exports
-    ├── rag_optimizer_environment.py  # Core environment logic
-    ├── app.py             # FastAPI application (HTTP + WebSocket endpoints)
-    └── Dockerfile         # Container image definition
-```
+---
+title: Rag Optimizer Env
+emoji: 🧹
+colorFrom: green
+colorTo: gray
+sdk: docker
+app_port: 8000
+pinned: false
+license: mit
+tags:
+  - reinforcement-learning
+  - data-engineering
+  - rag
+  - openenv
+  - llm-agent
+base_path: /web
+---
+<br>
+
+<p align="center">
+  <img src="https://img.shields.io/badge/Hugging%20Face-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black"/>
+  <img src="https://img.shields.io/badge/HF%20Spaces-FFBF00?style=for-the-badge&logo=huggingface&logoColor=black"/>
+  <img src="https://img.shields.io/badge/FastAPI-009688?style=for-the-badge&logo=fastapi&logoColor=white"/>
+  <img src="https://img.shields.io/badge/Docker-2496ED?style=for-the-badge&logo=docker&logoColor=white"/>
+  <img src="https://img.shields.io/badge/OpenEnv-4B5563?style=for-the-badge&logo=envato&logoColor=white"/>
+  <img src="https://img.shields.io/badge/scikit--learn-F7931E?style=for-the-badge&logo=scikit-learn&logoColor=white"/>
+</p>
+<br>
+
+<h1 align="center">RagOptimizerEnv</h1>
+<p align="center">
+  A high-fidelity Reinforcement Learning environment modeling the complex dynamics of enterprise Knowledge Base curation. LLM-driven agents must iteratively optimize, chunk, and groom raw document structures to natively improve vector-search retrieval accuracy over a continuous continuous state space.
+  <br />
+</p>
+
+<br>
+
+## System Architecture & Theme
+
+**DATA ENGINEERING & AI INFRASTRUCTURE**
+
+RagOptimizerEnv simulates the critical and computationally intensive role of an AI Data Engineer. The environment forces an autonomous agent to resolve conflicting semantic documentation, heuristically categorize disjointed metadata, and structurally splinter monolithic text corpora to prevent embedding decay within a Retrieval-Augmented Generation (RAG) pipeline.
+
+<br>
+<p align="center">
+  <img src="assets/Architecture_diagram.png" width="800" alt="System Architecture Diagram" />
+</p>
+<br>
+
+## The Engineering Problem
+
+A pervasive engineering bottleneck in deployed RAG systems is **embedding decay caused by underlying data swamps**. A corpus containing overlapping legacy documentation, unstructured support tickets, and bloated monolithic manuals causes downstream deterministic embedding models to suffer from severe multi-collinearity and contextual wash-out.
+
+To resolve this, human data engineers must navigate the database, read dense architectural texts, deduce semantic boundaries, and execute targeted CRUD operations to optimize the topological search space. Simulating and automating this workflow poses a multi-hop, highly contextual reinforcement learning challenge that strictly tests an agent's reasoning bounds, context retention, and operational planning over long task trajectories.
+
+## The Proposed Solution
+
+We present an OpenEnv RL environment featuring an embedded, deterministic `scikit-learn` continuous Grader. The environment vectorizes the current database state after every single agent action, evaluating the structural integrity of the Knowledge Base in real-time. This provides a continuous reward density mapping—grading the agent purely on whether its structural operations inherently improved the theoretical Recall@3 limit of a downstream similarity search algorithm.
+
+---
+
+## Technical Novelty
+
+- **Continuous Deterministic Grader:** Rather than relying on sparse terminal rewards, the environment maintains a live embedded `TfidfVectorizer`. It continuously restructures its internal sparse embedding matrix on every action step, calculating the cosine-similarity of hidden semantic payloads to supply dynamic gradient signals.
+- **Topological State Feedback:** The environment streams a live topological representation of the knowledge base schema back to the agent on every interaction. Providing the `doc_id` indices mapped to character `length` and structural `metadata` ensures that agents operate efficiently without hallucinating system state across long inference contexts.
+- **Dynamic Noise Mitigation:** The target corpus is embedded within an array of generated distractor documents designed to mathematically dilute the TF-IDF search space, testing the robust operational precision of deployed LLM agents.
+- **Multi-Step Deductive Curriculum:** Sub-tasks require agents to temporarily retain the historical context of prior operations to successfully coordinate complex restructurings, pushing agents beyond single-shot function calling.
+
+---
+
+## Table of Contents
+
+1. [Environment Abstractions](#1-environment-abstractions)
+2. [Action & Observation Theory](#2-action--observation-theory)
+3. [The Grader & Reward Analytics](#3-the-grader--reward-analytics)
+4. [Curriculum Topologies](#4-curriculum-topologies)
+5. [Inference Baseline Sandbox](#5-inference-baseline-sandbox)
+6. [Containerized Deployment](#6-containerized-deployment)
+
+---
+
+## 1. Environment Abstractions
+
+RagOptimizerEnv is an **OpenEnv** benchmark environment. Built atop stable microservice interfaces (`/reset`, `/step`, `/state`), it natively conforms to robust distributed training paradigms for Policy Gradient reinforcement algorithms.
+
+**Complexity Vectors:**
+- The underlying corpus intentionally mirrors unstructured enterprise deployments. 
+- Actions compound natively. Wiping an incorrect document permanently alters the topological alignment of the search algorithm.
+- Frontier agents are challenged to execute a non-trivial "discover, contextualize, execute, verify" loop under a rigid step budget.
+
+---
+
+## 2. Action & Observation Theory
+
+### Action Abstraction (`RagOptimizerAction`)
+
+Agents manipulate the embedding search space via a strictly defined remote schema:
+
+| Action Primitive | Arguments | Effect Profile |
+|---|---|---|
+| `read_document` | `doc_id` | Fetches the specific byte-buffer into the context string without impacting reward density. |
+| `update_document` | `doc_id`, `text` | Hard-overwrites existant node structures, or instantiates a fresh target node to achieve semantic splintering. |
+| `delete_document` | `doc_id` | Destructively isolates and purges contradictory embeddings from the similarity array. |
+| `add_metadata` | `doc_id`, `metadata_key`, `metadata_value` | Injects structured ontological markers to deterministically force embedding activations. |
+| `submit` | None | Terminates the operational trajectory. |
+
+### Feedback Space (`RagOptimizerObservation`)
+
+Pydantic-typed JSON states returned immediately upon trajectory execution:
+1. `message`: Terminal I/O logs, error stack-traces, or `read_document` raw buffers.
+2. `current_docs`: A hierarchical mapping of the existing document index payload ensuring contextual grounding.
+3. `reward`: The live theoretical model convergence rate formulated between `0.0` and `1.0`.
+
+---
+
+## 3. The Grader & Reward Analytics
+
+Evaluation is robust, empirical, and mathematically bounded:
+
+1. **State Mutation:** The agent executes an array transaction within the topological space.
+2. **Re-Embedding:** The environment abstracts the updated schema into a standard English-corpus `TfidfVectorizer`.
+3. **Validation Probes:** The internal testing suite embeds an array of control queries targeting defined semantic concepts.
+4. **Retrieval Benchmark:** Sparse `cosine_similarity` calculates the topological displacement.
+5. **Score Allocation:** A hit is awarded conditionally if the modified Knowledge Base successfully forces the `target_concept` vector upward into the Top 3 search indices.
+
+**Reward Yield:** `(Successful Vectors / Total Vector Payload)` providing high-density intermediate signals mapping continuously toward the `±1.0` upper bound.
+
+---
+
+## 4. Curriculum Topologies
+
+The environment tests agents across three progressively demanding task distributions mirroring production RAG degradation scenarios.
+
+### Level I: Contamination Purging
+**The Vector Issue:** The base contains heavily overlapping parameters (competing versions of legacy and modern timeline protocols).
+**System Goal:** Autonomously survey the semantic differences, deduce the temporal conflict, and execute `delete_document` sweeps to purge vector hallucination triggers.
+
+### Level II: Ontological Tagging
+**The Vector Issue:** Textual segments representing technical telemetry are too dense and sparse on specific categorical keywords, resulting in low coordinate density for deterministic algorithms.
+**System Goal:** Execute inferential reading, deduce categorical bounds organically, and route exact programmatic `metadata` tags onto corresponding payloads to anchor the search vectors.
+
+### Level III: Syntactic Splintering (The Monolith)
+**The Vector Issue:** Extreme embedding decay caused by disparate conceptual structures compacted under a single referential document. This represents the well-known "PDF chunk wash-out" phenomenon.
+**System Goal:** Methodically `read` the extensive parent block, temporarily cache the semantic context limits, and utilize rapid consecutive `update_document` calls to mechanically splinter and redistribute the knowledge logic across multiple fine-grained nodes.
+
+---
+
+## 5. Inference Baseline Sandbox
+
+The root tree provides `inference.py`, a reference baseline executing an OpenAI-compliant autonomous data-engineering agent.
+
+Telemetry formats strictly adhere to high-velocity logging required for large-scale evaluation pipelines:
+
+```text
+[START] task=rag_optimizer_env env=OpenEnv model=meta-llama/Meta-Llama-3.1-8B-Instruct
+[STEP] step=1 action=read('doc_monolithic_onboarding') reward=0.33 done=false error=null
+[STEP] step=2 action=update('doc_vpn_policy') reward=0.50 done=false error=null
+...
+[END] success=true steps=12 score=1.00 rewards=0.33,0.50,... 
+```
+
+### Execution
+Ensure the environment contains an initialized `.env` matching standard model provider interfaces.
+```bash
+source env/bin/activate
+python inference.py
+```
+
+---
+
+## 6. Containerized Deployment
+
+This architecture packages seamlessly into distributed environments utilizing strict Docker integrations for Hugging Face Spaces compatibility.
+
+```bash
+# Compile SDK Instance
+docker build -f server/Dockerfile -t ragoptimizer:latest .
+
+# Execute API Server Loop
+docker run -p 8000:8000 ragoptimizer:latest
+```
+
+The primary instance binds over standard ASGI loops, broadcasting stable interfaces over standard HTTP routing (`/reset`, `/step`).

inference.py

@@ -3,14 +3,35 @@ import sys
 import json
 from openai import OpenAI
 
+
+def load_local_env() -> None:
+    """Lightweight .env loader for local runs without requiring python-dotenv."""
+    env_path = os.path.join(os.path.dirname(__file__), ".env")
+    if not os.path.exists(env_path):
+        return
+
+    with open(env_path, "r", encoding="utf-8") as f:
+        for raw_line in f:
+            line = raw_line.strip()
+            if not line or line.startswith("#") or "=" not in line:
+                continue
+            key, value = line.split("=", 1)
+            key = key.strip()
+            value = value.strip().strip('"').strip("'")
+            if key and key not in os.environ:
+                os.environ[key] = value
+
+
+load_local_env()
+
 # Add the envs module to path so we can import client and models
 sys.path.append(os.path.join(os.path.dirname(__file__), "envs", "rag_optimizer_env"))
 from client import RagOptimizerEnvClient
 from models import RagOptimizerAction
 
 # Load environment variables
-API_BASE_URL = os.getenv("API_BASE_URL")
-MODEL_NAME = os.getenv("MODEL_NAME")
+API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+MODEL_NAME = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-72B-Instruct")
 HF_TOKEN = os.getenv("HF_TOKEN")
 
 MAX_STEPS = 30

rag_optimizer_environment.py

@@ -0,0 +1,212 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""
+Rag Optimizer Environment Implementation.
+The agent acts as a Data Engineer to un-block a broken RAG pipeline.
+"""
+
+from uuid import uuid4
+from typing import Dict, Any, List
+
+from openenv.core.env_server.interfaces import Environment
+from openenv.core.env_server.types import State
+
+# Import scikit-learn for our Grader
+from sklearn.feature_extraction.text import TfidfVectorizer
+from sklearn.metrics.pairwise import cosine_similarity
+import numpy as np
+
+try:
+    from models import RagOptimizerAction, RagOptimizerObservation
+except ImportError:
+    from models import RagOptimizerAction, RagOptimizerObservation
+
+
+class RagOptimizerEnvironment(Environment):
+    """
+    RAG Optimizer Engine.
+    Maintains a simulated Knowledge Base and grades it using TF-IDF.
+    """
+
+    SUPPORTS_CONCURRENT_SESSIONS: bool = True
+
+    def __init__(self):
+        self._state = State(episode_id=str(uuid4()), step_count=0)
+        self.kb = self._get_initial_kb()
+        
+        # Hidden test suite for the grader
+        self.test_suite = [
+            {
+                "query": "What is the current 2024 price for standard?",
+                "target_concept": "750/mo"
+            },
+            {
+                "query": "What is the refund policy for enterprise?",
+                "target_concept": "Refunds are not permitted"
+            },
+            {
+                "query": "UI issues frontend CSS missing button",
+                "target_concept": "frontend team fixed the button"
+            },
+            {
+                "query": "How long does shipping take to branch offices?",
+                "target_concept": "5-7 business days"
+            },
+            {
+                "query": "What months do parking passes need to be renewed?",
+                "target_concept": "March"
+            },
+            {
+                "query": "What holidays are we off in 2024?",
+                "target_concept": "July 4"
+            }
+        ]
+
+    def _get_initial_kb(self) -> Dict[str, Dict]:
+        """Returns a fresh copy of the initial messy knowledge base."""
+        return {
+            "doc_pricing_legacy": {
+                "text": "Pricing for 2021: Enterprise tier is $1000/mo. Standard is $500/mo. All plans include 10 users.",
+                "metadata": {"type": "pricing"}
+            },
+            "doc_pricing_current_v2": {
+                "text": "Current Pricing 2024: Enterprise is $1500/mo. Standard is $750/mo. Refunds are not permitted on the enterprise tier.",
+                "metadata": {}
+            },
+            "doc_shipping_policy": {
+                "text": "All internal shipments to remote branch offices take 5-7 business days. Overnight shipping is only available for C-suite.",
+                "metadata": {"department": "logistics"}
+            },
+            "doc_messy_support_ticket_1": {
+                "text": "User complained the button disappeared on the frontend. Another user said the database latency was high. The frontend team fixed the button by updating CSS.",
+                "metadata": {}
+            },
+            "doc_messy_support_ticket_2": {
+                "text": "Email integration is failing with error 401 Unauthorized. The API key was rotated on Tuesday.",
+                "metadata": {}
+            },
+            "doc_monolithic_onboarding": {
+                "text": "Welcome to the company! Here are some rules. 1) VPN access requires DUO. 2) The cafetaria opens at 8 AM. 3) For HR issues, email hr@company.com. 4) The 2024 holiday schedule includes Dec 25, Jan 1, and July 4. 5) Parking passes must be renewed annually in March.",
+                "metadata": {}
+            },
+            # Add distractor files
+            **{f"doc_distractor_hr_{i}": {"text": f"This is an old HR policy document regarding {['pto', 'sick leave', 'travel', 'expenses'][i%4]} from 201{i%10}.", "metadata":{}} for i in range(10)},
+            **{f"doc_distractor_eng_{i}": {"text": f"Engineering architecture decision record {i}. We decided to use {['React', 'Postgres', 'Redis', 'Kafka'][i%4]} because of scaling concerns.", "metadata":{}} for i in range(10)},
+            **{f"doc_distractor_random_{i}": {"text": f"Weekly team update notes. Nothing important here, just discussed the weather and the upcoming launch {i}.", "metadata":{}} for i in range(10)},
+        }
+
+    def _get_kb_summary(self) -> Dict[str, Dict]:
+        """Returns a summary of the KB for the observation."""
+        summary = {}
+        for k, v in self.kb.items():
+            summary[k] = {"metadata": v.get("metadata", {}), "length": len(v.get("text", ""))}
+        return summary
+
+    def reset(self) -> RagOptimizerObservation:
+        self._state = State(episode_id=str(uuid4()), step_count=0)
+        self.kb = self._get_initial_kb()
+        return RagOptimizerObservation(
+            message="RagOptimizerEnv Initialized. You have messy chunks in the KB. Resolve conflicts, add metadata tags to short tickets, and splinter monolithic files to win.",
+            current_docs=self._get_kb_summary(),
+            done=False,
+            reward=self._evaluate_kb()
+        )
+
+    def _evaluate_kb(self) -> float:
+        """The Grader: Evaluates the agent's current KB using TF-IDF."""
+        if not self.kb:
+            return 0.0
+            
+        doc_texts = [doc["text"] for doc in self.kb.values()]
+        
+        vectorizer = TfidfVectorizer(stop_words='english')
+        try:
+            doc_vectors = vectorizer.fit_transform(doc_texts)
+        except ValueError:
+            return 0.0
+            
+        score = 0.0
+        
+        for case in self.test_suite:
+            query_vec = vectorizer.transform([case["query"]])
+            similarities = cosine_similarity(query_vec, doc_vectors)[0]
+            
+            # Get top 3
+            top_k_indices = similarities.argsort()[-3:][::-1]
+            
+            found = False
+            for idx in top_k_indices:
+                if similarities[idx] > 0.01:
+                    if case["target_concept"].lower() in doc_texts[idx].lower():
+                        found = True
+                        break
+            if found:
+                score += 1.0
+                
+        return float(score / len(self.test_suite))
+
+    def step(self, action: RagOptimizerAction) -> RagOptimizerObservation:  # type: ignore[override]
+        self._state.step_count += 1
+        
+        msg = ""
+        done = False
+        reward = 0.0
+        
+        try:
+            if action.action_type == "read_document":
+                if action.doc_id in self.kb:
+                    msg = f"Content of {action.doc_id}: {self.kb[action.doc_id]['text']}"
+                else:
+                    msg = f"Error: doc_id {action.doc_id} not found."
+            
+            elif action.action_type == "delete_document":
+                if action.doc_id in self.kb:
+                    del self.kb[action.doc_id]
+                    msg = f"Deleted {action.doc_id}."
+                else:
+                    msg = f"Error: doc_id {action.doc_id} not found."
+                    
+            elif action.action_type == "update_document":
+                if not action.doc_id or not action.text:
+                    msg = "Error: doc_id and text required for update_document."
+                else:
+                    if action.doc_id not in self.kb:
+                        self.kb[action.doc_id] = {"text": "", "metadata": {}}
+                    self.kb[action.doc_id]["text"] = action.text
+                    msg = f"Updated text for {action.doc_id}."
+                    
+            elif action.action_type == "add_metadata":
+                if not action.doc_id or not action.metadata_key or not action.metadata_value:
+                    msg = "Error: doc_id, metadata_key, and metadata_value required."
+                else:
+                    if action.doc_id not in self.kb:
+                        msg = f"Error: doc_id {action.doc_id} not found."
+                    else:
+                        self.kb[action.doc_id]["metadata"][action.metadata_key] = action.metadata_value
+                        msg = f"Added metadata to {action.doc_id}."
+                        
+            elif action.action_type == "submit":
+                done = True
+                reward = self._evaluate_kb()
+                msg = f"Evaluation complete. Final reward: {reward:.2f}"
+                
+        except Exception as e:
+            msg = f"Action failed: {str(e)}"
+
+        if not done:
+            reward = self._evaluate_kb()
+
+        return RagOptimizerObservation(
+            message=msg,
+            current_docs=self._get_kb_summary(),
+            done=done,
+            reward=reward,
+        )
+
+    @property
+    def state(self) -> State:
+        return self._state

run.sh

@@ -0,0 +1,180 @@
+#
+# Checks that your HF Space is live, Docker image builds, and openenv validate passes.
+#
+# Prerequisites:
+#   - Docker:       https://docs.docker.com/get-docker/
+#   - openenv-core: pip install openenv-core
+#   - curl (usually pre-installed)
+#
+# Run:
+#   curl -fsSL https://raw.githubusercontent.com/<owner>/<repo>/main/scripts/validate-submission.sh | bash -s -- <ping_url> [repo_dir]
+#
+#   Or download and run locally:
+#     chmod +x validate-submission.sh
+#     ./validate-submission.sh <ping_url> [repo_dir]
+#
+# Arguments:
+#   ping_url   Your HuggingFace Space URL (e.g. https://your-space.hf.space)
+#   repo_dir   Path to your repo (default: current directory)
+#
+# Examples:
+#   ./validate-submission.sh https://my-team.hf.space
+#   ./validate-submission.sh https://my-team.hf.space ./my-repo
+#
+
+set -uo pipefail
+
+DOCKER_BUILD_TIMEOUT=600
+if [ -t 1 ]; then
+  RED='\033[0;31m'
+  GREEN='\033[0;32m'
+  YELLOW='\033[1;33m'
+  BOLD='\033[1m'
+  NC='\033[0m'
+else
+  RED='' GREEN='' YELLOW='' BOLD='' NC=''
+fi
+
+run_with_timeout() {
+  local secs="$1"; shift
+  if command -v timeout &>/dev/null; then
+    timeout "$secs" "$@"
+  elif command -v gtimeout &>/dev/null; then
+    gtimeout "$secs" "$@"
+  else
+    "$@" &
+    local pid=$!
+    ( sleep "$secs" && kill "$pid" 2>/dev/null ) &
+    local watcher=$!
+    wait "$pid" 2>/dev/null
+    local rc=$?
+    kill "$watcher" 2>/dev/null
+    wait "$watcher" 2>/dev/null
+    return $rc
+  fi
+}
+
+portable_mktemp() {
+  local prefix="${1:-validate}"
+  mktemp "${TMPDIR:-/tmp}/${prefix}-XXXXXX" 2>/dev/null || mktemp
+}
+
+CLEANUP_FILES=()
+cleanup() { rm -f "${CLEANUP_FILES[@]+"${CLEANUP_FILES[@]}"}"; }
+trap cleanup EXIT
+
+PING_URL="${1:-}"
+REPO_DIR="${2:-.}"
+
+if [ -z "$PING_URL" ]; then
+  printf "Usage: %s <ping_url> [repo_dir]\n" "$0"
+  printf "\n"
+  printf "  ping_url   Your HuggingFace Space URL (e.g. https://your-space.hf.space)\n"
+  printf "  repo_dir   Path to your repo (default: current directory)\n"
+  exit 1
+fi
+
+if ! REPO_DIR="$(cd "$REPO_DIR" 2>/dev/null && pwd)"; then
+  printf "Error: directory '%s' not found\n" "${2:-.}"
+  exit 1
+fi
+PING_URL="${PING_URL%/}"
+export PING_URL
+PASS=0
+
+log()  { printf "[%s] %b\n" "$(date -u +%H:%M:%S)" "$*"; }
+pass() { log "${GREEN}PASSED${NC} -- $1"; PASS=$((PASS + 1)); }
+fail() { log "${RED}FAILED${NC} -- $1"; }
+hint() { printf "  ${YELLOW}Hint:${NC} %b\n" "$1"; }
+stop_at() {
+  printf "\n"
+  printf "${RED}${BOLD}Validation stopped at %s.${NC} Fix the above before continuing.\n" "$1"
+  exit 1
+}
+
+printf "\n"
+printf "${BOLD}========================================${NC}\n"
+printf "${BOLD}  OpenEnv Submission Validator${NC}\n"
+printf "${BOLD}========================================${NC}\n"
+log "Repo:     $REPO_DIR"
+log "Ping URL: $PING_URL"
+printf "\n"
+
+log "${BOLD}Step 1/3: Pinging HF Space${NC} ($PING_URL/reset) ..."
+
+CURL_OUTPUT=$(portable_mktemp "validate-curl")
+CLEANUP_FILES+=("$CURL_OUTPUT")
+HTTP_CODE=$(curl -s -o "$CURL_OUTPUT" -w "%{http_code}" -X POST \
+  -H "Content-Type: application/json" -d '{}' \
+  "$PING_URL/reset" --max-time 30 2>"$CURL_OUTPUT" || printf "000")
+
+if [ "$HTTP_CODE" = "200" ]; then
+  pass "HF Space is live and responds to /reset"
+elif [ "$HTTP_CODE" = "000" ]; then
+  fail "HF Space not reachable (connection failed or timed out)"
+  hint "Check your network connection and that the Space is running."
+  hint "Try: curl -s -o /dev/null -w '%%{http_code}' -X POST $PING_URL/reset"
+  stop_at "Step 1"
+else
+  fail "HF Space /reset returned HTTP $HTTP_CODE (expected 200)"
+  hint "Make sure your Space is running and the URL is correct."
+  hint "Try opening $PING_URL in your browser first."
+  stop_at "Step 1"
+fi
+
+log "${BOLD}Step 2/3: Running docker build${NC} ..."
+
+if ! command -v docker &>/dev/null; then
+  fail "docker command not found"
+  hint "Install Docker: https://docs.docker.com/get-docker/"
+  stop_at "Step 2"
+fi
+
+if [ -f "$REPO_DIR/Dockerfile" ]; then
+  DOCKER_CONTEXT="$REPO_DIR"
+elif [ -f "$REPO_DIR/server/Dockerfile" ]; then
+  DOCKER_CONTEXT="$REPO_DIR/server"
+else
+  fail "No Dockerfile found in repo root or server/ directory"
+  stop_at "Step 2"
+fi
+
+log "  Found Dockerfile in $DOCKER_CONTEXT"
+
+BUILD_OK=false
+BUILD_OUTPUT=$(run_with_timeout "$DOCKER_BUILD_TIMEOUT" docker build "$DOCKER_CONTEXT" 2>&1) && BUILD_OK=true
+
+if [ "$BUILD_OK" = true ]; then
+  pass "Docker build succeeded"
+else
+  fail "Docker build failed (timeout=${DOCKER_BUILD_TIMEOUT}s)"
+  printf "%s\n" "$BUILD_OUTPUT" | tail -20
+  stop_at "Step 2"
+fi
+
+log "${BOLD}Step 3/3: Running openenv validate${NC} ..."
+
+if ! command -v openenv &>/dev/null; then
+  fail "openenv command not found"
+  hint "Install it: pip install openenv-core"
+  stop_at "Step 3"
+fi
+
+VALIDATE_OK=false
+VALIDATE_OUTPUT=$(cd "$REPO_DIR" && openenv validate 2>&1) && VALIDATE_OK=true
+
+if [ "$VALIDATE_OK" = true ]; then
+  pass "openenv validate passed"
+  [ -n "$VALIDATE_OUTPUT" ] && log "  $VALIDATE_OUTPUT"
+else
+  fail "openenv validate failed"
+  printf "%s\n" "$VALIDATE_OUTPUT"
+  stop_at "Step 3"
+fi
+
+printf "\n"
+printf "${BOLD}========================================${NC}\n"
+printf "${GREEN}${BOLD}  All 3/3 checks passed!${NC}\n"
+printf "${GREEN}${BOLD}  Your submission is ready to submit.${NC}\n"
+printf "${BOLD}========================================${NC}\n"
+printf "\n"