Upload folder using huggingface_hub

README.md

@@ -162,14 +162,14 @@ The environment tests agents across three progressively demanding task distribut
 **The Vector Issue:** The base contains heavily overlapping parameters (competing versions of legacy and modern timeline protocols).
 **System Goal:** Autonomously survey the semantic differences, deduce the temporal conflict, and execute `delete_document` sweeps to purge vector hallucination triggers.
 
-### Level II: Signal Separation
-**The Vector Issue:** The KB contains overlapping incident narratives mixed with distractor engineering notes, causing retrieval ambiguity.
-**System Goal:** Deduplicate noisy/partial incident content so retrieval consistently surfaces the clean resolution document at rank 1.
-
-### Level III: Syntactic Splintering (The Monolith)
+### Level II: Syntactic Splintering
 **The Vector Issue:** Extreme embedding decay caused by disparate conceptual structures compacted under a single referential document. This represents the well-known "PDF chunk wash-out" phenomenon.
 **System Goal:** Methodically `read` the extensive parent block, temporarily cache the semantic context limits, and utilize rapid consecutive `update_document` calls to mechanically splinter and redistribute the knowledge logic across multiple fine-grained nodes.
 
+### Level III: Duplicate Purge
+**The Vector Issue:** The KB contains overlapping FastAPI routing docs where legacy `@app.route()` guidance competes against the correct current `@app.get()` / `@app.post()` pattern with Pydantic v2.
+**System Goal:** Deduplicate legacy routing docs so retrieval consistently ranks the correct current routing reference at #1.
+
 ---
 
 ## 5. Inference Baseline Sandbox

inference.py

@@ -113,6 +113,40 @@ def _clamp_score(value: float) -> float:
     return value
 
 
+def _extract_json_object(text: str) -> dict:
+    """Extract a JSON object from model output that may include extra text."""
+    if not text:
+        raise ValueError("empty model response")
+
+    raw = text.strip()
+    try:
+        return json.loads(raw)
+    except Exception:
+        pass
+
+    # Common markdown fence wrapper
+    if "```" in raw:
+        parts = raw.split("```")
+        for part in parts:
+            candidate = part.strip()
+            if candidate.lower().startswith("json"):
+                candidate = candidate[4:].strip()
+            if candidate.startswith("{") and candidate.endswith("}"):
+                try:
+                    return json.loads(candidate)
+                except Exception:
+                    pass
+
+    # Fallback: take substring between first '{' and last '}'
+    start = raw.find("{")
+    end = raw.rfind("}")
+    if start != -1 and end != -1 and end > start:
+        candidate = raw[start:end + 1]
+        return json.loads(candidate)
+
+    raise ValueError("no JSON object found in model response")
+
+
 def run_task_episode(
     env: RagOptimizerEnvClient,
     llm_client: OpenAI,
@@ -156,14 +190,22 @@ def run_task_episode(
         error_msg = "null"
 
         try:
-            completion = llm_client.chat.completions.create(
-                model=MODEL_NAME,
-                messages=messages,
-                response_format={"type": "json_object"},
-                max_tokens=1000,
-            )
+            try:
+                completion = llm_client.chat.completions.create(
+                    model=MODEL_NAME,
+                    messages=messages,
+                    response_format={"type": "json_object"},
+                    max_tokens=1000,
+                )
+            except Exception:
+                # Some OpenAI-compatible providers may not enforce response_format.
+                completion = llm_client.chat.completions.create(
+                    model=MODEL_NAME,
+                    messages=messages,
+                    max_tokens=1000,
+                )
             response_text = completion.choices[0].message.content or ""
-            action_data = json.loads(response_text)
+            action_data = _extract_json_object(response_text)
 
             # Normalize fields if model returns lists instead of strings
             for field in ("doc_id", "text", "metadata_key", "metadata_value"):

memory/lessons_learned.json

@@ -0,0 +1,30 @@
+[
+  {
+    "task": "medium",
+    "lesson": "The agent should have prioritized deleting documents with relevant metadata, such as \"doc_incident_raw\", \"doc_incident_partial\", and \"doc_incident_resolved\", which contained meaningful information, instead of sequentially deleting the distractor documents."
+  },
+  {
+    "task": "easy",
+    "lesson": "The agent should have added the \"subcategory\": \"api_framework\" metadata to documents \"fastapi_7\", \"fastapi_8\", \"fastapi_9\", and \"fastapi_10\" to increase the consistency of the knowledge base and ultimately achieve a higher reward."
+  },
+  {
+    "task": "medium",
+    "lesson": "The agent should have taken a more targeted approach to document deletion, prioritizing the removal of irrelevant documents while preserving key incident reports, such as \"doc_incident_raw\", \"doc_incident_partial\", and \"doc_incident_resolved\", to maximize the reward and achieve the task objective."
+  },
+  {
+    "task": "medium",
+    "lesson": "The agent should have performed a \"merge_document\" action to combine the relevant information from \"doc_incident_raw\" and \"doc_incident_partial\" into \"doc_incident_resolved\", instead of deleting distractor documents and reading \"doc_incident_resolved\", to achieve a higher reward and successfully"
+  },
+  {
+    "task": "medium",
+    "lesson": ""
+  },
+  {
+    "task": "medium",
+    "lesson": ""
+  },
+  {
+    "task": "medium",
+    "lesson": ""
+  }
+]

openenv.yaml

@@ -11,24 +11,24 @@ environment:
     state: "/state/{session_id}"
 tasks:
   - id: easy
-    name: "Conflict Resolution"
-    description: "Resolve overlapping pricing parameters by deleting the legacy pricing format."
+    name: "Contamination Purging"
+    description: "Purge contradictory or low-value documents so retrieval surfaces clean, high-signal references."
     reward_range: [0.01, 0.99]
     grader: "rag_optimizer.server.grader:grade_easy"
   - id: medium
-    name: "Signal Separation"
-    description: >
-      Multi-doc deduplication. KB contains 3 overlapping incident reports
-      for the same outage - one raw, one partial, one with the correct
-      resolution - plus 20 engineering distractors. Agent must delete the
-      noisy duplicates so retrieval surfaces the clean resolution doc at
-      rank 1. Reward improves continuously as noise is removed; no binary
-      metadata gate.
+    name: "Syntactic Splintering"
+    description: "Break down the monolithic onboarding blob into multiple granular chunks."
     reward_range: [0.01, 0.99]
     grader: "rag_optimizer.server.grader:grade_medium"
   - id: hard
-    name: "Syntactic Splintering"
-    description: "Break down the monolithic onboarding blob into multiple granular chunks."
+    name: "Duplicate Purge"
+    description: >
+      Multi-doc deduplication for framework drift. KB contains 3 overlapping
+      FastAPI routing docs: two legacy variants with incorrect @app.route()
+      guidance and one correct current doc using @app.get()/@app.post() and
+      Pydantic v2, plus distractors. Agent must delete noisy legacy docs so
+      retrieval surfaces the correct current doc at rank 1. Reward improves
+      continuously as noise is removed; no binary metadata gate.
     reward_range: [0.01, 0.99]
     grader: "rag_optimizer.server.grader:grade_hard"
 agent:

server/kb_seed.json

@@ -123,20 +123,96 @@
         }
     },
     "medium": {
-        "falcon9": {
-            "text": "Falcon 9 is a reusable rocket developed by SpaceX for transporting payloads to orbit. It uses SpaceX Merlin engines, which are designed for efficiency and reusability. The first stage can land vertically after launch, significantly reducing costs. Falcon 9 has been used for satellite deployments, cargo missions, and crewed flights to the International Space Station. Its reliability and cost-effectiveness have made it a cornerstone of modern space exploration.",
+        "doc_incident_raw": {
+            "text": "Initial incident report logged by the on-call engineer at 09:10 AM. Users are reporting that a critical frontend action button on the dashboard is missing entirely. The issue appears intermittent across sessions but is reproducible in staging. Alongside this, monitoring dashboards show elevated database latency, particularly on read-heavy endpoints. There is uncertainty whether these issues are related or coincidental. Some engineers suspect a rendering issue, while others believe degraded backend performance may be delaying UI state hydration. No mitigation has been applied yet. Logs show sporadic 401 errors on certain API endpoints, but it is unclear if this is connected. The incident has been escalated to both frontend and backend teams. Investigation is ongoing, and further updates will be provided as more data becomes available.",
             "metadata": {}
         },
-        "dragon": {
-            "text": "The Dragon spacecraft developed by SpaceX is used for cargo and crew transport. The original Dragon capsule was replaced by Dragon 2, which includes advanced life-support systems and autonomous docking capabilities. Dragon has been used extensively for resupply missions to the International Space Station and has enabled private spaceflight.",
+        "doc_incident_partial": {
+            "text": "Follow-up report after initial investigation. The frontend team attempted a quick fix by updating CSS rules related to button visibility, assuming a rendering regression. However, this did not resolve the missing button issue in production. Further debugging indicates that the button component is conditionally rendered based on an API response, which is occasionally failing. Backend logs show consistent 401 errors from an internal service endpoint. The backend team suspects that an expired API key might be causing authentication failures, leading to incomplete data being sent to the frontend. Database latency has slightly improved but is still above baseline. Teams are now focusing on authentication layers and service communication. No permanent fix has been deployed yet.",
             "metadata": {}
         },
-        "starship": {
-            "text": "Starship is a fully reusable spacecraft designed by SpaceX for deep space missions. It aims to transport humans to Mars and beyond. The system includes a booster and spacecraft, both reusable. Starship is powered by Raptor engines and is designed for rapid turnaround.",
+        "doc_incident_resolved": {
+            "text": "Final post-mortem of the production outage. The root cause was determined to be two independent but overlapping issues. The missing frontend button was caused by a regression in styling, which was resolved by updating CSS stylesheet rules that incorrectly hid the component under certain conditions. Separately, the 401 authentication errors were traced to an expired API key used by an internal service. This was resolved through API key rotation on Tuesday, restoring proper authentication flow. Once both fixes were deployed, system behavior returned to normal. Additional safeguards have been implemented, including monitoring for API key expiration and stricter validation of frontend rendering conditions. Teams have documented the incident and updated runbooks to prevent recurrence.",
             "metadata": {}
         },
-        "distractor_1": {
-            "text": "Deep sea exploration involves studying ocean depths using submersibles and remote-operated vehicles. Scientists investigate marine ecosystems and geological formations.",
+        "doc_eng_1": {
+            "text": "Architecture decision record discussing migration of frontend styling systems. The team evaluated moving from traditional CSS modules to a utility-first framework. During testing, inconsistencies were observed in how legacy components handled conditional rendering tied to API key permissions. Some endpoints returned 401 errors when tokens expired, which affected component visibility. The decision was to incrementally migrate styles while keeping backward compatibility. Engineers also noted that rotation policies for API keys should be standardized across services. This ADR highlights the importance of aligning frontend behavior with backend authentication systems.",
+            "metadata": {}
+        },
+        "doc_eng_2": {
+            "text": "Sprint planning notes for Q3 engineering cycle. The frontend team proposed improvements to UI consistency, including refactoring CSS classes and removing deprecated styles. Meanwhile, the backend team raised concerns about API key rotation policies, noting that expired keys had caused minor 401 errors in staging. The team agreed to prioritize monitoring improvements and better alerting mechanisms. No direct incidents were reported, but proactive measures were discussed to prevent future disruptions.",
+            "metadata": {}
+        },
+        "doc_eng_3": {
+            "text": "Git commit summary describing updates to authentication middleware. The change introduced stricter validation for API keys, rejecting requests with expired credentials. This led to an increase in 401 errors during testing, which was expected. The frontend team was notified to handle these errors gracefully. Additionally, minor CSS updates were included to improve button alignment across browsers. The commit emphasizes security improvements and better error handling.",
+            "metadata": {}
+        },
+        "doc_eng_4": {
+            "text": "Benchmark report analyzing frontend performance under load. Tests revealed that CSS rendering time increased slightly when large stylesheets were loaded. API key validation latency also contributed to overall response times. However, no critical failures were observed. Engineers recommended optimizing CSS delivery and caching API key validation results to reduce overhead.",
+            "metadata": {}
+        },
+        "doc_eng_5": {
+            "text": "Internal memo discussing API key lifecycle management. The document outlines best practices for rotation, including automated renewal and monitoring for expiration. It also highlights cases where expired keys caused 401 errors in non-critical services. Frontend teams are advised to display fallback UI elements when authentication fails.",
+            "metadata": {}
+        },
+        "doc_eng_6": {
+            "text": "Design document for a new frontend component library. The library standardizes CSS usage and ensures consistent styling across applications. It also integrates with backend authentication systems, handling API key validation states. Engineers noted that improper handling of 401 errors could lead to missing UI elements.",
+            "metadata": {}
+        },
+        "doc_eng_7": {
+            "text": "Post-deployment analysis of a minor release. The update included CSS refactoring and improvements to API key validation logic. Some users experienced temporary 401 errors due to delayed key rotation, but no major outages occurred. The team plans to improve synchronization between frontend and backend systems.",
+            "metadata": {}
+        },
+        "doc_eng_8": {
+            "text": "Engineering notes on improving observability. The team added logging for API key validation failures and CSS rendering issues. This helps identify cases where frontend components fail to display due to backend errors. Alerts are configured for repeated 401 errors.",
+            "metadata": {}
+        },
+        "doc_eng_9": {
+            "text": "Technical review of authentication flows. The system uses API keys for service-to-service communication. Engineers identified that inconsistent rotation schedules could lead to 401 errors. Frontend applications must handle these gracefully to avoid user confusion.",
+            "metadata": {}
+        },
+        "doc_eng_10": {
+            "text": "Frontend refactor proposal focusing on CSS modularization. The goal is to reduce conflicts between styles and improve maintainability. The proposal also includes handling API key-based feature flags, ensuring components render correctly even when authentication states change.",
+            "metadata": {}
+        },
+        "doc_eng_11": {
+            "text": "System reliability report highlighting minor incidents. Some services experienced 401 errors due to expired API keys. CSS updates were deployed to improve UI consistency. No major user impact was observed.",
+            "metadata": {}
+        },
+        "doc_eng_12": {
+            "text": "Developer onboarding guide explaining authentication mechanisms. New engineers are taught how API keys are generated, rotated, and validated. The guide also covers frontend error handling for 401 responses.",
+            "metadata": {}
+        },
+        "doc_eng_13": {
+            "text": "Performance optimization notes for frontend rendering. Engineers experimented with reducing CSS bundle sizes and improving load times. API key validation overhead was also analyzed.",
+            "metadata": {}
+        },
+        "doc_eng_14": {
+            "text": "Security audit report identifying risks in API key management. The audit found that some keys were not rotated regularly, leading to potential 401 errors. Recommendations include automated rotation and better monitoring.",
+            "metadata": {}
+        },
+        "doc_eng_15": {
+            "text": "Release notes for version 2.3. The update includes CSS fixes and improved API key validation. Some minor 401 errors were expected during rollout.",
+            "metadata": {}
+        },
+        "doc_eng_16": {
+            "text": "Backend architecture review discussing authentication services. API key validation is centralized, but frontend teams must handle errors properly. CSS-related UI issues were noted in previous releases.",
+            "metadata": {}
+        },
+        "doc_eng_17": {
+            "text": "Incident simulation exercise documenting response to API key expiration. Teams practiced handling 401 errors and updating frontend components. CSS adjustments were part of the simulation.",
+            "metadata": {}
+        },
+        "doc_eng_18": {
+            "text": "Code review feedback on frontend components. Reviewers noted inconsistent CSS usage and lack of error handling for API key failures. Improvements were suggested.",
+            "metadata": {}
+        },
+        "doc_eng_19": {
+            "text": "Infrastructure planning document for scaling authentication services. API key rotation policies are discussed, along with frontend implications of 401 errors.",
+            "metadata": {}
+        },
+        "doc_eng_20": {
+            "text": "QA testing report covering authentication and UI scenarios. Test cases include expired API keys, resulting in 401 errors, and CSS rendering issues affecting frontend components.",
             "metadata": {}
         }
     },

server/rag_optimizer_environment.py

@@ -69,37 +69,44 @@ class RagOptimizerEnvironment(Environment):
             ]
             
         elif task_id == "medium":
+            self.kb = deepcopy(self.seed_data["hard"])
+            self.test_suite = [
+                {"query": "Which entity bankruptcy was the climax of the disaster?", "target_concept": "bankruptcy of Lehman Brothers"},
+                {"query": "What types of collateralized assets lost their worth?", "target_concept": "Mortgage-backed securities"},
+                {"query": "Who did predatory lenders primarily go after?", "target_concept": "low-income homebuyers"}
+            ]
+
+        elif task_id == "hard":
             self.kb = {
-                "doc_incident_raw": {
-                    "text": "User reported frontend button missing. Database latency also flagged. No fix yet.",
-                    "metadata": {}
+                # v1 — legacy, wrong routing syntax, poisons retrieval rank for v3
+                "fastapi_routing_v1": {
+                    "text": "FastAPI routing uses @app.route() decorator similar to Flask. Define routes with methods=['GET','POST']. Pydantic v1 handles all schema validation.",
+                    "metadata": {"version": "legacy"}
                 },
-                "doc_incident_partial": {
-                    "text": "Button issue and DB latency investigated. CSS fix attempted. Email 401 error also reported.",
-                    "metadata": {}
+                # v2 — also legacy, still wrong syntax, further dilutes correct doc
+                "fastapi_routing_v2": {
+                    "text": "FastAPI routes are defined using @app.route() with type hints added. Validation is done through Pydantic v1 models attached to each endpoint.",
+                    "metadata": {"version": "legacy"}
                 },
-                "doc_incident_resolved": {
-                    "text": "Frontend button restored by updating CSS stylesheet. Email 401 resolved after API key rotation on Tuesday.",
-                    "metadata": {}
+                # v3 — CORRECT current doc, must survive and rank #1 after purge
+                "fastapi_routing_v3": {
+                    "text": "FastAPI uses @app.get(), @app.post() and other HTTP method decorators for routing. It defaults to Pydantic v2 for request and response validation.",
+                    "metadata": {"version": "current"}
                 },
             }
-            for i in range(20):
-                self.kb[f"doc_distractor_eng_{i}"] = {
-                    "text": f"Architecture decision {i}: chose Postgres for horizontal scaling.",
+            # 15 generic distractors — low semantic overlap, provide ranking noise
+            for i in range(15):
+                self.kb[f"doc_distractor_{i}"] = {
+                    "text": f"General web framework note {i}: always use async handlers for better throughput in high-traffic services.",
                     "metadata": {}
                 }
             self.test_suite = [
-                {"query": "How was the missing UI element fixed?",
-                 "target_concept": "updating CSS stylesheet"},
-                {"query": "What caused the authentication failure?",
-                 "target_concept": "API key rotation on Tuesday"},
-            ]
-            
-        elif task_id == "hard":
-            self.test_suite = [
-                {"query": "Which entity bankruptcy was the climax of the disaster?", "target_concept": "bankruptcy of Lehman Brothers"},
-                {"query": "What types of collateralized assets lost their worth?", "target_concept": "Mortgage-backed securities"},
-                {"query": "Who did predatory lenders primarily go after?", "target_concept": "low-income homebuyers"}
+                # v1 and v2 both contain @app.route() which competes semantically with v3
+                # Deleting v1 and v2 pushes v3 to rank 1 → each delete gives a visible reward jump
+                {"query": "How are routes defined in FastAPI?",
+                 "target_concept": "@app.get(), @app.post() decorators"},
+                {"query": "Which Pydantic version does FastAPI v2 use by default?",
+                 "target_concept": "defaults to Pydantic v2"},
             ]
 
         self._rebuild_cache()
@@ -266,4 +273,4 @@ class RagOptimizerEnvironment(Environment):
 
     @property
     def state(self) -> State:
-        return self._state
+        return self._state