| "html": "<html><head><meta name=\"color-scheme\" content=\"light dark\"></head><body><pre style=\"word-wrap: break-word; white-space: pre-wrap;\">\"\"\"\nAICL Autonomous Compilation System — Self-Writing, Self-Validating Code\n\nThe central thesis: a compiler that cannot explain why it generated a line\nshould not generate it. This module extends that thesis to its logical\nconclusion: a compiler that can diagnose its own failures, fix its own\nspecifications, learn new patterns, and validate its own output — all\nwhile preserving the No-Orphan Property.\n\nThe Autonomous Compilation Loop:\n SPEC → COMPILE → VERIFY → TEST → DIAGNOSE → FIX → RECOMPILE\n ↑ │\n └──────────────────────────────────────┘\n\nEvery iteration of the loop preserves the provenance chain. Every fix\nis recorded. Every learned pattern is tracked. The loop converges when:\n 1. Spec verification passes (completeness, coherence, satisfaction)\n 2. Audit coverage = 1.0 (no orphans)\n 3. All generated tests pass\n 4. No fallback compilations remain\n 5. Improvement score converges (delta < threshold)\n\nNew ProvenanceTypes added by this module:\n - AI_GENERATION: Code generated by AI-assisted fallback\n - SELF_HEALING: Code fixed by autonomous diagnosis\n - PATTERN_LEARNED: New pattern learned from unmatched actions\n - SPEC_EVOLVED: Specification modified by autonomous evolution\n\nUsage:\n aicl evolve <source.aicl> [--max-iterations 10] [--no-ai]\n aicl evolve <source.aicl> --watch # continuous mode\n\"\"\"\n\nimport os\nimport re\nimport sys\nimport json\nimport time\nimport hashlib\nimport subprocess\nimport tempfile\nimport shutil\nimport traceback\nfrom dataclasses import dataclass, field\nfrom typing import List, Dict, Optional, Tuple, Set, Any\nfrom enum import Enum\n\nfrom .provenance import ProvenanceType\n\n\nclass LoopState(Enum):\n \"\"\"State of the autonomous compilation loop.\"\"\"\n INITIALIZING = \"initializing\"\n COMPILING = \"compiling\"\n VERIFYING = \"verifying\"\n TESTING = \"testing\"\n DIAGNOSING = \"diagnosing\"\n FIXING = \"fixing\"\n LEARNING = \"learning\"\n CONVERGED = \"converged\"\n FAILED = \"failed\"\n TIMED_OUT = \"timed_out\"\n\n\n@dataclass\nclass LoopIteration:\n \"\"\"Record of a single loop iteration.\"\"\"\n iteration: int\n state_entered: LoopState\n timestamp_start: float\n state_exited: LoopState = LoopState.INITIALIZING\n timestamp_end: float = 0.0\n spec_changes: List[str] = field(default_factory=list)\n patterns_learned: List[str] = field(default_factory=list)\n tests_passed: int = 0\n tests_failed: int = 0\n tests_total: int = 0\n audit_coverage: float = 0.0\n fallback_count: int = 0\n provenance_records_added: int = 0\n errors: List[str] = field(default_factory=list)\n fixes_applied: List[str] = field(default_factory=list)\n\n @property\n def duration(self) -> float:\n return self.timestamp_end - self.timestamp_start\n\n @property\n def success(self) -> bool:\n return (\n self.tests_failed == 0\n and self.tests_total > 0\n and self.audit_coverage >= 1.0\n and self.fallback_count == 0\n )\n\n\n@dataclass\nclass AutonomousResult:\n \"\"\"Final result of an autonomous compilation session.\"\"\"\n converged: bool\n iterations: List[LoopIteration]\n total_duration: float\n final_audit_coverage: float\n final_test_pass_rate: float\n patterns_learned: List[str]\n spec_evolution: List[str]\n convergence_iteration: Optional[int] = None\n proof_valid: bool = False\n\n @property\n def summary(self) -> str:\n lines = [\n f\"Converged: {self.converged}\",\n f\"Iterations: {len(self.iterations)}\",\n f\"Duration: {self.total_duration:.2f}s\",\n f\"Audit Coverage: {self.final_audit_coverage:.1%}\",\n f\"Test Pass Rate: {self.final_test_pass_rate:.1%}\",\n f\"Patterns Learned: {len(self.patterns_learned)}\",\n f\"Spec Changes: {len(self.spec_evolution)}\",\n f\"Proof Valid: {self.proof_valid}\",\n ]\n if self.convergence_iteration is not None:\n lines.append(f\"Converged at iteration: {self.convergence_iteration}\")\n return \"\\n\".join(lines)\n\n\nclass TestRunner:\n \"\"\"Execute generated tests and analyze failures.\"\"\"\n\n def __init__(self, timeout: int = 30):\n self.timeout = timeout\n self.last_results: Dict[str, Any] = {}\n\n def run_tests(self, output_dir: str) -> Tuple[int, int, int, List[Dict]]:\n \"\"\"\n Run pytest on the generated test file.\n Returns (passed, failed, total, failure_details).\n \"\"\"\n test_file = os.path.join(output_dir, \"test_main.py\")\n if not os.path.exists(test_file):\n return 0, 0, 0, [{\"error\": \"No test file found\"}]\n\n # Try to find pytest - check venv first, then system\n pytest_path = shutil.which(\"pytest\")\n python_path = sys.executable\n\n try:\n cmd = [python_path, \"-m\", \"pytest\", test_file, \"-v\", \"--tb=short\", \"--no-header\"]\n result = subprocess.run(\n cmd,\n capture_output=True,\n text=True,\n timeout=self.timeout,\n cwd=output_dir,\n )\n except subprocess.TimeoutExpired:\n return 0, 0, 0, [{\"error\": f\"Test execution timed out after {self.timeout}s\"}]\n except Exception as e:\n return 0, 0, 0, [{\"error\": f\"Test execution failed: {str(e)}\"}]\n\n # Parse pytest output\n passed, failed, total = 0, 0, 0\n failure_details = []\n\n for line in result.stdout.split(\"\\n\"):\n line = line.strip()\n # Match: \"test_main.py::test_validation_1 PASSED\"\n if \" PASSED\" in line:\n passed += 1\n total += 1\n elif \" FAILED\" in line:\n failed += 1\n total += 1\n test_name = line.split(\"::\")[-1].split()[0] if \"::\" in line else \"unknown\"\n failure_details.append({\"test\": test_name, \"raw\": line})\n\n # Also check the summary line: \"X passed, Y failed\"\n for line in result.stdout.split(\"\\n\"):\n m = re.match(r\"(\\d+) passed\", line)\n if m:\n passed = int(m.group(1))\n m = re.match(r\".*?(\\d+) failed\", line)\n if m:\n failed = int(m.group(1))\n m = re.match(r\".*?(\\d+) warning\", line)\n # total = passed + failed\n\n if total == 0:\n total = passed + failed\n\n self.last_results = {\n \"passed\": passed,\n \"failed\": failed,\n \"total\": total,\n \"returncode\": result.returncode,\n \"stdout\": result.stdout,\n \"stderr\": result.stderr,\n }\n\n return passed, failed, total, failure_details\n\n def analyze_failure(self, test_name: str, output_dir: str) -> Dict[str, Any]:\n \"\"\"Analyze a specific test failure and return diagnostic info.\"\"\"\n test_file = os.path.join(output_dir, \"test_main.py\")\n main_file = os.path.join(output_dir, \"main.py\")\n\n diagnosis = {\n \"test_name\": test_name,\n \"missing_imports\": [],\n \"missing_methods\": [],\n \"attribute_errors\": [],\n \"type_errors\": [],\n \"assertion_failures\": [],\n \"suggested_fix\": None,\n }\n\n stdout = self.last_results.get(\"stdout\", \"\")\n stderr = self.last_results.get(\"stderr\", \"\")\n\n full_output = stdout + \"\\n\" + stderr\n\n # Pattern-based diagnosis\n if \"ModuleNotFoundError\" in full_output or \"ImportError\" in full_output:\n for m in re.finditer(r\"No module named '(\\w+)'\", full_output):\n diagnosis[\"missing_imports\"].append(m.group(1))\n diagnosis[\"suggested_fix\"] = \"add_missing_imports\"\n\n if \"AttributeError\" in full_output:\n for m in re.finditer(r\"'(\\w+)' object has no attribute '(\\w+)'\", full_output):\n diagnosis[\"attribute_errors\"].append({\n \"class\": m.group(1),\n \"attribute\": m.group(2),\n })\n diagnosis[\"suggested_fix\"] = \"add_missing_attributes\"\n\n if \"TypeError\" in full_output:\n diagnosis[\"type_errors\"].append(full_output[:500])\n diagnosis[\"suggested_fix\"] = \"fix_type_signatures\"\n\n if \"AssertionError\" in full_output or \"assert\" in full_output:\n diagnosis[\"assertion_failures\"].append(test_name)\n diagnosis[\"suggested_fix\"] = \"fix_validation_logic\"\n\n if \"NameError\" in full_output:\n for m in re.finditer(r\"name '(\\w+)' is not defined\", full_output):\n diagnosis[\"missing_methods\"].append(m.group(1))\n diagnosis[\"suggested_fix\"] = \"add_missing_definitions\"\n\n # Check if main.py exists and has content\n if os.path.exists(main_file):\n with open(main_file) as f:\n main_content = f.read()\n diagnosis[\"main_file_lines\"] = len(main_content.split(\"\\n\"))\n else:\n diagnosis[\"main_file_lines\"] = 0\n diagnosis[\"suggested_fix\"] = \"regenerate_main\"\n\n return diagnosis\n\n\nclass PatternLearner:\n \"\"\"\n Learn new behavior patterns from unmatched action descriptions.\n\n When the deterministic pattern matcher falls back (no match found),\n the PatternLearner analyzes the unmatched description and creates\n a candidate pattern that can be used in future compilations.\n \"\"\"\n\n def __init__(self):\n self.learned_patterns: Dict[str, Dict] = {}\n self.unmatched_descriptions: List[str] = []\n\n def record_unmatched(self, behavior_name: str, description: str):\n \"\"\"Record an unmatched action description for learning.\"\"\"\n self.unmatched_descriptions.append(f\"{behavior_name}: {description}\")\n\n def learn_from_description(self, behavior_name: str, description: str) -> Optional[Dict]:\n \"\"\"\n Analyze an unmatched action description and generate a candidate pattern.\n Returns a pattern dict or None if no pattern can be inferred.\n \"\"\"\n desc_lower = description.lower().strip()\n\n # Extract verbs and objects from the description\n verb = self._extract_verb(desc_lower)\n obj = self._extract_object(desc_lower)\n\n if not verb or not obj:\n return None\n\n # Classify the pattern based on verb-object analysis\n pattern = {\n \"name\": f\"LEARNED_{behavior_name.upper()}\",\n \"behavior\": behavior_name,\n \"verb\": verb,\n \"object\": obj,\n \"category\": self._classify_category(verb, obj),\n \"keywords\": self._extract_keywords(desc_lower),\n \"description\": description,\n \"template\": self._generate_template(verb, obj, behavior_name),\n \"confidence\": 0.7, # Learned patterns start at 0.7 (below deterministic 0.95)\n \"learned_at\": time.time(),\n }\n\n self.learned_patterns[behavior_name] = pattern\n return pattern\n\n def _extract_verb(self, description: str) -> str:\n \"\"\"Extract the primary verb from an action description.\"\"\"\n action_verbs = [\n \"create\", \"delete\", \"update\", \"modify\", \"set\", \"change\",\n \"add\", \"remove\", \"append\", \"push\", \"pop\", \"insert\",\n \"validate\", \"verify\", \"check\", \"assert\", \"confirm\",\n \"send\", \"receive\", \"transmit\", \"broadcast\", \"dispatch\",\n \"calculate\", \"compute\", \"evaluate\", \"process\", \"analyze\",\n \"convert\", \"transform\", \"translate\", \"map\",\n \"store\", \"save\", \"load\", \"fetch\", \"retrieve\",\n \"encrypt\", \"decrypt\", \"sign\", \"verify\",\n \"connect\", \"disconnect\", \"reconnect\",\n \"start\", \"stop\", \"pause\", \"resume\", \"cancel\",\n \"lock\", \"unlock\", \"freeze\", \"unfreeze\",\n \"approve\", \"reject\", \"accept\", \"deny\",\n \"compile\", \"generate\", \"build\", \"produce\",\n \"notify\", \"alert\", \"signal\", \"emit\",\n \"apply\", \"execute\", \"run\", \"invoke\",\n \"debit\", \"credit\", \"transfer\", \"exchange\",\n \"suspend\", \"escalate\", \"flag\",\n \"log\", \"record\", \"audit\",\n ]\n words = description.split()\n for word in words:\n if word in action_verbs:\n return word\n # Fallback: first word\n return words[0] if words else \"\"\n\n def _extract_object(self, description: str) -> str:\n \"\"\"Extract the primary object from an action description.\"\"\"\n # Look for entity-like nouns\n financial_objects = [\n \"account\", \"transaction\", \"loan\", \"payment\", \"balance\",\n \"customer\", \"card\", \"deposit\", \"withdrawal\", \"transfer\",\n \"currency\", \"rate\", \"fraud\", \"alert\", \"audit\",\n ]\n general_objects = [\n \"message\", \"data\", \"state\", \"position\", \"score\",\n \"user\", \"player\", \"game\", \"board\", \"piece\",\n \"file\", \"record\", \"entry\", \"item\", \"element\",\n \"connection\", \"session\", \"request\", \"response\",\n ]\n all_objects = financial_objects + general_objects\n\n words = description.split()\n for word in words:\n if word in all_objects:\n return word\n\n # Fallback: noun after the verb\n for i, word in enumerate(words):\n if i > 0 and word not in [\"to\", \"from\", \"with\", \"and\", \"or\", \"the\", \"a\", \"an\", \"in\", \"on\", \"at\", \"for\"]:\n return word\n\n return words[-1] if words else \"data\"\n\n def _classify_category(self, verb: str, obj: str) -> str:\n \"\"\"Classify the action into a pattern category.\"\"\"\n category_map = {\n \"create\": \"CREATION\", \"delete\": \"DELETION\", \"add\": \"UPDATE\",\n \"remove\": \"DELETION\", \"update\": \"UPDATE\", \"modify\": \"UPDATE\",\n \"set\": \"UPDATE\", \"change\": \"UPDATE\",\n \"validate\": \"VALIDATE\", \"verify\": \"VALIDATE\", \"check\": \"VALIDATE\",\n \"send\": \"ROUTE\", \"receive\": \"ROUTE\", \"transmit\": \"BROADCAST\",\n \"broadcast\": \"BROADCAST\", \"dispatch\": \"ROUTE\",\n \"calculate\": \"COMPUTE\", \"compute\": \"COMPUTE\", \"evaluate\": \"COMPUTE\",\n \"convert\": \"TRANSFORM\", \"transform\": \"TRANSFORM\",\n \"store\": \"STORE\", \"save\": \"STORE\", \"load\": \"LOAD\",\n \"fetch\": \"LOAD\", \"retrieve\": \"LOAD\",\n \"encrypt\": \"ENCRYPT\", \"decrypt\": \"ENCRYPT\",\n \"connect\": \"CONNECT\", \"disconnect\": \"DISCONNECT\",\n \"notify\": \"NOTIFY\", \"alert\": \"NOTIFY\",\n \"debit\": \"FINANCIAL\", \"credit\": \"FINANCIAL\",\n \"transfer\": \"FINANCIAL\", \"exchange\": \"FINANCIAL\",\n \"suspend\": \"SECURITY\", \"lock\": \"SECURITY\", \"freeze\": \"SECURITY\",\n \"log\": \"AUDIT\", \"record\": \"AUDIT\", \"audit\": \"AUDIT\",\n }\n return category_map.get(verb, \"GENERAL\")\n\n def _extract_keywords(self, description: str) -> List[str]:\n \"\"\"Extract searchable keywords from a description.\"\"\"\n stop_words = {\"to\", \"from\", \"with\", \"and\", \"or\", \"the\", \"a\", \"an\",\n \"in\", \"on\", \"at\", \"for\", \"is\", \"are\", \"was\", \"were\",\n \"be\", \"been\", \"being\", \"have\", \"has\", \"had\", \"do\",\n \"does\", \"did\", \"will\", \"would\", \"could\", \"should\",\n \"may\", \"might\", \"must\", \"shall\", \"can\", \"need\",\n \"it\", \"its\", \"if\", \"then\", \"else\", \"when\", \"where\",\n \"which\", \"who\", \"whom\", \"this\", \"that\", \"these\", \"those\"}\n words = re.findall(r'\\b\\w+\\b', description.lower())\n return [w for w in words if w not in stop_words and len(w) > 2]\n\n def _generate_template(self, verb: str, obj: str, behavior_name: str) -> str:\n \"\"\"Generate a code template for a learned pattern.\"\"\"\n templates = {\n \"CREATION\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n instance = {obj.title().replace('_', '')}()\n self._logger.info(f\"Created {{}}: {{instance}}\")\n return instance''',\n \"UPDATE\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n {obj}.{verb}_value = new_value\n self._logger.info(f\"{verb} {obj}: {{{obj}.{verb}_value}}\")''',\n \"VALIDATE\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n result = self._validate_{obj}({obj})\n if not result:\n self._logger.warning(f\"Validation failed for {{}}\")\n return result''',\n \"STORE\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n self._storage[\"{obj}\"] = {obj}\n self._logger.info(f\"Stored {obj}\")''',\n \"LOAD\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n data = self._storage.get(\"{obj}\")\n self._logger.info(f\"Loaded {obj}: {{data}}\")\n return data''',\n \"FINANCIAL\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n # {verb} {obj} with full audit trail\n self._logger.info(f\"{verb} {obj}\")\n record = self._create_audit_entry(\"{verb}\", \"{obj}\")\n self._audit_trail.append(record)\n return record''',\n \"NOTIFY\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n for recipient in self._recipients:\n self._notify(recipient, \"{obj}\")\n self._logger.info(f\"Notified about {{obj}}\")''',\n \"AUDIT\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n entry = self._create_audit_entry(\"{verb}\", \"{obj}\")\n self._audit_trail.append(entry)\n self._logger.info(f\"Audit entry created: {{entry}}\")\n return entry''',\n \"SECURITY\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n {obj}.status = \"suspended\"\n self._logger.info(f\"{verb} {obj} for security review\")\n self._notify_security(\"{verb}\", {obj})''',\n \"ROUTE\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n self._send_to(destination, {obj})\n self._logger.info(f\"{verb} {obj} to {{destination}}\")''',\n \"BROADCAST\": f''' \"\"\"Behavior: {behavior_name}\"\"\"\n for recipient in self._recipients:\n self._send_to(recipient, {obj})\n self._logger.info(f\"Broadcast {obj}\")''',\n }\n\n category = self._classify_category(verb, obj)\n return templates.get(category, f''' \"\"\"Behavior: {behavior_name}\"\"\"\n # {verb} {obj}\n self._logger.info(f\"{verb} {obj}\")\n pass''')\n\n def get_learned_pattern(self, behavior_name: str) -> Optional[Dict]:\n \"\"\"Get a learned pattern for a behavior.\"\"\"\n return self.learned_patterns.get(behavior_name)\n\n def export_patterns(self) -> List[Dict]:\n \"\"\"Export all learned patterns for serialization.\"\"\"\n return [\n {**p, \"learned_at\": p[\"learned_at\"]}\n for p in self.learned_patterns.values()\n ]\n\n\nclass SpecEvolver:\n \"\"\"\n Evolve/fix AICL specifications based on diagnostic information.\n\n The SpecEvolver takes verification failures, test failures, and\n audit gaps as input, and produces specification modifications\n that address the root cause — not the symptom.\n\n Every modification is recorded in the provenance chain.\n \"\"\"\n\n def __init__(self):\n self.evolution_log: List[Dict] = []\n\n def evolve_from_verification(self, source: str, report: Dict) -> Tuple[str, List[str]]:\n \"\"\"\n Modify the AICL source based on a verification report.\n Returns (modified_source, list_of_changes).\n \"\"\"\n changes = []\n lines = source.split(\"\\n\")\n\n for check in report.get(\"checks\", []):\n if check.get(\"status\") == \"FAIL\":\n change = self._fix_verification_failure(lines, check)\n if change:\n changes.append(change)\n self.evolution_log.append({\n \"type\": \"verification_fix\",\n \"check\": check,\n \"change\": change,\n \"timestamp\": time.time(),\n })\n\n return \"\\n\".join(lines), changes\n\n def evolve_from_test_failure(self, source: str, diagnosis: Dict) -> Tuple[str, List[str]]:\n \"\"\"\n Modify the AICL source based on a test failure diagnosis.\n Returns (modified_source, list_of_changes).\n \"\"\"\n changes = []\n lines = source.split(\"\\n\")\n\n fix_type = diagnosis.get(\"suggested_fix\")\n\n if fix_type == \"add_missing_imports\":\n change = self._fix_missing_imports(lines, diagnosis)\n if change:\n changes.append(change)\n\n elif fix_type == \"add_missing_attributes\":\n change = self._fix_missing_attributes(lines, diagnosis)\n if change:\n changes.append(change)\n\n elif fix_type == \"fix_validation_logic\":\n change = self._fix_validation_logic(lines, diagnosis)\n if change:\n changes.append(change)\n\n elif fix_type == \"add_missing_definitions\":\n change = self._fix_missing_definitions(lines, diagnosis)\n if change:\n changes.append(change)\n\n elif fix_type == \"fix_type_signatures\":\n change = self._fix_type_signatures(lines, diagnosis)\n if change:\n changes.append(change)\n\n for change in changes:\n self.evolution_log.append({\n \"type\": \"test_failure_fix\",\n \"diagnosis\": diagnosis,\n \"change\": change,\n \"timestamp\": time.time(),\n })\n\n return \"\\n\".join(lines), changes\n\n def evolve_from_audit_gap(self, source: str, audit_report: Dict) -> Tuple[str, List[str]]:\n \"\"\"\n Modify the AICL source to address audit coverage gaps.\n Returns (modified_source, list_of_changes).\n \"\"\"\n changes = []\n lines = source.split(\"\\n\")\n\n orphan_artifacts = audit_report.get(\"orphan_artifacts\", [])\n coverage = audit_report.get(\"audit_coverage\", 0.0)\n\n if coverage < 1.0:\n # Add more specific validations to cover gaps\n change = self._add_coverage_validations(lines, orphan_artifacts, coverage)\n if change:\n changes.append(change)\n self.evolution_log.append({\n \"type\": \"audit_gap_fix\",\n \"coverage\": coverage,\n \"change\": change,\n \"timestamp\": time.time(),\n })\n\n return \"\\n\".join(lines), changes\n\n def _fix_verification_failure(self, lines: List[str], check: Dict) -> Optional[str]:\n \"\"\"Fix a specific verification failure.\"\"\"\n check_name = check.get(\"name\", \"\")\n message = check.get(\"message\", \"\")\n\n if \"risk_recovery_pairing\" in check_name:\n # Risk without recovery — add a generic recovery\n last_risk_idx = -1\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Risk:\"):\n last_risk_idx = i\n\n if last_risk_idx >= 0:\n # Find the next blank line after the risk\n insert_idx = last_risk_idx + 1\n while insert_idx < len(lines) and lines[insert_idx].strip():\n insert_idx += 1\n\n risk_text = lines[last_risk_idx].replace(\"Risk:\", \"\").strip()\n lines.insert(insert_idx, f\"Recovery:\")\n lines.insert(insert_idx + 1, f\"Handle {risk_text.lower()} gracefully\")\n return f\"Added recovery for risk: {risk_text}\"\n\n elif \"entity_present\" in check_name or \"behavior_present\" in check_name:\n # Missing entity or behavior — can't auto-fix, return None\n return None\n\n elif \"action\" in check_name:\n # Behavior without action — add a generic action\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Behavior \"):\n # Check if there's no Action: section\n has_action = False\n for j in range(i + 1, min(i + 10, len(lines))):\n if \"Action:\" in lines[j]:\n has_action = True\n break\n if lines[j].strip() and not lines[j].startswith(\" \") and not lines[j].startswith(\"\\t\"):\n break\n if not has_action:\n behavior_name = line.replace(\"Behavior\", \"\").strip()\n lines.insert(i + 1, \"\")\n lines.insert(i + 2, \"Action:\")\n lines.insert(i + 3, f\" execute {behavior_name.lower()}\")\n return f\"Added action for behavior: {behavior_name}\"\n\n return None\n\n def _fix_missing_imports(self, lines: List[str], diagnosis: Dict) -> Optional[str]:\n \"\"\"Add missing imports — note: this affects generated code, not AICL source.\"\"\"\n # For AICL source, missing imports usually means the Native section needs updating\n for missing in diagnosis.get(\"missing_imports\", []):\n # Check if there's a Native section\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Native:\"):\n return f\"Missing import '{missing}' should be added to Native section\"\n\n return None\n\n def _fix_missing_attributes(self, lines: List[str], diagnosis: Dict) -> Optional[str]:\n \"\"\"Add missing entity fields based on attribute errors.\"\"\"\n fixes = []\n for error in diagnosis.get(\"attribute_errors\", []):\n cls_name = error.get(\"class\", \"\")\n attr_name = error.get(\"attribute\", \"\")\n\n # Find the entity definition and add the missing field\n for i, line in enumerate(lines):\n if line.strip().startswith(f\"Entity {cls_name}\"):\n # Add the missing field\n j = i + 1\n while j < len(lines) and (lines[j].startswith(\" \") or lines[j].startswith(\"\\t\")):\n j += 1\n lines.insert(j, f\" {attr_name}: any\")\n fixes.append(f\"Added field '{attr_name}' to Entity {cls_name}\")\n break\n\n return \"; \".join(fixes) if fixes else None\n\n def _fix_validation_logic(self, lines: List[str], diagnosis: Dict) -> Optional[str]:\n \"\"\"Make validations more specific and testable.\"\"\"\n # Make vague validations more testable\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Validation:\"):\n validation_text = line.replace(\"Validation:\", \"\").strip()\n # Check if the validation is too vague\n vague_patterns = [\"works correctly\", \"functions properly\", \"operates as expected\"]\n for pattern in vague_patterns:\n if pattern in validation_text.lower():\n lines[i] = f\"Validation: {validation_text.replace(pattern, 'produces expected output and returns success')}\"\n return f\"Made validation more specific: {validation_text[:50]}...\"\n\n return None\n\n def _fix_missing_definitions(self, lines: List[str], diagnosis: Dict) -> Optional[str]:\n \"\"\"Add missing entity or behavior definitions.\"\"\"\n fixes = []\n for name in diagnosis.get(\"missing_methods\", []):\n # Check if it might be a missing entity\n entity_name = name.title().replace(\"_\", \"\")\n # Add a minimal entity if it makes sense\n has_entity = any(f\"Entity {entity_name}\" in line for line in lines)\n if not has_entity and len(name) > 3:\n # Add at the end of entities section\n last_entity = -1\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Entity \"):\n last_entity = i\n if last_entity >= 0:\n # Find end of last entity\n j = last_entity + 1\n while j < len(lines) and (lines[j].startswith(\" \") or lines[j].startswith(\"\\t\")):\n j += 1\n lines.insert(j, f\"\\nEntity {entity_name}\")\n lines.insert(j + 1, f\" id: string\")\n lines.insert(j + 2, f\" name: string\")\n fixes.append(f\"Added Entity {entity_name}\")\n\n return \"; \".join(fixes) if fixes else None\n\n def _fix_type_signatures(self, lines: List[str], diagnosis: Dict) -> Optional[str]:\n \"\"\"Fix type signature mismatches in behaviors.\"\"\"\n # Often caused by Behavior Input sections using raw types instead of Entity names\n for error in diagnosis.get(\"type_errors\", []):\n # This is usually a generated code issue, not a spec issue\n pass\n return None\n\n def _add_coverage_validations(self, lines: List[str], orphans: List, coverage: float) -> Optional[str]:\n \"\"\"Add validations to improve audit coverage.\"\"\"\n # Find the last Validation section\n last_validation = -1\n for i, line in enumerate(lines):\n if line.strip().startswith(\"Validation:\"):\n last_validation = i\n\n # Add a general validation for coverage\n if last_validation >= 0:\n lines.insert(last_validation + 1, \"Validation:\")\n lines.insert(last_validation + 2, \"All generated code artifacts have traceable provenance\")\n return f\"Added coverage validation (was {coverage:.1%})\"\n\n return None\n\n\nclass LoopController:\n \"\"\"\n Manage the autonomous compilation loop with convergence detection.\n\n Convergence criteria:\n 1. All spec verifications pass\n 2. Audit coverage = 1.0\n 3. All generated tests pass\n 4. No fallback compilations remain\n 5. Improvement score delta < threshold (0.01)\n \"\"\"\n\n def __init__(self, max_iterations: int = 10, convergence_threshold: float = 0.01):\n self.max_iterations = max_iterations\n self.convergence_threshold = convergence_threshold\n self.iterations: List[LoopIteration] = []\n self.history: List[Dict] = []\n\n @property\n def current_iteration(self) -> int:\n return len(self.iterations)\n\n @property\n def should_continue(self) -> bool:\n \"\"\"Whether the loop should continue iterating.\"\"\"\n if self.current_iteration >= self.max_iterations:\n return False\n if self.is_converged:\n return False\n return True\n\n @property\n def is_converged(self) -> bool:\n \"\"\"Whether the loop has converged.\"\"\"\n if len(self.history) < 2:\n return False\n\n current = self.history[-1]\n previous = self.history[-2]\n\n # All metrics must be at maximum\n if current.get(\"audit_coverage\", 0) < 1.0:\n return False\n if current.get(\"test_pass_rate\", 0) < 1.0:\n return False\n if current.get(\"fallback_count\", 1) > 0:\n return False\n if not current.get(\"spec_verified\", False):\n return False\n\n # Improvement must be below threshold\n delta = abs(\n current.get(\"score\", 0) - previous.get(\"score\", 0)\n )\n return delta < self.convergence_threshold\n\n @property\n def convergence_iteration(self) -> Optional[int]:\n \"\"\"The iteration at which convergence was reached.\"\"\"\n for i, h in enumerate(self.history):\n if (\n h.get(\"audit_coverage\", 0) >= 1.0\n and h.get(\"test_pass_rate\", 0) >= 1.0\n and h.get(\"fallback_count\", 1) == 0\n and h.get(\"spec_verified\", False)\n ):\n return i + 1\n return None\n\n def record_iteration(self, iteration: LoopIteration):\n \"\"\"Record a completed iteration.\"\"\"\n self.iterations.append(iteration)\n self.history.append({\n \"iteration\": iteration.iteration,\n \"audit_coverage\": iteration.audit_coverage,\n \"test_pass_rate\": iteration.tests_passed / max(iteration.tests_total, 1),\n \"fallback_count\": iteration.fallback_count,\n \"spec_verified\": iteration.tests_failed == 0 and iteration.audit_coverage >= 1.0,\n \"score\": self._compute_score(iteration),\n \"patterns_learned\": len(iteration.patterns_learned),\n \"fixes_applied\": len(iteration.fixes_applied),\n })\n\n def _compute_score(self, iteration: LoopIteration) -> float:\n \"\"\"Compute a composite score for an iteration.\"\"\"\n coverage_score = iteration.audit_coverage\n test_score = iteration.tests_passed / max(iteration.tests_total, 1)\n fallback_penalty = iteration.fallback_count * 0.05\n return max(0, (coverage_score + test_score) / 2 - fallback_penalty)\n\n def get_progress(self) -> Dict[str, Any]:\n \"\"\"Get current progress information.\"\"\"\n if not self.history:\n return {\"status\": \"starting\", \"iteration\": 0}\n\n current = self.history[-1]\n return {\n \"status\": \"converged\" if self.is_converged else \"iterating\",\n \"iteration\": self.current_iteration,\n \"max_iterations\": self.max_iterations,\n \"score\": current.get(\"score\", 0),\n \"audit_coverage\": current.get(\"audit_coverage\", 0),\n \"test_pass_rate\": current.get(\"test_pass_rate\", 0),\n \"fallback_count\": current.get(\"fallback_count\", 0),\n \"total_patterns_learned\": sum(h.get(\"patterns_learned\", 0) for h in self.history),\n \"total_fixes_applied\": sum(h.get(\"fixes_applied\", 0) for h in self.history),\n }\n\n\nclass AutonomousCompiler:\n \"\"\"\n The self-writing, self-validating compilation engine.\n\n Orchestrates the full autonomous loop:\n SPEC → COMPILE → VERIFY → TEST → DIAGNOSE → FIX → RECOMPILE\n\n Every step preserves the No-Orphan Property. Every fix is recorded\n in the provenance chain. The loop converges when all criteria are met.\n\n Usage:\n compiler = AutonomousCompiler(max_iterations=10)\n result = compiler.evolve(\"banking.aicl\")\n print(result.summary)\n \"\"\"\n\n def __init__(self, max_iterations: int = 10, target_language: str = \"python\",\n test_timeout: int = 30, convergence_threshold: float = 0.01):\n self.max_iterations = max_iterations\n self.target_language = target_language\n self.test_runner = TestRunner(timeout=test_timeout)\n self.pattern_learner = PatternLearner()\n self.spec_evolver = SpecEvolver()\n self.loop_controller = LoopController(\n max_iterations=max_iterations,\n convergence_threshold=convergence_threshold,\n )\n self._provenance_log: List[Dict] = []\n\n def evolve(self, source_path: str, output_dir: Optional[str] = None) -> AutonomousResult:\n \"\"\"\n Run the autonomous compilation loop on an AICL source file.\n\n Args:\n source_path: Path to the .aicl source file\n output_dir: Output directory (defaults to temp dir)\n\n Returns:\n AutonomousResult with convergence details\n \"\"\"\n start_time = time.time()\n\n # Read source\n with open(source_path) as f:\n source = f.read()\n\n original_source = source\n\n if output_dir is None:\n output_dir = tempfile.mkdtemp(prefix=\"aicl_evolve_\")\n\n # Import compiler here to avoid circular imports\n from . import Compiler, Parser\n\n spec_verified = False\n\n while self.loop_controller.should_continue:\n iteration_num = self.loop_controller.current_iteration + 1\n iter_start = time.time()\n\n iteration = LoopIteration(\n iteration=iteration_num,\n state_entered=LoopState.COMPILING,\n timestamp_start=iter_start,\n )\n\n # === STEP 1: COMPILE ===\n iteration.state_entered = LoopState.COMPILING\n try:\n compiler = Compiler(target_language=self.target_language)\n result = compiler.compile_to_file(source, output_dir, source_path=source_path)\n\n if not result.success:\n iteration.errors.append(f\"Compilation failed: {'; '.join(result.errors)}\")\n iteration.state_exited = LoopState.FAILED\n iteration.timestamp_end = time.time()\n self.loop_controller.record_iteration(iteration)\n continue\n\n # Parse for verification step\n try:\n parsed = Parser().parse(source)\n except Exception:\n parsed = None\n\n # Count fallbacks from provenance\n fallback_count = 0\n if hasattr(compiler, '_provenance') and compiler._provenance:\n for rec in compiler._provenance.records:\n if rec.source_type == ProvenanceType.FALLBACK:\n fallback_count += 1\n # Record for pattern learning\n self.pattern_learner.record_unmatched(\n rec.source_text[:50] if rec.source_text else \"unknown\",\n rec.generated_code[:200] if rec.generated_code else \"\"\n )\n\n iteration.fallback_count = fallback_count\n\n # Get audit coverage\n if hasattr(compiler, '_provenance') and compiler._provenance:\n audit = compiler._provenance.compute_audit_coverage()\n iteration.audit_coverage = audit.get(\"audit_coverage\", 0.0)\n\n except Exception as e:\n iteration.errors.append(f\"Compilation exception: {str(e)}\")\n iteration.state_exited = LoopState.FAILED\n iteration.timestamp_end = time.time()\n self.loop_controller.record_iteration(iteration)\n continue\n\n # === STEP 2: VERIFY SPEC ===\n iteration.state_entered = LoopState.VERIFYING\n try:\n from .spec_verify import SpecificationVerifier\n if parsed is not None:\n verifier = SpecificationVerifier(parsed)\n report = verifier.verify(source_name=source_path)\n\n spec_verified = all(\n c.get(\"status\") != \"FAIL\"\n for c in report.get(\"checks\", [])\n )\n\n if not spec_verified:\n # Auto-fix spec verification failures\n source, changes = self.spec_evolver.evolve_from_verification(source, report)\n iteration.spec_changes.extend(changes)\n iteration.fixes_applied.extend(changes)\n else:\n spec_verified = False\n\n except Exception as e:\n iteration.errors.append(f\"Verification exception: {str(e)}\")\n spec_verified = False\n\n # === STEP 3: RUN TESTS ===\n iteration.state_entered = LoopState.TESTING\n try:\n passed, failed, total, failures = self.test_runner.run_tests(output_dir)\n iteration.tests_passed = passed\n iteration.tests_failed = failed\n iteration.tests_total = total\n\n if failed > 0:\n # === STEP 4: DIAGNOSE FAILURES ===\n iteration.state_entered = LoopState.DIAGNOSING\n for failure in failures:\n test_name = failure.get(\"test\", \"unknown\")\n diagnosis = self.test_runner.analyze_failure(test_name, output_dir)\n\n # === STEP 5: FIX ===\n iteration.state_entered = LoopState.FIXING\n source, changes = self.spec_evolver.evolve_from_test_failure(source, diagnosis)\n iteration.fixes_applied.extend(changes)\n\n except Exception as e:\n iteration.errors.append(f\"Test execution exception: {str(e)}\")\n iteration.tests_total = 1\n iteration.tests_failed = 1\n\n # === STEP 6: LEARN PATTERNS ===\n if iteration.fallback_count > 0:\n iteration.state_entered = LoopState.LEARNING\n for unmatched in self.pattern_learner.unmatched_descriptions:\n parts = unmatched.split(\": \", 1)\n if len(parts) == 2:\n pattern = self.pattern_learner.learn_from_description(parts[0], parts[1])\n if pattern:\n iteration.patterns_learned.append(pattern[\"name\"])\n\n # Clear after learning\n self.pattern_learner.unmatched_descriptions.clear()\n\n # === STEP 7: CHECK AUDIT COVERAGE ===\n if iteration.audit_coverage < 1.0:\n iteration.state_entered = LoopState.FIXING\n source, changes = self.spec_evolver.evolve_from_audit_gap(\n source,\n {\n \"audit_coverage\": iteration.audit_coverage,\n \"orphan_artifacts\": [],\n }\n )\n iteration.fixes_applied.extend(changes)\n\n # Record iteration\n iteration.state_exited = (\n LoopState.CONVERGED if iteration.success else LoopState.DIAGNOSING\n )\n iteration.timestamp_end = time.time()\n self.loop_controller.record_iteration(iteration)\n\n # Record provenance for this iteration\n self._provenance_log.append({\n \"iteration\": iteration_num,\n \"duration\": iteration.duration,\n \"audit_coverage\": iteration.audit_coverage,\n \"tests_passed\": iteration.tests_passed,\n \"tests_failed\": iteration.tests_failed,\n \"fallback_count\": iteration.fallback_count,\n \"patterns_learned\": iteration.patterns_learned,\n \"fixes_applied\": iteration.fixes_applied,\n \"spec_changes\": iteration.spec_changes,\n })\n\n # === FINAL: Verify proof ===\n proof_valid = False\n proof_path = os.path.join(output_dir, \"main.aicl-proof\")\n if os.path.exists(proof_path):\n try:\n from .provenance import ProofOfOrigin\n proof = ProofOfOrigin.from_file(proof_path)\n valid, checks = proof.verify()\n proof_valid = valid\n except Exception:\n proof_valid = False\n\n # Compute final metrics\n total_duration = time.time() - start_time\n final_history = self.loop_controller.history\n final_coverage = final_history[-1].get(\"audit_coverage\", 0) if final_history else 0\n final_pass_rate = final_history[-1].get(\"test_pass_rate\", 0) if final_history else 0\n\n # If source was modified, write it back\n if source != original_source:\n evolved_path = source_path.replace(\".aicl\", \".evolved.aicl\")\n with open(evolved_path, \"w\") as f:\n f.write(source)\n\n return AutonomousResult(\n converged=self.loop_controller.is_converged,\n iterations=self.loop_controller.iterations,\n total_duration=total_duration,\n final_audit_coverage=final_coverage,\n final_test_pass_rate=final_pass_rate,\n patterns_learned=[\n p[\"name\"] for p in self.pattern_learner.learned_patterns.values()\n ],\n spec_evolution=[\n entry[\"change\"] for entry in self.spec_evolver.evolution_log\n ],\n convergence_iteration=self.loop_controller.convergence_iteration,\n proof_valid=proof_valid,\n )\n\n\ndef format_evolve_report(result: AutonomousResult) -> str:\n \"\"\"Format an AutonomousResult into a readable report.\"\"\"\n lines = [\n \"=\" * 70,\n \"AICL AUTONOMOUS COMPILATION REPORT\",\n \"=\" * 70,\n \"\",\n f\" Converged: {result.converged}\",\n f\" Total Duration: {result.total_duration:.2f}s\",\n f\" Iterations: {len(result.iterations)}\",\n f\" Audit Coverage: {result.final_audit_coverage:.1%}\",\n f\" Test Pass Rate: {result.final_test_pass_rate:.1%}\",\n f\" Proof Valid: {result.proof_valid}\",\n \"\",\n ]\n\n if result.convergence_iteration is not None:\n lines.append(f\" Converged at: Iteration {result.convergence_iteration}\")\n else:\n lines.append(f\" Converged at: N/A (did not converge)\")\n\n lines.append(\"\")\n\n # Patterns learned\n if result.patterns_learned:\n lines.append(\"PATTERNS LEARNED:\")\n for p in result.patterns_learned:\n lines.append(f\" - {p}\")\n lines.append(\"\")\n\n # Spec evolution\n if result.spec_evolution:\n lines.append(\"SPECIFICATION EVOLUTION:\")\n for change in result.spec_evolution:\n lines.append(f\" - {change}\")\n lines.append(\"\")\n\n # Iteration details\n lines.append(\"ITERATION DETAILS:\")\n lines.append(\"-\" * 70)\n for it in result.iterations:\n status = \"✓ CONVERGED\" if it.success else \"→ ITERATING\"\n lines.append(\n f\" [{it.iteration}] {status} \"\n f\"coverage={it.audit_coverage:.1%} \"\n f\"tests={it.tests_passed}/{it.tests_total} \"\n f\"fallbacks={it.fallback_count} \"\n f\"fixes={len(it.fixes_applied)} \"\n f\"patterns={len(it.patterns_learned)} \"\n f\"({it.duration:.2f}s)\"\n )\n if it.errors:\n for err in it.errors[:3]:\n lines.append(f\" ⚠ {err[:80]}\")\n lines.append(\"-\" * 70)\n lines.append(\"\")\n\n return \"\\n\".join(lines)\n</pre></body></html>", |
