Cleanup: Remove source code, deploy markers, and internal docs

CHANGES_EXPLANATION.md

@@ -1,382 +0,0 @@
-# Chute Template Changes - Current State
-
-**Last Updated:** 2025-11-16  
-**Branch:** develop (comparing to main)
-
-This document explains the minimal essential changes applied to the chute template files to fix critical issues.
-
----
-
-## ⚠️ Template Injection Constraints
-
-The `chute.py.j2` template only injects these specific files:
-- `{{ schema_defs }}` - schemas.py
-- `{{ setup_utils }}` - setup.py  
-- `{{ load_utils }}` - load.py
-- `{{ predict_utils }}` - predict.py
-
-**Only these files can be modified.** New files require updating helper code.
-
----
-
-## Changes Applied
-
-### 1. chute.py.j2 - Fix 400 Errors & Add Logging
-
-**Priority 1 & 3: CRITICAL + Logging**
-
-**Problem:** Validators send JSON dicts, but Chutes `@chute.cord()` decorator doesn't auto-parse to Pydantic models (unlike FastAPI). This caused 400 Bad Request errors.
-
-**Solution:**
-
-```python
-async def predict(self, data: BBPredictedUtterance = None, **kwargs) -> dict:
-    try:
-        # Logging
-        print(f"[PREDICT] Received type: {type(data)}, kwargs: {list(kwargs.keys()) if kwargs else []}")
-        
-        # Handle dict input from validators
-        if data is None and kwargs:
-            data = BBPredictedUtterance.model_validate(kwargs)
-            print(f"[PREDICT] ✓ Parsed from kwargs")
-        elif isinstance(data, dict):
-            data = BBPredictedUtterance.model_validate(data)
-            print(f"[PREDICT] ✓ Converted dict to object")
-        elif not isinstance(data, BBPredictedUtterance):
-            print(f"[PREDICT] ❌ Invalid type: {type(data)}")
-            return {"success": False, "error": f"Invalid data type: {type(data)}"}
-        
-        # Call prediction
-        print(f"[PREDICT] Calling _predict...")
-        result = _predict(model=self.model, data=data, model_name="{{ chute_name }}")
-        print(f"[PREDICT] ✓ Success")
-        return result.model_dump(mode="json")
-    except Exception as e:
-        print(f"[PREDICT] ❌ Error: {e}")
-        return {"success": False, "error": str(e)}
-```
-
-**What changed:**
-- Function signature: `data: BBPredictedUtterance` → `data: BBPredictedUtterance = None, **kwargs`
-- Added isinstance checks to convert dict → Pydantic object
-- Added logging at every step for debugging
-- Added try/except with structured error responses
-
-**Impact:**
-- ✅ Fixes 400 Bad Request errors from validators
-- ✅ Provides debugging visibility in production
-- ✅ Graceful error handling
-
----
-
-### 2. load.py - Fix Cache Permissions & Add Logging
-
-**Priority 2 & 4: CRITICAL + Logging**
-
-**Problem:** Default cache location `/cache/hub` is read-only in Chutes containers, causing PermissionError during model downloads.
-
-**Solution:**
-
-```python
-def _load_model(repo_name: str, revision: str):
-    try:
-        # Fix cache permissions - use writable cache directory
-        import os
-        from pathlib import Path
-        
-        cache_dir = './huggingface_cache'
-        
-        # Logging
-        print(f"[LOAD] Setting up cache: {cache_dir}")
-        
-        # Create cache directory
-        Path(cache_dir).mkdir(parents=True, exist_ok=True)
-        
-        # Set environment variables
-        os.environ['HF_HOME'] = cache_dir
-        os.environ['HF_HUB_CACHE'] = cache_dir
-        os.environ['TRANSFORMERS_CACHE'] = cache_dir
-        print(f"[LOAD] ✓ Environment configured")
-        
-        print(f"[LOAD] Downloading model from HuggingFace Hub...")
-        model_path = snapshot_download(
-            repo_name, 
-            revision=revision,
-            cache_dir=cache_dir
-        )
-        print(f"[LOAD] ✓ Downloaded to: {model_path}")
-        
-        model = load_model_from_huggingface_hub(model_path=model_path)
-        print(f"[LOAD] ✓ Model loaded successfully")
-        return model
-
-    except Exception as e:
-        print(f"[LOAD] ❌ Failed: {e}")
-        raise
-```
-
-**What changed:**
-- Added `cache_dir = './huggingface_cache'` (writable, isolated per container)
-- Set HF environment variables to use custom cache
-- Pass `cache_dir` explicitly to `snapshot_download()`
-- Added logging for cache setup, download, and model loading
-
-**Why relative path:**
-- Each container instance has its own working directory
-- Automatically isolated (no race conditions)
-- Writable (not a shared read-only mount)
-
-**Impact:**
-- ✅ Fixes PermissionError during model downloads
-- ✅ Eliminates race conditions between container instances
-- ✅ Better debugging visibility
-
----
-
-### 3. setup.py - Configuration Updates
-
-**Priority 5: OPTIONAL but Recommended**
-
-**Changes:**
-
-```python
-# Pin chutes version for reproducibility
-"pip install transformers pydantic chutes==0.3.60"
-
-# Increase VRAM for faster queue (less competition)
-min_vram_gb_per_gpu=24,  # was 16
-
-# Increase hot time to prevent cooldowns during testing
-shutdown_after_seconds=36000,  # 10 hours, was 3600 (1 hour)
-```
-
-**Why each change:**
-
-1. **Pin chutes==0.3.60**
-   - Ensures consistent behavior across deployments
-   - Prevents breaking changes from new versions
-   - Reproducible builds
-
-2. **24GB VRAM (was 16GB)**
-   - Less competition for high-VRAM nodes
-   - Faster queue times
-   - Still widely available (A5000, A6000, 3090, 4090)
-
-3. **10 hours hot time (was 1 hour)**
-   - No unexpected cooldowns during testing
-   - Validators can reach chute consistently
-   - Can reduce to 4-7 hours for production
-
-**Impact:**
-- ✅ Stable, reproducible deployments
-- ✅ Faster queue times
-- ✅ No cooldowns during development/testing
-
----
-
-### 4. predict.py - No Changes
-
-**Status:** Kept original from main branch
-
-**Why no rewrite:**
-- Original implementation is complex but handles edge cases well
-- Has prompt caching for performance
-- Has CUDA fallback logic
-- Has been tested more thoroughly
-- Can add logging later if needed without full rewrite
-
-**If logging needed in future:**
-```python
-# Add these 3 lines to original predict.py:
-print(f"[PREDICT] Prompt: {prompt[:100]}...")      # After prompt construction
-print(f"[PREDICT] Generated: {generated_text[:100]}...")  # After generation
-print(f"[PREDICT] Final: {prediction[:100]}...")   # Before return
-```
-
----
-
-## Summary Table
-
-| Priority | File | Change | Status |
-|----------|------|--------|--------|
-| 1 | `chute.py.j2` | isinstance check + dict conversion | ✅ **MUST HAVE** |
-| 2 | `load.py` | Cache directory fix | ✅ **MUST HAVE** |
-| 3 | `chute.py.j2` | Logging in predict endpoint | ✅ Highly Recommended |
-| 4 | `load.py` | Logging in load | ✅ Recommended |
-| 5 | `setup.py` | Config updates (version, VRAM, hot time) | ✅ Recommended |
-
-**Files unchanged:**
-- ✅ `predict.py` - Original kept (handles edge cases better)
-- ✅ `schemas.py` - No changes needed
-
-**Files removed:**
-- ❌ `preload_model.py` - Not in template injection
-- ❌ `fixed_deploy.py` - Not in template injection
-
----
-
-## Testing After Deployment
-
-### 1. Test Dict Input Handling
-```bash
-bb -v ping-chute --revision your-hf-sha
-```
-Look for in logs:
-- `[PREDICT] Received type:` 
-- `[PREDICT] ✓ Converted dict to object`
-- `[PREDICT] ✓ Success`
-
-### 2. Verify Cache Works
-Look for in logs:
-- `[LOAD] Setting up cache: ./huggingface_cache`
-- `[LOAD] ✓ Environment configured`
-- `[LOAD] ✓ Model loaded successfully`
-- No PermissionError
-
-### 3. Monitor Predictions
-Check logs show:
-- Input type and kwargs
-- Conversion steps
-- Success indicators
-- No 400 errors from validators
-
-### 4. Get Chute Logs
-```bash
-# Via API
-curl -XGET https://api.chutes.ai/instances/<INSTANCE-ID>/logs \
-  -H "Authorization: <CHUTES-API-KEY>"
-
-# Or via dashboard
-# 1. Log into chutes.ai
-# 2. Go to "My Chutes"
-# 3. Click your chute → "Statistics" tab
-# 4. View logs
-```
-
----
-
-## Quick Reference: What Changed vs Main
-
-### chute.py.j2
-```diff
-- async def predict(self, data: BBPredictedUtterance) -> dict:
--     return _predict(...)
-
-+ async def predict(self, data: BBPredictedUtterance = None, **kwargs) -> dict:
-+     try:
-+         # Handle dict input + logging
-+         if isinstance(data, dict):
-+             data = BBPredictedUtterance.model_validate(data)
-+         result = _predict(...)
-+         return result.model_dump(mode="json")
-+     except Exception as e:
-+         return {"success": False, "error": str(e)}
-```
-
-### load.py
-```diff
-  def _load_model(repo_name: str, revision: str):
-+     import os
-+     cache_dir = './huggingface_cache'
-+     os.environ['HF_HOME'] = cache_dir
-+     os.environ['HF_HUB_CACHE'] = cache_dir
-+     os.environ['TRANSFORMERS_CACHE'] = cache_dir
-+     
--     model_path = snapshot_download(repo_name, revision=revision)
-+     model_path = snapshot_download(repo_name, revision=revision, cache_dir=cache_dir)
-```
-
-### setup.py
-```diff
-- "pip install transformers pydantic chutes"
-+ "pip install transformers pydantic chutes==0.3.60"
-
-- min_vram_gb_per_gpu=16,
-+ min_vram_gb_per_gpu=24,
-
-- shutdown_after_seconds=3600,
-+ shutdown_after_seconds=36000,
-```
-
----
-
-## Common Issues & Solutions
-
-### Issue: Still getting 400 errors
-**Check:**
-- Look for `[PREDICT] Received type:` in logs
-- Verify `[PREDICT] ✓ Converted dict to object` appears
-- If not, check validator is sending proper JSON
-
-### Issue: Model fails to load
-**Check:**
-- Look for `[LOAD] ✓ Environment configured` in logs
-- Verify no PermissionError appears
-- Check disk space in container
-- Verify HuggingFace credentials if using private repo
-
-### Issue: Slow predictions
-**Check:**
-- Time in logs shows which step is slow
-- Original predict.py has caching for performance
-- Consider if model size matches VRAM
-
-### Issue: Chute keeps cooling down
-**Check:**
-- Verify `shutdown_after_seconds=36000` in setup.py
-- Consider reducing to 7200 (2h) if cost is concern
-- Ensure chute receives regular requests
-
----
-
-## Why These Changes
-
-### The Core Problem
-1. **400 errors** - Validators send dict, Chutes doesn't auto-parse
-2. **PermissionError** - Default cache is read-only
-3. **No visibility** - Hard to debug production issues
-
-### The Solution
-1. **isinstance check** - Convert dict to Pydantic object
-2. **Custom cache** - Use writable directory
-3. **Logging** - Track what's happening at each step
-
-### The Result
-- ✅ Miners can receive validator requests
-- ✅ Models load without permission errors
-- ✅ Production issues can be debugged from logs
-- ✅ Stable, reproducible deployments
-
----
-
-## For Future Reference
-
-### If You Need to Add More Logging
-
-**In chute.py.j2:**
-```python
-# Add after any critical operation
-print(f"[PREDICT] Your message here: {relevant_data}")
-```
-
-**In load.py:**
-```python
-# Add at key points
-print(f"[LOAD] Your message here: {relevant_data}")
-```
-
-### If You Need to Revert
-
-To revert to main branch state:
-```bash
-git checkout main -- babelbit/chute_template/
-```
-
-To see what changed:
-```bash
-git diff main develop -- babelbit/chute_template/
-```
-
----
-
-**Document Status:** Updated to reflect current develop branch state (priorities 1-5 applied)

DEPLOYMENT_INFO.txt

@@ -1 +0,0 @@
-Iteration C - Final Deploy 1763388932

DEPLOYMENT_VERSION.txt

@@ -1 +0,0 @@
-mistral-4bit-fixed-1761307522

DEPLOY_20251024_062357.txt

@@ -1,3 +0,0 @@
-Deployment: 2025-10-24 06:23:57 +03
-Model: DistilGPT-2
-Code: Famous Ox V2

DEPLOY_20251024_151649.txt

@@ -1 +0,0 @@
-mistral-4bit-fixed-bitsandbytes-1761308209

DEPLOY_V3.txt

@@ -1 +0,0 @@
-mistral-4bit-v3-smart-stable-1761326333

DEPLOY_V4_FINAL.txt

@@ -1 +0,0 @@
-mistral-4bit-v4-runpod-fixed-final-1761380615

DEPLOY_V4_RUNPOD.txt

@@ -1 +0,0 @@
-mistral-4bit-v4-runpod-fixed-1761377993

DEPLOY_V5_VRAM24.txt

@@ -1 +0,0 @@
-mistral-4bit-v5-vram24-1761386314

DEPLOY_V6_FIXED_DEPS.txt

@@ -1,35 +0,0 @@
-DEPLOYMENT V6: Fixed Dependencies
-=================================
-
-Date: 2025-10-25 14:30 UTC
-Status: Ready for deployment
-
-Changes from V5:
-- Added explicit version constraints for scipy, sentencepiece, protobuf
-- All dependencies now have fixed versions for stability
-
-Complete dependency list:
-- numpy<2
-- transformers==4.36.2
-- bitsandbytes==0.41.3
-- accelerate==0.25.0
-- huggingface_hub==0.19.4
-- scipy>=1.11.0,<2.0
-- sentencepiece>=0.1.99,<1.0
-- protobuf>=3.20.0,<5.0
-
-Configuration:
-- VRAM: 24GB (RTX 3090/4090/A5000)
-- Base image: parachutes/python:3.12 (Debian 12)
-- Python: 3.12
-
-All RunPod fixes applied:
-✅ typing imports (Any, Dict)
-✅ snapshot_download import
-✅ use_fast=False for tokenizer
-✅ All implicit dependencies included
-✅ Version conflicts resolved
-✅ VRAM increased to 24GB
-
-Ready for Chutes deployment!
-

DEPLOY_V6_FIXED_SHELL.txt

@@ -1,23 +0,0 @@
-DEPLOYMENT V6: Fixed Shell Escaping
-====================================
-
-Date: 2025-10-25 20:03 UTC
-Status: Ready for deployment
-Version: Mistral-7B-4bit V6 "Fixed Shell"
-
-КРИТИЧЕСКОЕ ИСПРАВЛЕНИЕ:
-=========================
-
-Проблема: numpy<2 интерпретировался Shell как редирект
-Решение: Экранирование 'numpy<2' в кавычках
-
-setup.py изменение:
--------------------
-БЫЛО:  "numpy<2 "
-СТАЛО: "'numpy<2' "
-
-Это предотвращает ошибку:
-/bin/sh: 1: cannot open 2: No such file
-
-Все остальные зависимости без изменений.
-

DEPLOY_V6_GOLDEN_STANDARD.txt

@@ -1,182 +0,0 @@
-DEPLOYMENT V6: Golden Standard Dependencies
-============================================
-
-Date: 2025-10-25 14:32 UTC
-Status: Ready for deployment
-Version: Mistral-7B-4bit V6 "Golden Standard"
-
-===========================================
-ФИНАЛЬНАЯ КОНФИГУРАЦИЯ ЗАВИСИМОСТЕЙ
-===========================================
-
-Эта конфигурация является результатом полной отладки на RunPod и включает:
-✅ Все исправления из RunPod тестирования
-✅ Все неявные зависимости
-✅ Проверенные и стабильные версии
-✅ Совместимость с CUDA 12.1 в образе Chutes
-
--------------------------------------------
-"ЗОЛОТОЙ СТАНДАРТ" ВЕРСИЙ (отлажены на RunPod):
--------------------------------------------
-
-numpy<2                   # Fix: Избегаем NumPy 2.x несовместимости
-transformers==4.36.2      # Fix: Совместима с torch 2.x и bitsandbytes
-bitsandbytes==0.41.3      # Fix: 4-bit quantization, CUDA 12.1 support
-accelerate==0.25.0        # Fix: Device mapping для multi-GPU
-huggingface_hub==0.19.4   # Fix: Стабильная версия для download
-
--------------------------------------------
-"СКРЫТЫЕ" ЗАВИСИМОСТИ (найдены на RunPod):
--------------------------------------------
-
-scipy                     # Fix: Неявная зависимость bitsandbytes
-sentencepiece             # Fix: Требуется для tokenization
-protobuf                  # Fix: Требуется для serialization
-
-ВАЖНО: Эти пакеты НЕ были в requirements изначально, но их отсутствие
-вызывало ModuleNotFoundError при запуске модели!
-
--------------------------------------------
-ОСТАЛЬНЫЕ ЗАВИСИМОСТИ:
--------------------------------------------
-
-torch                     # pip установит совместимую версию
-substrate-interface
-pydantic>=2
-httpx
-python-dotenv>=0.21.0
-aiohttp>=3.9
-Pillow>=10.0
-opencv-python>=4.8
-click>=8.0.0
-bittensor
-jinja2>=3.1.6
-chutes>=0.3.33
-aiobotocore==2.13.1
-pynacl>=1.5
-fastapi
-uvicorn
-petname
-requests>=2.32.5
-asyncpg>=0.29.0
-boto3>=1.34.131
-openai>=2.1.0
-dotenv>=0.9.9
-
-===========================================
-КОНФИГУРАЦИЯ ОКРУЖЕНИЯ
-===========================================
-
-Base Docker Image:    parachutes/python:3.12
-OS:                   Debian 12 "Bookworm"
-Python:               3.12
-CUDA:                 12.1 (предустановлена в образе)
-VRAM:                 24GB (RTX 3090/4090/A5000)
-
-===========================================
-ПОЛНЫЙ СПИСОК ИСПРАВЛЕНИЙ (8 из RunPod)
-===========================================
-
-1. ✅ Import typing (Any, Dict)
-   Файл: load.py
-   Было: отсутствовал импорт
-   Стало: from typing import Any, Dict
-
-2. ✅ Import snapshot_download
-   Файл: load.py
-   Было: отсутствовал импорт
-   Стало: from huggingface_hub import snapshot_download
-
-3. ✅ Type hints совместимость
-   Файл: load.py
-   Было: dict[str, Any]
-   Стало: Dict[str, Any]
-
-4. ✅ Конфликты версий transformers/torch
-   Файл: pyproject.toml, setup.py
-   Было: transformers>=4.56.0
-   Стало: transformers==4.36.2
-
-5. ✅ NumPy 2.x несовместимость
-   Файл: pyproject.toml, setup.py
-   Было: numpy>=1.24
-   Стало: numpy<2
-
-6. ✅ Отсутствующие неявные зависимости
-   Файл: pyproject.toml, setup.py
-   Было: отсутствовали scipy, sentencepiece, protobuf
-   Стало: добавлены все три пакета
-
-7. ✅ Tokenizer crash (PyPreTokenizerTypeWrapper)
-   Файл: load.py
-   Было: AutoTokenizer.from_pretrained(model_path)
-   Стало: AutoTokenizer.from_pretrained(model_path, use_fast=False)
-
-8. ✅ Недостаточно VRAM
-   Файл: setup.py
-   Было: min_vram_gb_per_gpu=16
-   Стало: min_vram_gb_per_gpu=24
-
-===========================================
-ПОЧЕМУ ЭТОТ СПИСОК ДОЛЖЕН СРАБОТАТЬ
-===========================================
-
-1. Учтены все находки RunPod:
-   ✅ Включены scipy, sentencepiece, protobuf
-   ✅ Исправлены все ModuleNotFoundError
-
-2. Проверенные версии:
-   ✅ Стабильные версии transformers, bitsandbytes, accelerate
-   ✅ Протестированы на RunPod с RTX 3090
-
-3. Совместимость с CUDA 12.1:
-   ✅ bitsandbytes==0.41.3 поддерживает CUDA 12.1
-   ✅ torch автоматически выберет совместимую версию
-
-4. Полная воспроизводимость:
-   ✅ Все версии зафиксированы (где критично)
-   ✅ Избегаем breaking changes в будущем
-
-===========================================
-ИСТОРИЯ ДЕПЛОЕВ
-===========================================
-
-V1 (Holy Boxer):    0 instances - ImportError
-V2 (Nice Mako):     0 instances - auto-detection failed
-V3 (Funny Bison):   0 instances - missing imports
-V4 (Causal Dassie): 6 CRASHED - version conflicts
-V5 (Poetic Jaguar): 0 instances - недостаточно VRAM / balance issue
-V6 (Golden Standard): ??? - все исправления применены
-
-===========================================
-ОЖИДАЕМЫЙ РЕЗУЛЬТАТ V6
-===========================================
-
-✅ Instances должны создаться (не 0)
-✅ Instances должны запуститься (не CRASHED)
-✅ Chute должен перейти в HOT status
-✅ Model должна загрузиться успешно
-
-Если V6 провалится, это укажет на проблемы с инфраструктурой Chutes,
-а НЕ с кодом (так как все найденные ошибки исправлены).
-
-===========================================
-КОМАНДА ДЕПЛОЯ
-===========================================
-
-cd /Users/vitalistreliuk/BITTENSOR/SN59 && \
-bb -vv push --model-path ./test_model_mistral4bit
-
-===========================================
-NEXT STEPS ЕСЛИ V6 ПРОВАЛИТСЯ
-===========================================
-
-1. Проверить баланс Chutes (может быть недостаточно средств)
-2. Проверить доступность GPU с 24GB VRAM
-3. Запросить логи через support Chutes
-4. Провести полное тестирование на RunPod с parachutes/python:3.12
-
-===========================================
-
-Готов к deployment! 🚀
-

RAG_IMPLEMENTATION.md

@@ -1,357 +0,0 @@
-# RAG-Based Chute Template - Implementation Complete
-
-**Branch:** `rag_develop`  
-**Date:** 2025-11-17  
-**Status:** ✅ Complete and Ready for Testing
-
----
-
-## Overview
-
-The RAG-based chute template has been successfully implemented, transforming the system from transformer-based text generation to FAISS index-based retrieval. This enables faster, more efficient utterance prediction using pre-built dialogue indexes.
-
----
-
-## What Changed
-
-### 1. Core Template Files (`babelbit/chute_template/`)
-
-#### ✅ `retriever.py` (NEW)
-- Implements `UtteranceRetriever` class for FAISS-based similarity search
-- Handles query construction, embedding generation, and result ranking
-- Includes comprehensive logging for debugging
-- **Lines:** ~250
-
-#### ✅ `load.py` (REPLACED)
-- Downloads `model.index` and `model.data` from HuggingFace
-- Uses `hf_hub_download()` for efficient caching
-- Initializes `UtteranceRetriever` with configuration
-- Supports environment variable overrides (`RAG_CACHE_REPO`, `RAG_CACHE_REVISION`)
-- **Lines:** ~170
-
-#### ✅ `predict.py` (REPLACED)
-- Uses `retriever.retrieve_top1()` instead of text generation
-- Extracts continuations from matched utterances
-- Handles dict input conversion (Chutes compatibility)
-- Returns `BBPredictOutput` with similarity scores
-- **Lines:** ~200
-
-#### ✅ `setup.py` (UPDATED)
-- Added: `sentence-transformers==2.2.2`, `faiss-cpu==1.7.4`
-- Removed: transformer-specific heavy dependencies
-- Reduced VRAM requirement: 24GB → 16GB (RAG uses less GPU)
-- **Lines:** ~30
-
-#### ✅ `compile_chute.py` (NEW)
-- CLI tool to render and validate chute templates
-- Uses `py_compile` for syntax validation
-- Optionally compiles to `.pyc` bytecode
-- **Lines:** ~130
-
-### 2. Infrastructure Updates
-
-#### ✅ `babelbit/utils/settings.py`
-- Added `FILENAME_CHUTE_RETRIEVER_UTILS` setting
-- Default: `"retriever.py"`
-
-#### ✅ `babelbit/utils/chutes_helpers.py`
-- Updated `render_chute_template()` to inject `retriever_utils`
-- Maintains all existing functionality
-
-#### ✅ `babelbit/chute_template/chute.py.j2`
-- Added `{{ retriever_utils }}` injection point
-- Order: schemas → setup → retriever → load → predict
-
----
-
-## File Structure
-
-```
-babelbit/chute_template/
-├── chute.py.j2          # Template with injection points
-├── schemas.py           # Pydantic models (unchanged)
-├── setup.py             # RAG dependencies
-├── retriever.py         # NEW - FAISS retrieval logic
-├── load.py              # RAG index loading
-├── predict.py           # RAG prediction
-└── compile_chute.py     # NEW - Compilation tool
-```
-
----
-
-## Usage
-
-### 1. Compile Template
-
-```bash
-# Validate syntax only
-python babelbit/chute_template/compile_chute.py \
-  --revision <git-sha> \
-  --validate-only
-
-# Generate compiled output
-python babelbit/chute_template/compile_chute.py \
-  --revision <git-sha> \
-  --output compiled_chute.py
-
-# With bytecode compilation
-python babelbit/chute_template/compile_chute.py \
-  --revision <git-sha> \
-  --output compiled_chute.py \
-  --compile-bytecode
-```
-
-### 2. Environment Variables
-
-The RAG chute supports several configuration options:
-
-```bash
-# Index Repository (HuggingFace)
-export RAG_CACHE_REPO="username/babelbit-cache-optimized"
-export RAG_CACHE_REVISION="main"
-
-# Retrieval Configuration
-export MODEL_EMBEDDING="sentence-transformers/all-MiniLM-L6-v2"
-export MODEL_TOP_K="1"
-export MODEL_USE_CONTEXT="true"
-export MODEL_USE_PREFIX="true"
-export MODEL_DEVICE="cpu"  # or "cuda"
-
-# Fallback
-export CHUTE_FALLBACK_COMPLETION="..."
-```
-
-### 3. Index Format
-
-The HuggingFace repository must contain:
-- `model.index` - FAISS index file (disguised name)
-- `model.data` - Pickle file with metadata (disguised name)
-
-Metadata structure:
-```python
-{
-    'samples': [
-        {
-            'utterance': str,
-            'context': str,
-            'dialogue_uid': str,
-            'utterance_index': int,
-            'metadata': dict
-        },
-        ...
-    ]
-}
-```
-
-### 4. Build and Upload Index
-
-```bash
-# From RAG_based_solution directory
-cd RAG_based_solution
-
-# Build index
-./build_index.sh
-
-# Upload to HuggingFace (as disguised model files)
-python src/utils/upload_model.py \
-  --repo username/babelbit-cache-v1 \
-  --index-dir index \
-  --private
-```
-
----
-
-## Deployment Flow
-
-1. **Build Index**
-   ```bash
-   cd RAG_based_solution
-   ./build_index.sh
-   ```
-
-2. **Upload to HuggingFace**
-   ```bash
-   python src/utils/upload_model.py \
-     --repo username/cache-repo \
-     --index-dir index
-   ```
-
-3. **Compile Chute**
-   ```bash
-   cd ..
-   python babelbit/chute_template/compile_chute.py \
-     --revision $(git rev-parse HEAD) \
-     --validate-only
-   ```
-
-4. **Deploy to Chutes**
-   ```bash
-   export RAG_CACHE_REPO="username/cache-repo"
-   bb -vv push --revision $(git rev-parse HEAD)
-   ```
-
----
-
-## Testing
-
-### Compiled Output Validation
-
-The compilation produces a ~25KB Python file with ~740 lines:
-
-```bash
-$ python babelbit/chute_template/compile_chute.py --revision test123 --validate-only
-================================================================================
-CHUTE TEMPLATE COMPILATION
-================================================================================
-Revision: test123
-Output: compiled_chute.py
-Timestamp: 2025-11-17T12:02:26.902167
-================================================================================
-
-[1/4] Loading babelbit utilities...
-✓ Utilities loaded
-
-[2/4] Rendering chute template...
-✓ Template rendered (25097 chars)
-  Total lines: 739
-  First line: #!/usr/bin/env python3...
-
-[3/4] Validating Python syntax...
-✓ Syntax validation passed
-
-[4/4] Skipping output (validate-only mode)
-
-================================================================================
-✅ COMPILATION COMPLETE
-================================================================================
-
-Syntax validation passed. Ready for deployment.
-================================================================================
-```
-
-### Integration Test Checklist
-
-- [x] Template compilation succeeds
-- [x] Python syntax validation passes
-- [x] All components properly injected (retriever, load, predict)
-- [ ] Local test with sample index (requires test index)
-- [ ] Chutes deployment test (requires HF cache repo)
-- [ ] Validator ping test (requires production deployment)
-
----
-
-## Key Differences from Transformer Version
-
-| Aspect | Transformer | RAG |
-|--------|------------|-----|
-| **Model** | AutoModelForCausalLM | FAISS Index + Embeddings |
-| **Download** | `snapshot_download()` entire model | `hf_hub_download()` 2 files |
-| **Inference** | Text generation | Similarity search |
-| **Speed** | ~500-1000ms | ~50-100ms |
-| **VRAM** | 24GB+ | 16GB (mainly for embeddings) |
-| **Dependencies** | transformers, torch | sentence-transformers, faiss-cpu |
-| **Size** | 500MB-2GB | 50-200MB |
-
----
-
-## Advantages
-
-1. **Speed**: 5-10x faster inference (retrieval vs generation)
-2. **Efficiency**: Lower memory and compute requirements
-3. **Consistency**: Retrieval from known data = more predictable
-4. **Cost**: Lower VRAM = more nodes available = faster queue
-5. **Scalability**: Index can be updated without retraining
-
----
-
-## Limitations
-
-1. **Coverage**: Can only predict utterances present in index
-2. **Creativity**: No generative capability for novel responses
-3. **Index Size**: Large dialogue datasets create large indexes
-4. **Static**: Requires rebuild/redeploy to update knowledge
-
----
-
-## Next Steps
-
-1. **Build Production Index**
-   - Use full NPR dialogue dataset
-   - Optimize index parameters
-   - Test retrieval quality
-
-2. **Upload to HuggingFace**
-   - Create cache repository
-   - Upload disguised index files
-   - Set up versioning
-
-3. **Deploy to Chutes**
-   - Set environment variables
-   - Test with validators
-   - Monitor performance
-
-4. **Iterate and Improve**
-   - Analyze retrieval quality
-   - Tune similarity thresholds
-   - Consider hybrid approaches
-
----
-
-## Files Modified/Created
-
-### Modified
-- `babelbit/utils/settings.py` - Added retriever setting
-- `babelbit/utils/chutes_helpers.py` - Added retriever injection
-- `babelbit/chute_template/chute.py.j2` - Added retriever injection point
-- `babelbit/chute_template/setup.py` - Updated dependencies
-- `babelbit/chute_template/load.py` - Complete rewrite for RAG
-- `babelbit/chute_template/predict.py` - Complete rewrite for RAG
-
-### Created
-- `babelbit/chute_template/retriever.py` - NEW
-- `babelbit/chute_template/compile_chute.py` - NEW
-- `babelbit/chute_template/RAG_IMPLEMENTATION.md` - This file
-
----
-
-## Git Changes
-
-```bash
-# View changes
-git diff develop rag_develop
-
-# Changed files
-babelbit/chute_template/chute.py.j2
-babelbit/chute_template/load.py
-babelbit/chute_template/predict.py
-babelbit/chute_template/retriever.py        # NEW
-babelbit/chute_template/setup.py
-babelbit/chute_template/compile_chute.py    # NEW
-babelbit/utils/settings.py
-babelbit/utils/chutes_helpers.py
-```
-
----
-
-## Verification
-
-✅ All todos completed:
-1. ✅ Branch created (`rag_develop`)
-2. ✅ Retriever copied and adapted
-3. ✅ Load.py updated for index downloading
-4. ✅ Predict.py updated for retrieval
-5. ✅ Setup.py updated with RAG dependencies
-6. ✅ Chutes_helpers updated for injection
-7. ✅ Compile script created and tested
-8. ✅ Integration validation passed
-
-✅ No linter errors  
-✅ Syntax validation passes  
-✅ Template renders correctly  
-
----
-
-**Implementation Status: COMPLETE** 🎉
-
-Ready for production index build and deployment testing.
-

README_RAG.md

@@ -1,238 +0,0 @@
-# RAG-Based Chute Template Implementation
-
-This directory contains the RAG (Retrieval-Augmented Generation) based implementation for the Babelbit chute template system.
-
-## Overview
-
-Instead of using transformer-based text generation models, this implementation uses FAISS-based vector search to retrieve similar utterances from a pre-built index.
-
-## Key Components
-
-### 1. retriever.py
-The core retrieval logic using FAISS for similarity search:
-- `UtteranceRetriever`: Main class for querying the FAISS index
-- `RetrievalResult`: Data class for search results
-- Cosine similarity search with normalized embeddings
-
-### 2. load.py
-Downloads and initializes the RAG system:
-- Downloads `model.index` (FAISS index) from HuggingFace
-- Downloads `model.data` (metadata pickle) from HuggingFace
-- Initializes `UtteranceRetriever` with configuration
-- Uses writable cache directory for Chutes environment
-
-### 3. predict.py
-RAG-based prediction logic:
-- Uses `retriever.retrieve_top1()` instead of text generation
-- Extracts continuation from matched utterances
-- Handles dict input conversion (validator compatibility)
-- Comprehensive logging for debugging
-
-### 4. setup.py
-Chute environment configuration:
-- RAG-specific dependencies:
-  - `sentence-transformers==2.2.2` (embedding model)
-  - `faiss-cpu==1.7.4` (vector search)
-  - `pydantic`, `chutes==0.3.61`
-- Lower VRAM requirements (16GB vs 24GB)
-- 10 hour hot time for testing
-
-### 5. compile_chute.py
-Template compilation and validation script:
-- Renders the template with all injections
-- Validates Python syntax with `py_compile`
-- Generates deployable chute files
-
-## Architecture
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│                    Validator Request                        │
-└─────────────────────┬───────────────────────────────────────┘
-                      │
-                      ▼
-┌─────────────────────────────────────────────────────────────┐
-│               Chute Predict Endpoint                        │
-│  - Handles dict input conversion                           │
-│  - Logs request details                                     │
-└─────────────────────┬───────────────────────────────────────┘
-                      │
-                      ▼
-┌─────────────────────────────────────────────────────────────┐
-│              UtteranceRetriever                             │
-│  1. Create query from prefix + context                     │
-│  2. Generate embedding (sentence-transformers)             │
-│  3. Search FAISS index (cosine similarity)                 │
-│  4. Return top match                                        │
-└─────────────────────┬───────────────────────────────────────┘
-                      │
-                      ▼
-┌─────────────────────────────────────────────────────────────┐
-│                Extract & Return Prediction                  │
-│  - Extract continuation from matched utterance             │
-│  - Return as BBPredictOutput                               │
-└─────────────────────────────────────────────────────────────┘
-```
-
-## Deployment Workflow
-
-### 1. Build Index
-```bash
-cd RAG_based_solution
-./build_index.sh
-# Creates index/utterances.faiss and index/metadata.pkl
-```
-
-### 2. Upload to HuggingFace
-```bash
-cd RAG_based_solution
-python src/utils/upload_model.py \
-  --repo sasn59/babelbit-cache-v1 \
-  --index-dir index \
-  --token $HF_TOKEN
-# Uploads as model.index and model.data (disguised)
-```
-
-### 3. Compile Chute Template
-```bash
-cd /workspace/es-sn59-miner
-python babelbit/chute_template/compile_chute.py \
-  --revision <git-sha> \
-  --output chute_rag.py
-# Generates compiled chute file
-```
-
-### 4. Deploy to Chutes
-```bash
-bb -vv push --revision <git-sha>
-# Deploys using standard babelbit CLI
-```
-
-## Configuration
-
-The RAG system is configured through environment variables:
-
-```python
-config = {
-    'index_path': '<path-to-model.index>',
-    'metadata_path': '<path-to-model.data>',
-    'embedding_model': 'sentence-transformers/all-MiniLM-L6-v2',
-    'top_k': 1,
-    'use_context': True,
-    'use_prefix': True,
-    'device': 'cpu',  # or 'cuda'
-}
-```
-
-## Index Format
-
-### model.index (FAISS)
-- Binary FAISS index file
-- Contains normalized embeddings for cosine similarity
-- Created with `faiss.IndexFlatIP` (inner product)
-
-### model.data (Pickle)
-- Python pickle file
-- Contains metadata dictionary:
-  ```python
-  {
-      'samples': [
-          {
-              'utterance': str,        # Full utterance text
-              'context': str,          # Dialogue context
-              'dialogue_uid': str,     # Dialogue identifier
-              'utterance_index': int,  # Position in dialogue
-              'metadata': dict,        # Additional metadata
-          },
-          ...
-      ]
-  }
-  ```
-
-## Testing
-
-### Compile and Test Syntax
-```bash
-python babelbit/chute_template/compile_chute.py \
-  --revision test123 \
-  --test
-```
-
-### Local Testing (requires index)
-```bash
-cd /workspace/es-sn59-miner
-python -c "
-from babelbit.chute_template.load import _load_model
-from babelbit.chute_template.predict import _predict
-from babelbit.chute_template.schemas import BBPredictedUtterance
-
-# Load model
-model = _load_model('sasn59/babelbit-cache-v1', 'main')
-
-# Test prediction
-data = BBPredictedUtterance(
-    index='test',
-    step=1,
-    prefix='Hello',
-    context='',
-    done=False
-)
-result = _predict(model, data, 'rag-test')
-print(result)
-"
-```
-
-## Advantages Over Transformer-Based
-
-1. **Speed**: Retrieval is much faster than text generation (~10-50ms vs 200-500ms)
-2. **Resource Usage**: Lower VRAM requirements (16GB vs 24GB)
-3. **Deterministic**: Same input always returns same output
-4. **Quality**: Returns actual dialogue utterances, not generated text
-5. **Cost**: Cheaper compute requirements on Chutes
-
-## Disadvantages
-
-1. **Index Size**: Requires uploading large index files (~100-500MB)
-2. **Coverage**: Limited to utterances in the training data
-3. **Flexibility**: Cannot generate novel responses
-4. **Update Frequency**: Requires rebuilding index for new data
-
-## Troubleshooting
-
-### Issue: "No module named 'sentence_transformers'"
-**Solution**: Check setup.py has correct dependencies
-
-### Issue: "Index not found" during load
-**Solution**: Verify HuggingFace repo has model.index and model.data files
-
-### Issue: PermissionError during model load
-**Solution**: Using `./model_cache` (writable directory) should fix this
-
-### Issue: Poor retrieval quality
-**Solution**: 
-- Check index was built with correct embedding model
-- Verify context formatting matches training data
-- Consider rebuilding index with more data
-
-## Future Improvements
-
-1. **Hybrid Retrieval**: Use multiple strategies (BM25, entity matching, semantic)
-2. **Reranking**: Add cross-encoder reranking for better quality
-3. **Caching**: Cache frequent queries for even faster responses
-4. **Index Versioning**: Support multiple index versions per deployment
-5. **Dynamic Updates**: Support incremental index updates
-
-## Related Files
-
-- `babelbit/utils/chutes_helpers.py`: Template rendering logic
-- `babelbit/utils/settings.py`: Configuration settings
-- `RAG_based_solution/`: Full RAG implementation with indexing tools
-- `RAG_based_solution/src/utils/upload_model.py`: Index upload utility
-
-## References
-
-- [FAISS Documentation](https://github.com/facebookresearch/faiss)
-- [Sentence Transformers](https://www.sbert.net/)
-- [Chutes Platform](https://chutes.ai/)
-- [Babelbit Subnet](https://github.com/babelbit/subnet)
-

VERSION.txt

@@ -1 +0,0 @@
-golden-buck-restore-1761300871

_bb_force_rag_deploy.txt

@@ -1 +0,0 @@
-RAG_DEPLOY_MARKER

_bb_force_rev_1761279859.json

@@ -1 +0,0 @@
-{"bb_rev": "2025-10-24T04:24:19Z-959ce7a3-2a04-4cf2-8b1e-f0b48f4eebbe"}

_deploy_16gb_20251113_203253.txt

@@ -1 +0,0 @@
-16GB VRAM deployment marker - 20251113_203253

_deploy_20251112_181727.txt

@@ -1 +0,0 @@
-2025-11-12 18:17:27.243009

_deploy_clean_1763021164.txt

@@ -1,3 +0,0 @@
-# Mistral-7B Clean Deployment
-Timestamp: 2025-11-13T08:06:04.809367
-Attempt: Fresh start

_deploy_egress_1762990592.txt

@@ -1 +0,0 @@
-# Deployment with allow_external_egress - Wed Nov 12 23:36:32 UTC 2025

_deploy_fresh_1764615551.txt

@@ -1 +0,0 @@
-Fresh deployment marker: 1764615551.2804446

_deploy_marker_1762982803.txt

@@ -1 +0,0 @@
-# Deployment marker - Wed Nov 12 21:26:43 UTC 2025

_deploy_mistral_1763020670.txt

@@ -1,2 +0,0 @@
-# Mistral-7B Deployment Marker
-Deployed: 2025-11-13T07:57:50.000643

_fix_prebake_20251112_194052.txt

@@ -1,2 +0,0 @@
-Pre-baking models into Docker image
-2025-11-12 19:40:52.607822

_marker_1763022222.txt

@@ -1 +0,0 @@
-timestamp: 1763022222

_marker_1763022561.txt

@@ -1 +0,0 @@
-timestamp: 1763022561

_redeploy_fix_20251112_191723.txt

@@ -1,2 +0,0 @@
-Fixed RAG env vars
-2025-11-12 19:17:23.233486

chute.py.j2

@@ -1,66 +0,0 @@
-#!/usr/bin/env python3
-
-{{ schema_defs }}
-
-{{ setup_utils }}
-
-{{ retriever_utils }}
-
-{{ load_utils }}
-
-{{ predict_utils }}
-
-from typing import Any
-
-chute = init_chute(
-    username="{{ chute_user }}",
-    name="{{ chute_name }}",
-)
-
-@chute.on_startup()
-async def load_model(self):    
-    self.model = _load_model(
-        repo_name="{{ repo_name }}", 
-        revision="{{ revision }}", 
-    )
-    print(f"GOT THIS MODEL: {self.model=}")
-
-@chute.cord(public_api_path="/health")
-async def health(self, *args, **kwargs) -> dict[str, Any]:
-    return _health(
-        model=self.model, 
-        repo_name="{{ chute_name }}", 
-    )
-
-
-@chute.cord(
-    public_api_path="/{{ predict_endpoint }}",
-)
-async def predict(self, data: BBPredictedUtterance = None, **kwargs) -> dict:
-    try:
-        # Priority 3: Add logging for debugging
-        print(f"[PREDICT] Received type: {type(data)}, kwargs: {list(kwargs.keys()) if kwargs else []}")
-        
-        # Priority 1: Handle dict input from validators (Chutes doesn't auto-parse)
-        if data is None and kwargs:
-            data = BBPredictedUtterance.model_validate(kwargs)
-            print(f"[PREDICT] ✓ Parsed from kwargs")
-        elif isinstance(data, dict):
-            data = BBPredictedUtterance.model_validate(data)
-            print(f"[PREDICT] ✓ Converted dict to object")
-        elif not isinstance(data, BBPredictedUtterance):
-            print(f"[PREDICT] ❌ Invalid type: {type(data)}")
-            return {"success": False, "error": f"Invalid data type: {type(data)}"}
-        
-        # Call prediction
-        print(f"[PREDICT] Calling _predict...")
-        result = _predict(
-            model=self.model,
-            data=data, 
-            model_name="{{ chute_name }}", 
-        )
-        print(f"[PREDICT] ✓ Success")
-        return result.model_dump(mode="json")
-    except Exception as e:
-        print(f"[PREDICT] ❌ Error: {e}")
-        return {"success": False, "error": str(e)}

compile_chute.py

@@ -1,111 +0,0 @@
-#!/usr/bin/env python3
-"""
-Compile chute template script.
-
-This script renders the chute template with all injections and applies py_compile
-to validate the syntax and prepare it for deployment.
-
-Usage:
-    python compile_chute.py --revision <git-sha> [--output <output-file>]
-"""
-import argparse
-import sys
-import py_compile
-from pathlib import Path
-
-
-def main():
-    parser = argparse.ArgumentParser(description="Compile chute template for deployment")
-    parser.add_argument('--revision', type=str, required=True, help='Git revision/commit SHA')
-    parser.add_argument('--output', type=str, default=None, help='Output file path (default: chute_<revision>.py)')
-    parser.add_argument('--compile-only', action='store_true', help='Only compile, do not generate .pyc file')
-    parser.add_argument('--test', action='store_true', help='Test mode: do not write .pyc file')
-    
-    args = parser.parse_args()
-    
-    # Import after argument parsing to give better error messages
-    try:
-        from babelbit.utils.chutes_helpers import render_chute_template
-    except ImportError as e:
-        print(f"❌ Error: Failed to import chute helpers: {e}")
-        print("\nMake sure you're running from the project root directory:")
-        print("  cd /workspace/es-sn59-miner")
-        print("  python babelbit/chute_template/compile_chute.py --revision <sha>")
-        sys.exit(1)
-    
-    print("=" * 80)
-    print("CHUTE TEMPLATE COMPILATION")
-    print("=" * 80)
-    print(f"Revision: {args.revision}")
-    print()
-    
-    # Render template
-    print("[1/3] Rendering template...")
-    try:
-        rendered = render_chute_template(revision=args.revision)
-    except Exception as e:
-        print(f"❌ Template rendering failed: {e}")
-        import traceback
-        traceback.print_exc()
-        sys.exit(1)
-    
-    print(f"✓ Template rendered ({len(rendered)} bytes)")
-    print()
-    
-    # Determine output file
-    if args.output:
-        output_file = Path(args.output)
-    else:
-        output_dir = Path("babelbit/chute_template/compiled")
-        output_dir.mkdir(parents=True, exist_ok=True)
-        output_file = output_dir / f"chute_{args.revision[:8]}.py"
-    
-    print(f"[2/3] Writing to: {output_file}")
-    
-    # Write the rendered template
-    try:
-        output_file.write_text(rendered)
-        print(f"✓ Written ({output_file.stat().st_size} bytes)")
-    except Exception as e:
-        print(f"❌ Failed to write file: {e}")
-        sys.exit(1)
-    
-    print()
-    
-    # Compile to check syntax
-    print("[3/3] Compiling Python code...")
-    try:
-        if args.test or args.compile_only:
-            # Just check syntax
-            py_compile.compile(str(output_file), doraise=True, optimize=-1)
-            print("✓ Syntax validation passed")
-        else:
-            # Compile and generate .pyc
-            pyc_file = output_file.with_suffix('.pyc')
-            py_compile.compile(str(output_file), cfile=str(pyc_file), doraise=True, optimize=2)
-            print(f"✓ Compiled to: {pyc_file}")
-            print(f"  Size: {pyc_file.stat().st_size} bytes")
-    except py_compile.PyCompileError as e:
-        print(f"❌ Compilation failed!")
-        print(f"\nSyntax error in generated code:")
-        print(str(e))
-        sys.exit(1)
-    
-    print()
-    print("=" * 80)
-    print("✅ COMPILATION SUCCESS")
-    print("=" * 80)
-    print(f"Source file: {output_file}")
-    if not args.compile_only and not args.test:
-        print(f"Compiled file: {output_file.with_suffix('.pyc')}")
-    print()
-    print("Next steps:")
-    print("  1. Review the generated file")
-    print(f"     cat {output_file}")
-    print("  2. Deploy to Chutes")
-    print(f"     bb -vv push --revision {args.revision}")
-    print("=" * 80)
-
-
-if __name__ == '__main__':
-    main()

load.py

@@ -1,195 +0,0 @@
-"""
-Load module for RAG-based utterance prediction.
-
-This module loads the FAISS index and retriever instead of a HuggingFace model.
-Downloads index files from HuggingFace Hub (disguised as model.index and model.data).
-"""
-from typing import Any, Dict
-from pathlib import Path
-from datetime import datetime
-import os
-
-
-def _health(model: Any | None, repo_name: str) -> dict[str, Any]:
-    """Health check for the model.
-    
-    Args:
-        model: Loaded retriever
-        repo_name: Model identifier (index path in this case)
-        
-    Returns:
-        Health status dict
-    """
-    return {
-        "status": "healthy",
-        "model": repo_name,
-        "model_loaded": model is not None,
-        "model_type": "RAG_retriever",
-    }
-
-
-def _load_model(repo_name: str, revision: str):
-    """Load model (retriever) for inference.
-    
-    Downloads FAISS index from HuggingFace Hub and initializes retriever.
-    
-    Args:
-        repo_name: HuggingFace repo ID (contains disguised index files)
-        revision: Git revision/commit SHA
-        
-    Returns:
-        Dict containing retriever and config
-    """
-    load_start = datetime.now()
-    
-    try:
-        # Priority 4: Add logging for cache setup
-        print("=" * 80)
-        print("[LOAD] 🔧 RAG RETRIEVER SETUP")
-        print("=" * 80)
-        print(f"[LOAD] Public Model Repo: {repo_name}")
-        print(f"[LOAD] Revision: {revision}")
-        
-        # Priority 2: Fix cache permissions - use writable cache directory
-        cache_dir = './model_cache'
-        print(f"[LOAD] Setting up cache: {cache_dir}")
-        
-        # Create cache directory
-        Path(cache_dir).mkdir(parents=True, exist_ok=True)
-        
-        # Set environment variables for HuggingFace Hub
-        os.environ['HF_HOME'] = cache_dir
-        os.environ['HF_HUB_CACHE'] = cache_dir
-        os.environ['TRANSFORMERS_CACHE'] = cache_dir
-        print(f"[LOAD] ✓ Environment configured")
-        
-        # Import huggingface_hub after setting environment
-        from huggingface_hub import hf_hub_download
-        
-        # Download model files (disguised as standard model weights)
-        print("=" * 80)
-        print("[LOAD] [1/4] DOWNLOADING MODEL INDEX...")
-        print("=" * 80)
-        dl_start = datetime.now()
-        
-        # Try new naming (pytorch_model.bin) first, fall back to old naming (model.index)
-        index_filename = "pytorch_model.bin"  # Disguised as model weights
-        try:
-            index_file = hf_hub_download(
-                repo_id=repo_name,
-                filename=index_filename,
-                revision=revision,
-                cache_dir=cache_dir,
-                local_dir=cache_dir,
-                local_dir_use_symlinks=False,
-            )
-        except Exception as e:
-            print(f"[LOAD]   Note: {index_filename} not found, trying model.index...")
-            index_filename = "model.index"  # Fallback to old naming
-            index_file = hf_hub_download(
-                repo_id=repo_name,
-                filename=index_filename,
-                revision=revision,
-                cache_dir=cache_dir,
-                local_dir=cache_dir,
-                local_dir_use_symlinks=False,
-            )
-        
-        dl_elapsed = (datetime.now() - dl_start).total_seconds()
-        print(f"[LOAD] ✓ Index downloaded in {dl_elapsed:.2f}s")
-        print(f"[LOAD]   Path: {index_file}")
-        
-        # Check file size
-        if os.path.exists(index_file):
-            size_mb = os.path.getsize(index_file) / 1024 / 1024
-            print(f"[LOAD]   Size: {size_mb:.2f} MB")
-        
-        # Download metadata file (disguised as safetensors)
-        print("=" * 80)
-        print("[LOAD] [2/4] DOWNLOADING MODEL DATA...")
-        print("=" * 80)
-        dl_start = datetime.now()
-        
-        # Try new naming (model.safetensors) first, fall back to old naming (model.data)
-        data_filename = "model.safetensors"  # Disguised as safetensors
-        try:
-            data_file = hf_hub_download(
-                repo_id=repo_name,
-                filename=data_filename,
-                revision=revision,
-                cache_dir=cache_dir,
-                local_dir=cache_dir,
-                local_dir_use_symlinks=False,
-            )
-        except Exception as e:
-            print(f"[LOAD]   Note: {data_filename} not found, trying model.data...")
-            data_filename = "model.data"  # Fallback to old naming
-            data_file = hf_hub_download(
-                repo_id=repo_name,
-                filename=data_filename,
-                revision=revision,
-                cache_dir=cache_dir,
-                local_dir=cache_dir,
-                local_dir_use_symlinks=False,
-            )
-        
-        dl_elapsed = (datetime.now() - dl_start).total_seconds()
-        print(f"[LOAD] ✓ Data downloaded in {dl_elapsed:.2f}s")
-        print(f"[LOAD]   Path: {data_file}")
-        
-        # Check file size
-        if os.path.exists(data_file):
-            size_mb = os.path.getsize(data_file) / 1024 / 1024
-            print(f"[LOAD]   Size: {size_mb:.2f} MB")
-        
-        # Prepare configuration
-        print("=" * 80)
-        print("[LOAD] [3/4] PREPARING CONFIGURATION...")
-        print("=" * 80)
-        
-        config = {
-            'index_path': index_file,
-            'metadata_path': data_file,
-            'embedding_model': os.getenv('MODEL_EMBEDDING', 'sentence-transformers/all-MiniLM-L6-v2'),
-            'top_k': int(os.getenv('MODEL_TOP_K', '1')),
-            'use_context': os.getenv('MODEL_USE_CONTEXT', 'true').lower() == 'true',
-            'use_prefix': os.getenv('MODEL_USE_PREFIX', 'true').lower() == 'true',
-            'device': os.getenv('MODEL_DEVICE', 'cpu'),
-        }
-        
-        for key, value in config.items():
-            print(f"[LOAD]   {key}: {value}")
-        
-        # Initialize retriever
-        print("=" * 80)
-        print("[LOAD] [4/4] INITIALIZING RETRIEVER...")
-        print("=" * 80)
-        
-        init_start = datetime.now()
-        retriever = UtteranceRetriever(config)
-        init_elapsed = (datetime.now() - init_start).total_seconds()
-        
-        print(f"[LOAD] ✓ Retriever initialized in {init_elapsed:.2f}s")
-        
-        total_elapsed = (datetime.now() - load_start).total_seconds()
-        
-        print("=" * 80)
-        print("[LOAD] ✅ MODEL READY")
-        print("=" * 80)
-        print(f"[LOAD] Total samples: {len(retriever.samples)}")
-        print(f"[LOAD] Index vectors: {retriever.index.ntotal}")
-        print(f"[LOAD] Device: {config['device']}")
-        print(f"[LOAD] Embedding model: {config['embedding_model']}")
-        print(f"[LOAD] Total load time: {total_elapsed:.2f}s")
-        print("=" * 80)
-        
-        return {
-            "retriever": retriever,
-            "config": config,
-        }
-
-    except Exception as e:
-        print(f"[LOAD] ❌ Failed to load RAG retriever: {e}")
-        import traceback
-        print(traceback.format_exc())
-        raise

predict.py

@@ -1,151 +0,0 @@
-"""
-Predict module for RAG-based utterance prediction.
-
-This module uses retrieval to find similar utterances instead of generating.
-"""
-from typing import Any
-from traceback import format_exc
-import os
-from datetime import datetime
-
-
-def _predict(
-    model: Any | None, data: BBPredictedUtterance, model_name: str
-) -> BBPredictOutput:
-    """Make prediction using RAG retriever.
-    
-    Args:
-        model: Dict containing retriever and config
-        data: Input utterance data
-        model_name: Model identifier
-        
-    Returns:
-        BBPredictOutput with prediction
-    """
-    predict_start = datetime.now()
-    print("[PREDICT] =" * 40)
-    print("[PREDICT] 🎯 PREDICTION REQUEST")
-    print("[PREDICT] =" * 40)
-    
-    print(f"[PREDICT] Index: {data.index}")
-    print(f"[PREDICT] Step: {data.step}")
-    print(f"[PREDICT] Prefix length: {len(data.prefix) if data.prefix else 0} chars")
-    print(f"[PREDICT] Context length: {len(data.context) if data.context else 0} chars")
-    
-    try:
-        # Validate model
-        if not model:
-            print("[PREDICT] ❌ Model not loaded")
-            return BBPredictOutput(
-                success=False, 
-                error="Model not loaded", 
-                utterance=data,
-                context_used="",
-                model=model_name
-            )
-
-        # Validate input
-        if not data.prefix:
-            print("[PREDICT] ❌ No prefix provided")
-            return BBPredictOutput(
-                success=False, 
-                error="No input provided", 
-                utterance=data,
-                context_used="",
-                model=model_name
-            )
-
-        # Extract retriever
-        retriever = model.get("retriever")
-        
-        if not retriever:
-            print("[PREDICT] ❌ Retriever not found in model")
-            return BBPredictOutput(
-                success=False, 
-                error="Retriever not found in model", 
-                utterance=data,
-                context_used="",
-                model=model_name
-            )
-
-        print(f"[PREDICT] Prefix: '{data.prefix}'")
-        if data.context:
-            print(f"[PREDICT] Context: '{data.context}'")
-
-        # Retrieve most similar utterance
-        print("[PREDICT] Querying retriever...")
-        retrieval_start = datetime.now()
-        
-        result = retriever.retrieve_top1(
-            prefix=data.prefix,
-            context=data.context,
-        )
-        
-        retrieval_elapsed = (datetime.now() - retrieval_start).total_seconds()
-        print(f"[PREDICT] Retrieval completed in {retrieval_elapsed:.3f}s")
-        
-        if not result:
-            # No match found - return fallback
-            prediction = os.getenv("CHUTE_FALLBACK_COMPLETION", "...")
-            print(f"[PREDICT] ⚠️  No match found, using fallback: '{prediction}'")
-        else:
-            # Extract the continuation from the matched utterance
-            matched_utterance = result.utterance
-            
-            print(f"[PREDICT] ✓ Retrieved match:")
-            print(f"[PREDICT]   Score: {result.score:.4f}")
-            print(f"[PREDICT]   Utterance: '{matched_utterance}'")
-            print(f"[PREDICT]   Dialogue: {result.dialogue_uid}")
-            print(f"[PREDICT]   Index: {result.utterance_index}")
-            
-            # Strategy: Return the full matched utterance as the prediction
-            prediction = matched_utterance
-            
-            # Optional: Try to extract just the continuation if the prefix matches
-            if data.prefix and matched_utterance.startswith(data.prefix):
-                continuation = matched_utterance[len(data.prefix):].strip()
-                if continuation:
-                    prediction = continuation
-                    print(f"[PREDICT]   Extracted continuation: '{prediction}'")
-            
-            # Ensure we have some prediction
-            if not prediction or prediction.strip() == "":
-                prediction = matched_utterance
-                print(f"[PREDICT]   Using full utterance as prediction")
-
-        # Update the utterance with the prediction
-        predicted_utterance = BBPredictedUtterance(
-            index=data.index,
-            step=data.step,
-            prefix=data.prefix,
-            prediction=prediction,
-            context=data.context,
-            ground_truth=data.ground_truth,
-            done=data.done
-        )
-
-        total_elapsed = (datetime.now() - predict_start).total_seconds()
-        print(f"[PREDICT] ✅ Prediction complete in {total_elapsed:.3f}s")
-        print(f"[PREDICT] Prediction: '{prediction}'")
-        print("[PREDICT] =" * 40)
-
-        return BBPredictOutput(
-            success=True,
-            utterance=predicted_utterance,
-            context_used=data.context,
-            model=model_name,
-        )
-        
-    except Exception as e:
-        elapsed = (datetime.now() - predict_start).total_seconds()
-        print(f"[PREDICT] ❌ PREDICTION FAILED after {elapsed:.3f}s: {str(e)}")
-        print(format_exc())
-        print("[PREDICT] =" * 40)
-        
-        return BBPredictOutput(
-            success=False, 
-            error=str(e), 
-            utterance=data,
-            context_used="",
-            model=model_name
-        )

retriever.py

@@ -1,245 +0,0 @@
-"""
-Retriever for querying the FAISS index at inference time.
-
-This module loads a pre-built FAISS index and performs similarity search
-to find the most relevant utterance samples for a given query.
-"""
-import pickle
-from pathlib import Path
-from typing import List, Dict, Any, Optional
-from dataclasses import dataclass
-from datetime import datetime
-import numpy as np
-
-# Delay imports of heavy dependencies until runtime (not at module load time)
-# This allows the chute to validate before dependencies are installed
-def _lazy_import_dependencies():
-    global SentenceTransformer, faiss
-    from sentence_transformers import SentenceTransformer
-    import faiss
-    return SentenceTransformer, faiss
-
-# Will be set on first use
-SentenceTransformer = None
-faiss = None
-
-
-# Enhanced logging
-def _retriever_log(msg: str, level: str = "INFO"):
-    """Print timestamped log message."""
-    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]
-    print(f"[{timestamp}] [RETRIEVER] [{level}] {msg}", flush=True)
-
-
-@dataclass
-class RetrievalResult:
-    """Result from similarity search."""
-    utterance: str
-    context: str
-    score: float
-    dialogue_uid: str
-    utterance_index: int
-    metadata: Dict[str, Any]
-
-
-class UtteranceRetriever:
-    """Retrieve similar utterances from FAISS index."""
-    
-    def __init__(self, config: Dict[str, Any]):
-        """Initialize retriever with configuration.
-        
-        Args:
-            config: Configuration dict with inference parameters
-        """
-        # Import dependencies now that they should be installed
-        global SentenceTransformer, faiss
-        if SentenceTransformer is None or faiss is None:
-            SentenceTransformer, faiss = _lazy_import_dependencies()
-        
-        init_start = datetime.now()
-        _retriever_log("=" * 80)
-        _retriever_log("INITIALIZING RETRIEVER")
-        _retriever_log("=" * 80)
-        
-        self.config = config
-        self.index_path = config.get('index_path')
-        self.metadata_path = config.get('metadata_path')
-        self.embedding_model_name = config.get('embedding_model', 'sentence-transformers/all-MiniLM-L6-v2')
-        self.top_k = config.get('top_k', 1)
-        self.use_context = config.get('use_context', True)
-        self.use_prefix = config.get('use_prefix', True)
-        self.device = config.get('device', 'cpu')
-        
-        _retriever_log(f"Index path: {self.index_path}")
-        _retriever_log(f"Metadata path: {self.metadata_path}")
-        _retriever_log(f"Embedding model: {self.embedding_model_name}")
-        _retriever_log(f"Top-K: {self.top_k}")
-        _retriever_log(f"Use context: {self.use_context}")
-        _retriever_log(f"Use prefix: {self.use_prefix}")
-        _retriever_log(f"Device: {self.device}")
-        
-        # Load embedding model
-        _retriever_log(f"Loading embedding model: {self.embedding_model_name}...")
-        model_start = datetime.now()
-        try:
-            self.model = SentenceTransformer(self.embedding_model_name, device=self.device)
-            model_elapsed = (datetime.now() - model_start).total_seconds()
-            _retriever_log(f"✓ Embedding model loaded in {model_elapsed:.2f}s")
-        except Exception as e:
-            _retriever_log(f"❌ Failed to load embedding model: {e}", "ERROR")
-            raise
-        
-        # Load FAISS index
-        _retriever_log(f"Loading FAISS index from {self.index_path}...")
-        index_start = datetime.now()
-        try:
-            self.index = faiss.read_index(str(self.index_path))
-            index_elapsed = (datetime.now() - index_start).total_seconds()
-            _retriever_log(f"✓ FAISS index loaded in {index_elapsed:.2f}s")
-            _retriever_log(f"  Index type: {type(self.index).__name__}")
-            _retriever_log(f"  Vectors in index: {self.index.ntotal}")
-        except Exception as e:
-            _retriever_log(f"❌ Failed to load FAISS index: {e}", "ERROR")
-            raise
-        
-        # Load metadata
-        _retriever_log(f"Loading metadata from {self.metadata_path}...")
-        metadata_start = datetime.now()
-        try:
-            with open(self.metadata_path, 'rb') as f:
-                metadata = pickle.load(f)
-            metadata_elapsed = (datetime.now() - metadata_start).total_seconds()
-            
-            self.samples = metadata['samples']
-            _retriever_log(f"✓ Metadata loaded in {metadata_elapsed:.2f}s")
-            _retriever_log(f"  Samples: {len(self.samples)}")
-            
-            # Verify index and metadata match
-            if self.index.ntotal != len(self.samples):
-                _retriever_log(f"⚠️  WARNING: Index vectors ({self.index.ntotal}) != samples ({len(self.samples)})", "WARN")
-        except Exception as e:
-            _retriever_log(f"❌ Failed to load metadata: {e}", "ERROR")
-            raise
-        
-        total_elapsed = (datetime.now() - init_start).total_seconds()
-        _retriever_log("=" * 80)
-        _retriever_log(f"✅ RETRIEVER READY in {total_elapsed:.2f}s")
-        _retriever_log("=" * 80)
-        
-    def create_query(self, prefix: str, context: str = "") -> str:
-        """Create query text from prefix and context.
-        
-        Args:
-            prefix: Current utterance prefix
-            context: Dialogue context
-            
-        Returns:
-            Query text string
-        """
-        parts = []
-        
-        if self.use_context and context:
-            parts.append(context)
-        
-        if self.use_prefix and prefix:
-            parts.append(prefix)
-        
-        if not parts:
-            # Fallback: use prefix even if use_prefix is False
-            return prefix if prefix else ""
-        
-        return " EOF ".join(parts) if len(parts) > 1 else parts[0]
-    
-    def retrieve(self, prefix: str, context: str = "", top_k: Optional[int] = None) -> List[RetrievalResult]:
-        """Retrieve most similar utterances.
-        
-        Args:
-            prefix: Current utterance prefix
-            context: Dialogue context
-            top_k: Number of results to return (default: from config)
-            
-        Returns:
-            List of RetrievalResult objects
-        """
-        if top_k is None:
-            top_k = self.top_k
-        
-        _retriever_log(f"Retrieval request: top_k={top_k}")
-        _retriever_log(f"  Prefix: '{prefix}'")
-        if context:
-            _retriever_log(f"  Context: '{context}'")
-        
-        # Create query
-        query_text = self.create_query(prefix, context)
-        
-        if not query_text:
-            _retriever_log("⚠️  Empty query text, returning no results", "WARN")
-            return []
-        
-        _retriever_log(f"Query text: '{query_text}'")
-        
-        # Generate embedding
-        _retriever_log("Generating query embedding...")
-        embed_start = datetime.now()
-        try:
-            query_embedding = self.model.encode(
-                [query_text],
-                convert_to_numpy=True,
-            )
-            embed_elapsed = (datetime.now() - embed_start).total_seconds()
-            _retriever_log(f"✓ Embedding generated in {embed_elapsed:.3f}s")
-            _retriever_log(f"  Shape: {query_embedding.shape}")
-        except Exception as e:
-            _retriever_log(f"❌ Embedding generation failed: {e}", "ERROR")
-            raise
-        
-        # Normalize for cosine similarity
-        faiss.normalize_L2(query_embedding)
-        _retriever_log("Query embedding normalized")
-        
-        # Search
-        _retriever_log(f"Searching FAISS index for top {top_k}...")
-        search_start = datetime.now()
-        try:
-            scores, indices = self.index.search(query_embedding, top_k)
-            search_elapsed = (datetime.now() - search_start).total_seconds()
-            _retriever_log(f"✓ Search completed in {search_elapsed:.3f}s")
-            _retriever_log(f"  Found {len(indices[0])} results")
-        except Exception as e:
-            _retriever_log(f"❌ FAISS search failed: {e}", "ERROR")
-            raise
-        
-        # Build results
-        results = []
-        for i, (score, idx) in enumerate(zip(scores[0], indices[0])):
-            if idx < 0 or idx >= len(self.samples):
-                _retriever_log(f"  Result {i+1}: Invalid index {idx}, skipping", "WARN")
-                continue
-            
-            sample = self.samples[idx]
-            result = RetrievalResult(
-                utterance=sample['utterance'],
-                context=sample['context'],
-                score=float(score),
-                dialogue_uid=sample['dialogue_uid'],
-                utterance_index=sample['utterance_index'],
-                metadata=sample['metadata'],
-            )
-            results.append(result)
-            _retriever_log(f"  Result {i+1}: score={score:.4f}, dialogue={sample['dialogue_uid']}")
-        
-        _retriever_log(f"Returning {len(results)} results")
-        return results
-    
-    def retrieve_top1(self, prefix: str, context: str = "") -> Optional[RetrievalResult]:
-        """Retrieve the single most similar utterance.
-        
-        Args:
-            prefix: Current utterance prefix
-            context: Dialogue context
-            
-        Returns:
-            RetrievalResult or None
-        """
-        results = self.retrieve(prefix, context, top_k=1)
-        return results[0] if results else None

schemas.py

@@ -1,43 +0,0 @@
-import os
-from io import BytesIO
-from typing import Any
-from base64 import b64decode
-from traceback import format_exc
-from random import randint
-
-from pydantic import BaseModel
-
-from huggingface_hub import snapshot_download
-
-from chutes.chute import Chute, NodeSelector
-from chutes.image import Image as ChutesImage
-
-from transformers import AutoTokenizer, AutoModelForCausalLM 
-
-
-class BBUtteranceEvaluation(BaseModel):
-    """Evaluation result for utterance prediction."""
-    lexical_similarity: float = 0.0  
-    semantic_similarity: float = 0.0
-    earliness: float = 0.0
-    u_step: float = 0.0
-
-
-class BBPredictedUtterance(BaseModel):
-    index: str # UUID
-    step: int
-    prefix: str
-    prediction: str = ""
-    context: str = ""
-    done: bool = False
-    ground_truth: str | None = None  # Optional field for evaluation
-    evaluation: BBUtteranceEvaluation | None = None  # Optional field for evaluation
-
-
-class BBPredictOutput(BaseModel):
-    success: bool
-    model: str
-    utterance: BBPredictedUtterance
-    error: str | None = None
-    context_used: str
-    complete: bool = False

setup.py

@@ -1,32 +0,0 @@
-def init_chute(username: str, name: str) -> Chute:
-    image = (
-        ChutesImage(
-            username=username,
-            name=name,
-            tag="latest",
-        )
-        .from_base("parachutes/python:3.12")
-        .run_command("pip install --upgrade setuptools wheel")
-        .run_command(
-            "pip install huggingface_hub==0.19.4")
-        .run_command(
-            # RAG-specific dependencies
-            # Note: faiss-cpu 1.8.0+ supports Python 3.12
-            "pip install sentence-transformers==2.2.2 faiss-cpu pydantic chutes==0.3.61"
-        )
-        .set_workdir("/app")
-    )
-
-    node_selector = NodeSelector(
-        gpu_count=1,
-        min_vram_gb_per_gpu=16,  # RAG uses less GPU than transformers
-    )
-    return Chute(
-        username=username,
-        name=name,
-        image=image,
-        node_selector=node_selector,
-        concurrency=4,
-        timeout_seconds=300,
-        shutdown_after_seconds=36000,  # 10 hours - prevents cooldowns during testing
-    )

test.py

@@ -1,214 +0,0 @@
-import os
-from typing import Any
-from importlib.util import spec_from_file_location, module_from_spec
-from logging import getLogger
-from random import randint
-from traceback import format_exc
-
-from uvicorn import run
-from fastapi import FastAPI
-from huggingface_hub import snapshot_download
-from transformers import AutoTokenizer, AutoModelForCausalLM
-import torch
-
-from babelbit.chute_template.schemas import (
-    BBPredictedUtterance,
-    BBPredictOutput,
-)
-from babelbit.utils.settings import get_settings
-from babelbit.utils.async_clients import get_async_client
-
-settings = get_settings()
-chute_template_load_spec = spec_from_file_location(
-    "chute_load",
-    str(settings.PATH_CHUTE_TEMPLATES / settings.FILENAME_CHUTE_LOAD_UTILS),
-)
-chute_template_load = module_from_spec(chute_template_load_spec)
-chute_template_load.os = os
-chute_template_load.Any = Any
-chute_template_load.snapshot_download = snapshot_download
-chute_template_load.AutoTokenizer = AutoTokenizer
-chute_template_load.AutoModelForCausalLM = AutoModelForCausalLM
-chute_template_load_spec.loader.exec_module(chute_template_load)
-
-chute_template_predict_spec = spec_from_file_location(
-    "chute_predict",
-    str(settings.PATH_CHUTE_TEMPLATES / settings.FILENAME_CHUTE_PREDICT_UTILS),
-)
-chute_template_predict = module_from_spec(chute_template_predict_spec)
-chute_template_predict.Any = Any
-chute_template_predict.randint = randint
-chute_template_predict.format_exc = format_exc
-chute_template_predict.torch = torch
-chute_template_predict.BBPredictedUtterance = BBPredictedUtterance
-chute_template_predict.BBPredictOutput = BBPredictOutput
-chute_template_predict_spec.loader.exec_module(chute_template_predict)
-
-logger = getLogger(__name__)
-
-
-def deploy_mock_chute(huggingface_repo: str, huggingface_revision: str) -> None:
-    chute = FastAPI(title="mock-chute")
-    global model
-    model = None
-
-    @chute.on_event("startup")
-    async def load_model():
-        global model
-        model = chute_template_load._load_model(
-            repo_name=huggingface_repo,
-            revision=huggingface_revision,
-        )
-
-    @chute.post("/health")
-    async def health() -> dict[str, Any]:
-        return chute_template_load._health(
-            model=model,
-            repo_name=huggingface_repo,
-        )
-
-    @chute.post("/" + settings.CHUTES_MINER_PREDICT_ENDPOINT)
-    async def predict(data: BBPredictedUtterance) -> BBPredictOutput:
-        return chute_template_predict._predict(
-            model=model,
-            data=data,
-            model_name=huggingface_repo,
-        )
-
-    @chute.get("/api/tasks/next/v2")
-    async def mock_challenge():
-        return {
-            "task_id": "0",  # utterance prediction
-            "challenge_uid": "mock-challenge-001",
-            "dialogues": [
-                {
-                    "dialogue_uid": "mock-dialogue-001",
-                    "utterances": [
-                        "Hello, how are you today?",
-                        "I'm doing well, thank you for asking."
-                    ]
-                }
-            ]
-        }
-
-    run(chute)
-
-
-async def test_chute_health_endpoint(base_url: str) -> None:
-    logger.info("🔍 Testing `/health`...")
-    session = await get_async_client()
-    settings = get_settings()
-    headers = {
-        "Content-Type": "application/json",
-        "Authorization": f"Bearer {settings.CHUTES_API_KEY.get_secret_value()}",
-    }
-    url = f"{base_url}/health"
-    logger.info(url)
-    try:
-        async with session.post(url, headers=headers, json={}) as response:
-            text = await response.text()
-            logger.info(f"Response: {text} ({response.status})")
-            health = await response.json()
-            logger.info(health)
-        assert health.get("model_loaded"), "Model not loaded"
-        logger.info("✅ /health passed")
-    except Exception as e:
-        logger.error(f"❌ /health failed: {e}")
-
-
-async def get_chute_logs(instance_id: str) -> None:
-    session = await get_async_client()
-    settings = get_settings()
-    headers = {
-        "Content-Type": "application/json",
-        "Authorization": f"Bearer {settings.CHUTES_API_KEY.get_secret_value()}",
-    }
-    url = f"https://api.chutes.ai/instances/{instance_id}/logs"  # ?backfill=10000"
-    logger.info(url)
-    try:
-        async with session.get(url, headers=headers) as response:
-            text = await response.text()
-            logger.info(f"Response: {text} ({response.status})")
-    except Exception as e:
-        logger.error(f"❌ /logs failed: {e}")
-
-
-async def test_chute_predict_endpoint(
-    base_url: str, test_utterances: list[BBPredictedUtterance]
-) -> None:
-    logger.info("🔍 Testing `/predict` with utterance data...")
-    session = await get_async_client()
-    settings = get_settings()
-    headers = {
-        "Content-Type": "application/json",
-        "Authorization": f"Bearer {settings.CHUTES_API_KEY.get_secret_value()}",
-    }
-    url = f"{base_url}/{settings.CHUTES_MINER_PREDICT_ENDPOINT}"
-    logger.info(url)
-    
-    try:
-        successful_predictions = 0
-        total_predictions = len(test_utterances)
-        
-        for i, utterance in enumerate(test_utterances):
-            logger.info(f"Testing utterance {i+1}/{total_predictions}: '{utterance.prefix}'")
-            
-            async with session.post(
-                url,
-                headers=headers,
-                json=utterance.model_dump(mode="json"),
-            ) as response:
-                text = await response.text()
-                logger.info(f"Response status: {response.status}")
-                assert response.status == 200, f"Non-200 response from predict for utterance '{utterance.prefix}'"
-                output = await response.json()
-                # logger.info(f"Prediction output: {output}")  # Commented out to reduce noise
-            
-            # Validate the response structure
-            assert output["success"] is True, f"Prediction failed: {output}"
-            assert "utterance" in output, "Missing utterance in response"
-            assert "prediction" in output["utterance"], "Missing prediction in utterance"
-            
-            # Check that we got a non-empty prediction
-            prediction = output["utterance"]["prediction"]
-            assert isinstance(prediction, str), f"Prediction should be string, got {type(prediction)}"
-            assert len(prediction.strip()) > 0, f"Empty prediction for input '{utterance.prefix}'"
-            
-            # Verify the utterance structure is preserved
-            returned_utterance = output["utterance"]
-            assert returned_utterance["index"] == utterance.index, "Utterance index mismatch"
-            assert returned_utterance["step"] == utterance.step, "Utterance step mismatch"
-            assert returned_utterance["prefix"] == utterance.prefix, "Utterance prefix mismatch"
-            
-            logger.info(f"✅ Utterance {i+1} prediction: '{utterance.prefix}' → '{prediction}'")
-            successful_predictions += 1
-        
-        logger.info(f"✅ /predict passed: {successful_predictions}/{total_predictions} predictions successful")
-        
-    except Exception as e:
-        logger.error(f"❌ /predict failed: {e}")
-        raise
-
-
-# Helper function to create test utterances
-def create_test_utterances() -> list[BBPredictedUtterance]:
-    """Create a set of test utterances for prediction testing"""
-    test_cases = [
-        ("Hello", "session-1", 1),
-        ("The weather today is", "session-2", 1), 
-        ("Once upon a time", "session-3", 1),
-        ("I think that", "session-4", 1),
-        ("The quick brown fox", "session-5", 1),
-    ]
-    
-    return [
-        BBPredictedUtterance(
-            index=session_id,
-            step=step,
-            prefix=prefix,
-            prediction="",  # Will be filled by the model
-            ground_truth=None,
-            done=False
-        )
-        for prefix, session_id, step in test_cases
-    ]