Clean rebuild: all features fixed

CHANGES.md

@@ -0,0 +1,51 @@
+# Perbaikan dari v2_0_5 → v2_0_5-fixed
+
+Ringkasan: 7 file diubah, 1 file diperbaiki tambahan. Semua perbaikan **defensive** (membuat
+kode lebih tahan banting), bukan refactor besar — supaya behaviour lama tidak berubah selama
+kondisi normal, tapi ada fallback yang jelas saat kondisi tidak normal.
+
+## Daftar perubahan
+
+| File | Jenis | Tujuan |
+|------|-------|--------|
+| `frontend/index.html` | Theme bulletproof | `try/catch` di sekitar `localStorage` & `matchMedia`; fallback `:root` CSS variables |
+| `rag_pipeline/src/config.py` | Config | `groq_api_key` jadi optional (default `""`) — service ngga crash kalau secret belum di-set |
+| `rag_pipeline/src/llm/groq_client.py` | LLM client | ChatGroq di-init **lazy** — pas pertama dipakai, bukan saat constructor jalan. Validasi API key di sini dengan pesan jelas. |
+| `rag_pipeline/src/api/routes.py` | Error handling | `/query` dan `/summarize` return **503** (bukan 500) saat API key missing, dengan pesan spesifik. `/ready` sekarang lapor status `groq_api_key`. `/collection` DELETE error reporting lebih baik. |
+| `rag_pipeline/src/retrieval/vector_store.py` | Robustness | `reset_collection()` punya **3-tier fallback**: `db.reset_collection()` → delete-by-ids → nuke + re-init |
+| `cv_module/src/cv_pipeline.py` | Thread safety | Per-model `Lock` di lazy property (mencegah double-init kalau 2+ request concurrent). `ThreadPoolExecutor(max_workers=2)` mencegah OOM di HF free tier. |
+| `cv_module/src/api/routes.py` | Thread safety | `_trigger_lock` mencegah TOCTOU race di `_trigger_and_wait`. Error handler tiap endpoint kasih pesan yg lebih informatif. |
+| `start.sh` | Diagnostics | Print sanity check (file paths exist, GROQ_API_KEY status) saat boot — gampang debug dari log. |
+
+## Yang TIDAK gw ubah
+
+- Dockerfile: aman, struktur `cv_module/` dan `rag_pipeline/` di repo lo udah match dengan `COPY` di Dockerfile.
+- `nginx.conf`, `supervisord.conf`: aman.
+- `requirements.txt`: aman, version pin udah konsisten.
+- Loaders (PDF/DOCX/TXT/JSON/Web): aman, ngga ada perubahan logic.
+- Theme system di v2_0_5 *secara fungsional* udah bener — gw cuma tambah defensive guards. Kalau di lo "ilang", penyebabnya bukan code-level missing tapi runtime issue (browser, deploy, cache).
+
+## Penjelasan kenapa fix ini bisa nyelesaiin "semua endpoint 500"
+
+Hipotesis paling mungkin penyebabnya:
+
+1. **GROQ_API_KEY belum di-set di HF Space.** Sebelum fix:
+   - `Settings()` raise `ValidationError` karena `Field(...)` mandatory.
+   - `RAGRetriever.__init__()` crash di `get_settings()`.
+   - Setiap endpoint yg manggil `get_retriever()` (≈ semua) → 500.
+   - Setelah fix: service start clean, `/stats` `/sources` `/ingest` tetep jalan, cuma `/query` `/summarize` yg 503 dengan pesan "GROQ_API_KEY belum di-set".
+
+2. **CV model concurrent load OOM.** Sebelum fix:
+   - 2+ request paralel ke endpoint yang butuh model yg sama → race di `if self._captioner is None` → 2x init paralel → RAM spike → OS kill process → 500/connection error.
+   - Setelah fix: per-model lock, cuma 1 thread yg load.
+
+3. **`db.reset_collection()` AttributeError di langchain_chroma versi tertentu.** Sebelum fix: fallback path bisa gagal di chromadb 0.5.3 + langchain_chroma 0.1.4 combo karena `_client.delete_collection` lalu re-init bisa race. Setelah fix: delete-by-ids jadi default fallback (lebih atomic).
+
+## Yang harus lo cek sebelum deploy
+
+1. **GROQ_API_KEY**: Pastiin udah di-set di HF Spaces → Settings → Variables and secrets.
+2. **Struktur repo HF**: Dockerfile `COPY rag_pipeline/src/...`. Pastiin folder ini ada di repo HF (bukan cuma di local zip). Kalau di HF strukturnya `rag/` (bukan `rag_pipeline/`), build bakal gagal — dan **semua endpoint return error karena container ngga jalan**, bukan karena bug kode.
+3. **HF Space rebuild**: Kadang HF nge-cache build layer. Setelah deploy versi baru, force rebuild via dashboard ("Restart this Space" → "Factory rebuild").
+
+Kalau semua 3 hal di atas udah OK dan endpoint masih error, share log dari HF Spaces dashboard
+(tab "Logs"), gw bisa pinpoint penyebab spesifik.

cv-requirements.txt

@@ -1,35 +0,0 @@
-# ── CV Core ───────────────────────────────────────────────
-transformers==4.35.2
-numpy==1.26.4
-Pillow>=10.4.0
-opencv-python-headless>=4.10.0
-
-# ── CLIP ──────────────────────────────────────────────────
-open-clip-torch>=2.26.1
-timm==0.9.16
-
-# ── Object Detection ──────────────────────────────────────
-ultralytics>=8.2.0
-
-# ── OCR ───────────────────────────────────────────────────
-pytesseract>=0.3.13
-easyocr>=1.7.1
-
-# ── Image utils ───────────────────────────────────────────
-imageio>=2.34.0
-scikit-image>=0.24.0
-
-# ── API ───────────────────────────────────────────────────
-fastapi==0.112.0
-uvicorn[standard]==0.30.6
-python-multipart==0.0.9
-pydantic==2.8.2
-pydantic-settings==2.4.0
-
-# ── MLOps ─────────────────────────────────────────────────
-mlflow==2.15.1
-
-# ── Utils ─────────────────────────────────────────────────
-loguru==0.7.2
-python-dotenv==1.0.1
-httpx==0.27.0

cv/src/api/main.py

@@ -1,48 +0,0 @@
-from fastapi import FastAPI
-from fastapi.middleware.cors import CORSMiddleware
-from loguru import logger
-import sys
-
-from .routes import router
-from ..config import get_cv_settings
-
-settings = get_cv_settings()
-
-logger.remove()
-logger.add(sys.stderr, level="INFO", colorize=True,
-           format="<green>{time:HH:mm:ss}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan> - {message}")
-logger.add("./logs/cv_api.log", rotation="10 MB", retention="7 days")
-
-app = FastAPI(
-    title="CV Pipeline API",
-    description="""
-## Multimodal AI Assistant — Computer Vision Module
-
-Endpoint untuk analisis gambar menggunakan:
-- **BLIP** — image captioning & visual QA
-- **YOLOv8** — real-time object detection (80 kelas COCO)
-- **CLIP** — zero-shot classification & image-text similarity
-- **EasyOCR** — text extraction dari gambar (80+ bahasa)
-- **MLflow** — latency & performance tracking
-
-### Integrasi dengan RAG Module
-Output `summary_text` dari `/analyze` bisa langsung dipakai sebagai
-konteks untuk RAG pipeline — gambar bisa menjadi bagian dari knowledge base.
-    """,
-    version="1.0.0",
-)
-
-app.add_middleware(
-    CORSMiddleware,
-    allow_origins=["*"],
-    allow_methods=["*"],
-    allow_headers=["*"],
-)
-
-app.include_router(router, prefix="/api/v1")
-
-
-@app.on_event("startup")
-async def startup():
-    logger.info("CV Pipeline API starting up...")
-    logger.info(f"Docs: http://{settings.api_host}:{settings.api_port}/docs")

cv/src/api/routes.py

@@ -1,246 +0,0 @@
-from fastapi import APIRouter, HTTPException, UploadFile, File
-from fastapi.responses import Response
-from pydantic import BaseModel
-from loguru import logger
-
-from .schemas import (
-    AnalyzeURLRequest, FullAnalysisResponse,
-    ClassifyRequest, ClassificationResponse,
-    SimilarityRequest, SimilarityResponse,
-    VisualQARequest, VisualQAResponse,
-    CaptionResponse, DetectionResponse, OCRResponse,
-)
-from ..cv_pipeline import CVPipeline
-
-router = APIRouter()
-
-_pipeline: CVPipeline = None
-
-
-def get_pipeline() -> CVPipeline:
-    global _pipeline
-    if _pipeline is None:
-        _pipeline = CVPipeline()
-    return _pipeline
-
-
-# === HEALTH ===
-
-@router.get("/health", tags=["system"])
-async def health():
-    return {"status": "ok", "service": "CV Pipeline API"}
-
-
-# === FULL ANALYSIS ===
-
-@router.post("/analyze/url", response_model=FullAnalysisResponse, tags=["analysis"])
-async def analyze_from_url(req: AnalyzeURLRequest):
-    """
-    Analisis gambar dari URL.
-    Jalankan caption, object detection, optional OCR + CLIP classification sekaligus.
-    """
-    try:
-        result = get_pipeline().analyze(
-            source=req.url,
-            run_caption=req.run_caption,
-            run_detection=req.run_detection,
-            run_ocr=req.run_ocr,
-            classification_labels=req.classification_labels,
-        )
-        return _to_response(result)
-    except Exception as e:
-        logger.error(f"Analyze error: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@router.post("/analyze/upload", response_model=FullAnalysisResponse, tags=["analysis"])
-async def analyze_upload(
-    file: UploadFile = File(...),
-    run_caption: bool = True,
-    run_detection: bool = True,
-    run_ocr: bool = False,
-):
-    """Upload dan analisis gambar langsung (multipart)."""
-    allowed = {"image/jpeg", "image/png", "image/webp", "image/gif"}
-    if file.content_type not in allowed:
-        raise HTTPException(400, detail=f"Tipe file tidak didukung: {file.content_type}")
-
-    data = await file.read()
-    if len(data) > 10 * 1024 * 1024:
-        raise HTTPException(400, detail="Ukuran file maksimum 10MB")
-
-    try:
-        result = get_pipeline().analyze(
-            source=data,
-            run_caption=run_caption,
-            run_detection=run_detection,
-            run_ocr=run_ocr,
-        )
-        return _to_response(result)
-    except Exception as e:
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-# === INDIVIDUAL TASKS ===
-
-@router.post("/caption", response_model=CaptionResponse, tags=["tasks"])
-async def caption(url: str, prompt: str = None):
-    """Generate deskripsi teks dari gambar."""
-    try:
-        from ..processors.image_preprocessor import ImagePreprocessor
-        image = ImagePreprocessor.load(url)
-        result = get_pipeline().captioner.caption(image, prompt=prompt)
-        return CaptionResponse(caption=result.caption, model=result.model)
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-@router.post("/detect", response_model=DetectionResponse, tags=["tasks"])
-async def detect(url: str, conf: float = None):
-    """Deteksi objek dalam gambar dengan YOLOv8."""
-    try:
-        result = get_pipeline().detect_objects(url, conf=conf)
-        return DetectionResponse(
-            detections=[d.to_dict() for d in result.detections],
-            count=result.count,
-            labels_summary=result.labels_summary,
-            image_width=result.image_width,
-            image_height=result.image_height,
-            inference_time_ms=result.inference_time_ms,
-        )
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-@router.post("/classify", response_model=ClassificationResponse, tags=["tasks"])
-async def classify(req: ClassifyRequest):
-    """
-    Zero-shot image classification dengan CLIP.
-    Tidak perlu training — cukup berikan daftar label kandidat.
-    """
-    try:
-        result = get_pipeline().classify_image(req.url, req.labels)
-        return ClassificationResponse(
-            top_label=result.top_label,
-            top_score=result.top_score,
-            labels=result.labels,
-            probabilities=result.probabilities,
-        )
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-class OCRRequest(BaseModel):
-    url: str
-
-
-@router.post("/ocr", response_model=OCRResponse, tags=["tasks"])
-async def ocr(req: OCRRequest):
-    """Ekstrak teks dari gambar menggunakan EasyOCR."""
-    try:
-        from ..processors.image_preprocessor import ImagePreprocessor
-        image = ImagePreprocessor.load(req.url)
-        result = get_pipeline().ocr.extract_text(image)
-        return OCRResponse(
-            full_text=result.full_text,
-            boxes=[b.to_dict() for b in result.boxes],
-            word_count=result.word_count,
-            language=result.language,
-            engine=result.engine,
-        )
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-@router.post("/similarity", response_model=SimilarityResponse, tags=["tasks"])
-async def image_text_similarity(req: SimilarityRequest):
-    """Hitung relevansi antara gambar dan teks (0.0 - 1.0)."""
-    try:
-        score = get_pipeline().image_text_similarity(req.url, req.text)
-        interpretation = (
-            "Sangat relevan" if score > 0.7
-            else "Cukup relevan" if score > 0.5
-            else "Kurang relevan"
-        )
-        return SimilarityResponse(
-            similarity_score=score,
-            text=req.text,
-            interpretation=interpretation,
-        )
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-@router.post("/visual-qa", response_model=VisualQAResponse, tags=["tasks"])
-async def visual_qa(req: VisualQARequest):
-    """Visual Question Answering — tanya tentang isi gambar."""
-    try:
-        answer = get_pipeline().visual_qa(req.url, req.question)
-        return VisualQAResponse(question=req.question, answer=answer)
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-@router.get("/annotate", tags=["tasks"])
-async def annotate(url: str):
-    """Return gambar dengan bounding box YOLO yang sudah digambar (JPEG)."""
-    try:
-        jpeg_bytes = get_pipeline().annotate_image(url)
-        return Response(content=jpeg_bytes, media_type="image/jpeg")
-    except Exception as e:
-        raise HTTPException(500, detail=str(e))
-
-
-# === HELPER ===
-
-def _to_response(result) -> FullAnalysisResponse:
-    """Convert CVAnalysisResult ke FullAnalysisResponse."""
-    caption_r = None
-    if result.caption:
-        caption_r = CaptionResponse(
-            caption=result.caption.caption,
-            model=result.caption.model,
-        )
-
-    det_r = None
-    if result.detections:
-        det_r = DetectionResponse(
-            detections=[d.to_dict() for d in result.detections.detections],
-            count=result.detections.count,
-            labels_summary=result.detections.labels_summary,
-            image_width=result.detections.image_width,
-            image_height=result.detections.image_height,
-            inference_time_ms=result.detections.inference_time_ms,
-        )
-
-    cls_r = None
-    if result.classification:
-        cls_r = ClassificationResponse(
-            top_label=result.classification.top_label,
-            top_score=result.classification.top_score,
-            labels=result.classification.labels,
-            probabilities=result.classification.probabilities,
-        )
-
-    ocr_r = None
-    if result.ocr:
-        ocr_r = OCRResponse(
-            full_text=result.ocr.full_text,
-            boxes=[b.to_dict() for b in result.ocr.boxes],
-            word_count=result.ocr.word_count,
-            language=result.ocr.language,
-            engine=result.ocr.engine,
-        )
-
-    return FullAnalysisResponse(
-        image_width=result.image_width,
-        image_height=result.image_height,
-        source=result.source,
-        caption=caption_r,
-        detections=det_r,
-        classification=cls_r,
-        ocr=ocr_r,
-        summary_text=result.to_summary(),
-        models_used=result.models_used,
-        total_latency_ms=result.total_latency_ms,
-    )

cv/src/api/schemas.py

@@ -1,110 +0,0 @@
-from pydantic import BaseModel, Field, HttpUrl
-from typing import List, Optional
-
-
-# === Shared ===
-
-class BBoxSchema(BaseModel):
-    x1: float
-    y1: float
-    x2: float
-    y2: float
-    width: float
-    height: float
-
-
-class DetectionSchema(BaseModel):
-    label: str
-    confidence: float
-    bbox: BBoxSchema
-    class_id: int
-
-
-class OCRBoxSchema(BaseModel):
-    text: str
-    confidence: float
-    bbox: list
-
-
-# === Requests ===
-
-class AnalyzeURLRequest(BaseModel):
-    url: str = Field(..., description="URL gambar yang akan dianalisis")
-    run_caption: bool = Field(True, description="Generate image caption")
-    run_detection: bool = Field(True, description="Deteksi objek dengan YOLO")
-    run_ocr: bool = Field(False, description="Ekstrak teks dari gambar")
-    classification_labels: Optional[List[str]] = Field(
-        None,
-        description="Label untuk zero-shot CLIP classification, e.g. ['kucing','anjing']",
-        example=["indoor", "outdoor", "nature", "city"],
-    )
-
-
-class ClassifyRequest(BaseModel):
-    url: str
-    labels: List[str] = Field(..., min_length=2, description="Minimal 2 label kandidat")
-
-
-class SimilarityRequest(BaseModel):
-    url: str
-    text: str = Field(..., min_length=1)
-
-
-class VisualQARequest(BaseModel):
-    url: str
-    question: str = Field(..., description="Pertanyaan tentang isi gambar")
-
-
-# === Responses ===
-
-class CaptionResponse(BaseModel):
-    caption: str
-    model: str
-
-
-class DetectionResponse(BaseModel):
-    detections: List[DetectionSchema]
-    count: int
-    labels_summary: dict
-    image_width: int
-    image_height: int
-    inference_time_ms: float
-
-
-class ClassificationResponse(BaseModel):
-    top_label: str
-    top_score: float
-    labels: List[str]
-    probabilities: List[float]
-
-
-class OCRResponse(BaseModel):
-    full_text: str
-    boxes: List[OCRBoxSchema]
-    word_count: int
-    language: str
-    engine: str
-
-
-class FullAnalysisResponse(BaseModel):
-    image_width: int
-    image_height: int
-    source: str
-    caption: Optional[CaptionResponse] = None
-    detections: Optional[DetectionResponse] = None
-    classification: Optional[ClassificationResponse] = None
-    ocr: Optional[OCRResponse] = None
-    summary_text: str = Field(..., description="Ringkasan teks dari semua model — siap dipakai sebagai konteks LLM")
-    models_used: List[str]
-    total_latency_ms: float
-
-
-class SimilarityResponse(BaseModel):
-    similarity_score: float
-    text: str
-    interpretation: str
-
-
-class VisualQAResponse(BaseModel):
-    question: str
-    answer: str

cv/src/config.py

@@ -1,55 +0,0 @@
-from pydantic_settings import BaseSettings
-from pydantic import Field
-from functools import lru_cache
-from pathlib import Path
-
-
-class CVSettings(BaseSettings):
-    # Device
-    device: str = Field("cpu", env="CV_DEVICE")           # "cpu" atau "cuda"
-
-    # CLIP
-    clip_model: str = Field("ViT-B-32", env="CLIP_MODEL")
-    clip_pretrained: str = Field("openai", env="CLIP_PRETRAINED")
-
-    # YOLO
-    yolo_model: str = Field("yolov8n.pt", env="YOLO_MODEL")  # n=nano, s=small, m=medium
-    yolo_conf_threshold: float = Field(0.25, env="YOLO_CONF")
-    yolo_iou_threshold: float = Field(0.45, env="YOLO_IOU")
-
-    # Image Captioning
-    caption_model: str = Field(
-        "Salesforce/blip-image-captioning-base", env="CAPTION_MODEL"
-    )
-
-    # OCR
-    ocr_engine: str = Field("easyocr", env="OCR_ENGINE")    # "easyocr" atau "tesseract"
-    ocr_languages: str = Field("en,id", env="OCR_LANGUAGES") # comma-separated
-
-    # API
-    api_host: str = Field("0.0.0.0", env="CV_API_HOST")
-    api_port: int = Field(8001, env="CV_API_PORT")
-    max_image_size_mb: float = Field(10.0, env="MAX_IMAGE_SIZE_MB")
-
-    # Storage
-    upload_dir: str = Field("./uploads", env="CV_UPLOAD_DIR")
-    models_cache_dir: str = Field("./model_cache", env="CV_MODELS_CACHE")
-
-    # MLflow
-    mlflow_tracking_uri: str = Field("./mlruns", env="MLFLOW_TRACKING_URI")
-    mlflow_experiment_name: str = Field("cv_pipeline", env="MLFLOW_CV_EXPERIMENT")
-
-    class Config:
-        env_file = ".env"
-        env_file_encoding = "utf-8"
-
-    def ensure_dirs(self):
-        for d in [self.upload_dir, self.models_cache_dir, "./logs", "./mlruns"]:
-            Path(d).mkdir(parents=True, exist_ok=True)
-
-
-@lru_cache()
-def get_cv_settings() -> CVSettings:
-    s = CVSettings()
-    s.ensure_dirs()
-    return s

cv/src/cv_pipeline.py

@@ -1,246 +0,0 @@
-from __future__ import annotations
-
-import time
-from typing import List, Optional, Union
-from dataclasses import dataclass, field
-from pathlib import Path
-
-import mlflow
-from loguru import logger
-
-from .config import get_cv_settings
-from .processors.image_preprocessor import ImagePreprocessor, ImageInput
-from .models.clip_model import CLIPModel, CLIPResult
-from .models.yolo_detector import YOLODetector, DetectionResult
-from .models.captioner import ImageCaptioner, CaptionResult
-from .processors.ocr_processor import OCRProcessor, OCRResult
-
-
-@dataclass
-class CVAnalysisResult:
-    """Hasil lengkap analisis gambar dari semua model."""
-
-    # Info gambar
-    image_width: int = 0
-    image_height: int = 0
-    source: str = ""
-
-    # Per-model results (None jika tidak dijalankan)
-    caption: Optional[CaptionResult] = None
-    detections: Optional[DetectionResult] = None
-    classification: Optional[CLIPResult] = None
-    ocr: Optional[OCRResult] = None
-
-    # Metadata
-    models_used: List[str] = field(default_factory=list)
-    total_latency_ms: float = 0.0
-
-    def to_summary(self) -> str:
-        """
-        Buat ringkasan teks dari hasil analisis.
-        Berguna sebagai input ke LLM (integrasi dengan RAG module).
-        """
-        parts = []
-
-        if self.caption:
-            parts.append(f"Deskripsi gambar: {self.caption.caption}")
-
-        if self.detections and self.detections.count > 0:
-            summary = self.detections.labels_summary
-            items = ", ".join(f"{count}x {label}" for label, count in summary.items())
-            parts.append(f"Objek terdeteksi: {items}")
-
-        if self.classification:
-            parts.append(
-                f"Klasifikasi: {self.classification.top_label} "
-                f"(confidence: {self.classification.top_score:.1%})"
-            )
-
-        if self.ocr and self.ocr.full_text:
-            preview = self.ocr.full_text[:300]
-            if len(self.ocr.full_text) > 300:
-                preview += "..."
-            parts.append(f"Teks dalam gambar: {preview}")
-
-        return "\n".join(parts) if parts else "Tidak ada informasi yang bisa diekstrak."
-
-
-class CVPipeline:
-    """
-    Orchestrator untuk semua CV models.
-    Lazy loading — model hanya di-load saat pertama kali dipakai.
-    Support modular: bisa run satu atau semua model sekaligus.
-    """
-
-    def __init__(self):
-        self.settings = get_cv_settings()
-        self._clip: Optional[CLIPModel] = None
-        self._yolo: Optional[YOLODetector] = None
-        self._captioner: Optional[ImageCaptioner] = None
-        self._ocr: Optional[OCRProcessor] = None
-        self._setup_mlflow()
-        logger.info("CVPipeline initialized (lazy loading).")
-
-    def _setup_mlflow(self):
-        mlflow.set_tracking_uri(self.settings.mlflow_tracking_uri)
-        mlflow.set_experiment(self.settings.mlflow_experiment_name)
-
-    # === Lazy loaders ===
-
-    @property
-    def clip(self) -> CLIPModel:
-        if self._clip is None:
-            self._clip = CLIPModel()
-        return self._clip
-
-    @property
-    def yolo(self) -> YOLODetector:
-        if self._yolo is None:
-            self._yolo = YOLODetector()
-        return self._yolo
-
-    @property
-    def captioner(self) -> ImageCaptioner:
-        if self._captioner is None:
-            self._captioner = ImageCaptioner()
-        return self._captioner
-
-    @property
-    def ocr(self) -> OCRProcessor:
-        if self._ocr is None:
-            self._ocr = OCRProcessor()
-        return self._ocr
-
-    # === Main analysis methods ===
-
-    def analyze(
-        self,
-        source: Union[str, bytes, Path],
-        run_caption: bool = True,
-        run_detection: bool = True,
-        run_ocr: bool = False,
-        classification_labels: Optional[List[str]] = None,
-    ) -> CVAnalysisResult:
-        """
-        Full pipeline analisis gambar.
-
-        Args:
-            source: Path, bytes, URL, atau base64 string
-            run_caption: Generate image caption dengan BLIP
-            run_detection: Deteksi objek dengan YOLO
-            run_ocr: Ekstrak teks dengan EasyOCR
-            classification_labels: Jika diisi, jalankan zero-shot CLIP classification
-
-        Returns:
-            CVAnalysisResult berisi semua hasil
-        """
-        start = time.perf_counter()
-        image = ImagePreprocessor.load(source)
-        models_used = []
-
-        with mlflow.start_run(run_name="cv_analyze"):
-            mlflow.log_params({
-                "source": str(source)[:100],
-                "image_size": f"{image.width}x{image.height}",
-                "run_caption": run_caption,
-                "run_detection": run_detection,
-                "run_ocr": run_ocr,
-            })
-
-            result = CVAnalysisResult(
-                image_width=image.width,
-                image_height=image.height,
-                source=image.source,
-            )
-
-            # 1. Image Captioning
-            if run_caption:
-                t0 = time.perf_counter()
-                result.caption = self.captioner.caption(image)
-                models_used.append("BLIP-caption")
-                logger.debug(f"Caption: {(time.perf_counter()-t0)*1000:.0f}ms")
-
-            # 2. Object Detection
-            if run_detection:
-                t0 = time.perf_counter()
-                result.detections = self.yolo.detect(image)
-                models_used.append("YOLOv8")
-                logger.debug(f"Detection: {(time.perf_counter()-t0)*1000:.0f}ms")
-
-            # 3. Zero-shot Classification (opsional)
-            if classification_labels:
-                t0 = time.perf_counter()
-                result.classification = self.clip.classify(image, classification_labels)
-                models_used.append("CLIP")
-                logger.debug(f"CLIP: {(time.perf_counter()-t0)*1000:.0f}ms")
-
-            # 4. OCR (opsional, lebih berat)
-            if run_ocr:
-                t0 = time.perf_counter()
-                result.ocr = self.ocr.extract_text(image)
-                models_used.append("EasyOCR")
-                logger.debug(f"OCR: {(time.perf_counter()-t0)*1000:.0f}ms")
-
-            total_ms = (time.perf_counter() - start) * 1000
-            result.models_used = models_used
-            result.total_latency_ms = round(total_ms, 2)
-
-            mlflow.log_metrics({
-                "total_latency_ms": total_ms,
-                "objects_detected": result.detections.count if result.detections else 0,
-                "ocr_chars": len(result.ocr.full_text) if result.ocr else 0,
-            })
-
-        logger.info(
-            f"CV analysis done in {total_ms:.0f}ms | "
-            f"Models: {models_used} | "
-            f"Objects: {result.detections.count if result.detections else 0}"
-        )
-        return result
-
-    # === Individual task methods ===
-
-    def caption_image(self, source, prompt: str = None) -> str:
-        """Shorthand: return caption string langsung."""
-        image = ImagePreprocessor.load(source)
-        return self.captioner.caption(image, prompt=prompt).caption
-
-    def detect_objects(self, source, conf: float = None) -> DetectionResult:
-        """Shorthand: return DetectionResult."""
-        image = ImagePreprocessor.load(source)
-        return self.yolo.detect(image, conf_threshold=conf)
-
-    def classify_image(self, source, labels: List[str]) -> CLIPResult:
-        """Shorthand: zero-shot CLIP classification."""
-        image = ImagePreprocessor.load(source)
-        return self.clip.classify(image, labels)
-
-    def extract_text(self, source) -> str:
-        """Shorthand: return OCR text string."""
-        image = ImagePreprocessor.load(source)
-        return self.ocr.extract_text_simple(image)
-
-    def visual_qa(self, source, question: str) -> str:
-        """Visual Question Answering: tanya tentang isi gambar."""
-        image = ImagePreprocessor.load(source)
-        return self.captioner.visual_qa(image, question).caption
-
-    def image_text_similarity(self, source, text: str) -> float:
-        """Hitung seberapa relevan teks dengan gambar (0-1)."""
-        image = ImagePreprocessor.load(source)
-        return self.clip.compute_similarity(image, text)
-
-    def annotate_image(self, source) -> bytes:
-        """
-        Return gambar dengan bounding box yang sudah digambar — untuk visualisasi.
-        Returns JPEG bytes.
-        """
-        import io
-        from PIL import Image
-
-        image = ImagePreprocessor.load(source)
-        annotated = self.yolo.detect_and_annotate(image)
-        pil_annotated = Image.fromarray(annotated)
-        buf = io.BytesIO()
-        pil_annotated.save(buf, format="JPEG", quality=90)
-        return buf.getvalue()

cv/src/models/captioner.py

@@ -1,105 +0,0 @@
-from __future__ import annotations
-
-from dataclasses import dataclass
-from loguru import logger
-
-from ..config import get_cv_settings
-from ..processors.image_preprocessor import ImageInput
-
-
-@dataclass
-class CaptionResult:
-    caption: str
-    model: str
-    confidence: float = 1.0
-
-
-class ImageCaptioner:
-    """
-    Image captioning menggunakan BLIP (Bootstrapped Language-Image Pre-training).
-    Model Salesforce/blip-image-captioning-base — ringan, akurat, bisa jalan di CPU.
-
-    Output: deskripsi teks natural dari gambar.
-    Berguna untuk: accessibility, content indexing, multimodal RAG.
-    """
-
-    def __init__(self):
-        settings = get_cv_settings()
-        logger.info(f"Loading captioning model: {settings.caption_model}")
-
-        try:
-            from transformers import BlipProcessor, BlipForConditionalGeneration
-            import torch
-
-            self.device = settings.device
-            self.processor = BlipProcessor.from_pretrained(
-                settings.caption_model,
-                cache_dir=settings.models_cache_dir,
-            )
-            self.model = BlipForConditionalGeneration.from_pretrained(
-                settings.caption_model,
-                cache_dir=settings.models_cache_dir,
-            ).to(self.device)
-            self.model.eval()
-            self.model_name = settings.caption_model
-            logger.info("Image captioner ready.")
-
-        except Exception as e:
-            logger.error(f"Gagal load captioning model: {e}")
-            raise
-
-    def caption(
-        self,
-        image: ImageInput,
-        prompt: str = None,
-        max_new_tokens: int = 100,
-    ) -> CaptionResult:
-        """
-        Generate caption untuk gambar.
-
-        Args:
-            image: ImageInput object
-            prompt: Optional — beri konteks/instruksi, e.g. "a photo of"
-            max_new_tokens: Panjang maksimum caption
-
-        Returns:
-            CaptionResult berisi caption string
-        """
-        import torch
-
-        if prompt:
-            inputs = self.processor(
-                image.pil_image, prompt, return_tensors="pt"
-            ).to(self.device)
-        else:
-            inputs = self.processor(
-                image.pil_image, return_tensors="pt"
-            ).to(self.device)
-
-        with torch.no_grad():
-            output = self.model.generate(
-                **inputs,
-                max_new_tokens=max_new_tokens,
-                num_beams=4,
-                early_stopping=True,
-            )
-
-        caption = self.processor.decode(output[0], skip_special_tokens=True)
-
-        # Bersihkan prefix prompt dari output
-        if prompt and caption.lower().startswith(prompt.lower()):
-            caption = caption[len(prompt):].strip()
-
-        logger.debug(f"Caption: {caption}")
-
-        return CaptionResult(
-            caption=caption,
-            model=self.model_name,
-        )
-
-    def visual_qa(self, image: ImageInput, question: str) -> CaptionResult:
-        """
-        Visual Question Answering — tanya tentang isi gambar.
-        Contoh: "What color is the car?" → "red"
-        """
-        return self.caption(image, prompt=question, max_new_tokens=50)

cv/src/models/clip_model.py

@@ -1,150 +0,0 @@
-from __future__ import annotations
-
-from typing import List
-from dataclasses import dataclass
-from functools import lru_cache
-
-import torch
-import open_clip
-from loguru import logger
-
-from ..config import get_cv_settings
-from ..processors.image_preprocessor import ImageInput
-
-
-@dataclass
-class CLIPResult:
-    """Hasil dari CLIP model."""
-    # Zero-shot classification
-    labels: List[str] = None
-    probabilities: List[float] = None
-    top_label: str = ""
-    top_score: float = 0.0
-
-    # Image-text similarity
-    similarity_score: float = None
-
-    # Image features (untuk downstream tasks)
-    image_features: "torch.Tensor" = None
-
-
-class CLIPModel:
-    """
-    Wrapper CLIP menggunakan open_clip.
-    Capabilities:
-    - Zero-shot image classification (tanpa training!)
-    - Image-text similarity scoring
-    - Image feature extraction untuk retrieval
-    """
-
-    def __init__(self):
-        settings = get_cv_settings()
-        self.device = settings.device
-        logger.info(f"Loading CLIP model: {settings.clip_model} ({settings.clip_pretrained})")
-
-        self.model, _, self.preprocess = open_clip.create_model_and_transforms(
-            settings.clip_model,
-            pretrained=settings.clip_pretrained,
-            device=self.device,
-        )
-        self.tokenizer = open_clip.get_tokenizer(settings.clip_model)
-        self.model.eval()
-        logger.info("CLIP model ready.")
-
-    @torch.no_grad()
-    def classify(self, image: ImageInput, labels: List[str]) -> CLIPResult:
-        """
-        Zero-shot classification — tentukan kategori gambar dari daftar label.
-        Tidak perlu training sama sekali!
-
-        Args:
-            image: ImageInput object
-            labels: List label kandidat, e.g. ["kucing", "anjing", "burung"]
-
-        Returns:
-            CLIPResult dengan probabilitas tiap label
-        """
-        # Preprocess image
-        img_tensor = self.preprocess(image.pil_image).unsqueeze(0).to(self.device)
-
-        # Tokenize labels
-        text_tokens = self.tokenizer(labels).to(self.device)
-
-        # Compute features
-        image_features = self.model.encode_image(img_tensor)
-        text_features = self.model.encode_text(text_tokens)
-
-        # Normalize
-        image_features /= image_features.norm(dim=-1, keepdim=True)
-        text_features /= text_features.norm(dim=-1, keepdim=True)
-
-        # Compute similarity (cosine similarity → softmax → probabilities)
-        logits = (100.0 * image_features @ text_features.T).softmax(dim=-1)
-        probs = logits[0].cpu().numpy().tolist()
-
-        top_idx = int(torch.argmax(logits[0]).item())
-
-        return CLIPResult(
-            labels=labels,
-            probabilities=[round(p, 4) for p in probs],
-            top_label=labels[top_idx],
-            top_score=round(probs[top_idx], 4),
-        )
-
-    @torch.no_grad()
-    def compute_similarity(self, image: ImageInput, text: str) -> float:
-        """
-        Hitung seberapa relevan teks dengan gambar (0.0 - 1.0).
-        Berguna untuk: image search, content moderation, caption scoring.
-        """
-        img_tensor = self.preprocess(image.pil_image).unsqueeze(0).to(self.device)
-        text_tokens = self.tokenizer([text]).to(self.device)
-
-        img_feat = self.model.encode_image(img_tensor)
-        txt_feat = self.model.encode_text(text_tokens)
-
-        img_feat /= img_feat.norm(dim=-1, keepdim=True)
-        txt_feat /= txt_feat.norm(dim=-1, keepdim=True)
-
-        similarity = (img_feat @ txt_feat.T).item()
-
-        # Normalize ke 0-1 (CLIP output biasanya -1 to 1)
-        return round((similarity + 1) / 2, 4)
-
-    @torch.no_grad()
-    def extract_features(self, image: ImageInput) -> "torch.Tensor":
-        """
-        Ekstrak image embedding untuk semantic image search / clustering.
-        Output: tensor shape (512,) untuk ViT-B-32
-        """
-        img_tensor = self.preprocess(image.pil_image).unsqueeze(0).to(self.device)
-        features = self.model.encode_image(img_tensor)
-        features /= features.norm(dim=-1, keepdim=True)
-        return features[0].cpu()
-
-    @torch.no_grad()
-    def rank_images_by_text(
-        self,
-        images: List[ImageInput],
-        query_text: str,
-    ) -> List[tuple[int, float]]:
-        """
-        Rank multiple images berdasarkan relevansi dengan teks query.
-        Returns: list of (original_index, score) sorted by score desc.
-        Berguna untuk: text-to-image search.
-        """
-        tensors = torch.stack([
-            self.preprocess(img.pil_image) for img in images
-        ]).to(self.device)
-
-        text_tokens = self.tokenizer([query_text]).to(self.device)
-
-        img_features = self.model.encode_image(tensors)
-        txt_features = self.model.encode_text(text_tokens)
-
-        img_features /= img_features.norm(dim=-1, keepdim=True)
-        txt_features /= txt_features.norm(dim=-1, keepdim=True)
-
-        scores = (img_features @ txt_features.T).squeeze(1).cpu().numpy()
-        ranked = sorted(enumerate(scores.tolist()), key=lambda x: x[1], reverse=True)
-        return [(idx, round(score, 4)) for idx, score in ranked]

cv/src/models/yolo_detector.py

@@ -1,208 +0,0 @@
-from __future__ import annotations
-
-from typing import List
-from dataclasses import dataclass, field
-
-import numpy as np
-from loguru import logger
-
-from ..config import get_cv_settings
-from ..processors.image_preprocessor import ImageInput
-
-
-@dataclass
-class BoundingBox:
-    x1: float
-    y1: float
-    x2: float
-    y2: float
-
-    @property
-    def width(self) -> float:
-        return self.x2 - self.x1
-
-    @property
-    def height(self) -> float:
-        return self.y2 - self.y1
-
-    @property
-    def area(self) -> float:
-        return self.width * self.height
-
-    def to_dict(self) -> dict:
-        return {
-            "x1": round(self.x1, 1), "y1": round(self.y1, 1),
-            "x2": round(self.x2, 1), "y2": round(self.y2, 1),
-            "width": round(self.width, 1), "height": round(self.height, 1),
-        }
-
-
-@dataclass
-class Detection:
-    label: str
-    confidence: float
-    bbox: BoundingBox
-    class_id: int
-
-    def to_dict(self) -> dict:
-        return {
-            "label": self.label,
-            "confidence": round(self.confidence, 4),
-            "bbox": self.bbox.to_dict(),
-            "class_id": self.class_id,
-        }
-
-
-@dataclass
-class DetectionResult:
-    detections: List[Detection] = field(default_factory=list)
-    image_width: int = 0
-    image_height: int = 0
-    model_name: str = ""
-    inference_time_ms: float = 0.0
-
-    @property
-    def count(self) -> int:
-        return len(self.detections)
-
-    @property
-    def labels_summary(self) -> dict[str, int]:
-        """Ringkasan: {label: count}"""
-        summary = {}
-        for d in self.detections:
-            summary[d.label] = summary.get(d.label, 0) + 1
-        return summary
-
-    def filter_by_label(self, label: str) -> List[Detection]:
-        return [d for d in self.detections if d.label.lower() == label.lower()]
-
-    def filter_by_confidence(self, min_conf: float) -> List[Detection]:
-        return [d for d in self.detections if d.confidence >= min_conf]
-
-
-class YOLODetector:
-    """
-    Object detection menggunakan YOLOv8 (Ultralytics).
-    Model: yolov8n (nano, cepat) → yolov8m (medium, akurat)
-    80 kelas COCO default, bisa di-finetune untuk domain spesifik.
-    """
-
-    def __init__(self):
-        settings = get_cv_settings()
-        logger.info(f"Loading YOLO model: {settings.yolo_model}")
-
-        try:
-            from ultralytics import YOLO
-            self.model = YOLO(settings.yolo_model)
-        except Exception as e:
-            logger.error(f"Gagal load YOLO: {e}")
-            raise
-
-        self.conf_threshold = settings.yolo_conf_threshold
-        self.iou_threshold = settings.yolo_iou_threshold
-        self.model_name = settings.yolo_model
-        logger.info("YOLO detector ready.")
-
-    def detect(
-        self,
-        image: ImageInput,
-        conf_threshold: float = None,
-        classes: List[int] = None,
-    ) -> DetectionResult:
-        """
-        Deteksi objek dalam gambar.
-
-        Args:
-            image: ImageInput object
-            conf_threshold: Override confidence threshold (default dari config)
-            classes: Filter kelas spesifik (COCO class IDs), None = semua kelas
-
-        Returns:
-            DetectionResult berisi semua deteksi
-        """
-        import time
-        conf = conf_threshold or self.conf_threshold
-
-        start = time.perf_counter()
-        results = self.model.predict(
-            source=image.numpy,
-            conf=conf,
-            iou=self.iou_threshold,
-            classes=classes,
-            verbose=False,
-        )
-        elapsed_ms = (time.perf_counter() - start) * 1000
-
-        detections = []
-        if results and results[0].boxes is not None:
-            boxes = results[0].boxes
-            for i in range(len(boxes)):
-                x1, y1, x2, y2 = boxes.xyxy[i].cpu().numpy()
-                conf_val = float(boxes.conf[i].cpu().numpy())
-                cls_id = int(boxes.cls[i].cpu().numpy())
-                label = self.model.names[cls_id]
-
-                detections.append(Detection(
-                    label=label,
-                    confidence=conf_val,
-                    bbox=BoundingBox(x1=x1, y1=y1, x2=x2, y2=y2),
-                    class_id=cls_id,
-                ))
-
-        logger.debug(
-            f"Detected {len(detections)} objects in {elapsed_ms:.1f}ms | "
-            f"Summary: {DetectionResult(detections=detections).labels_summary}"
-        )
-
-        return DetectionResult(
-            detections=detections,
-            image_width=image.width,
-            image_height=image.height,
-            model_name=self.model_name,
-            inference_time_ms=round(elapsed_ms, 2),
-        )
-
-    def detect_and_annotate(self, image: ImageInput, **kwargs) -> "np.ndarray":
-        """
-        Detect dan return gambar dengan bounding box yang sudah digambar.
-        Berguna untuk visualisasi / demo.
-        """
-        import cv2
-
-        result_img = image.numpy.copy()
-        det_result = self.detect(image, **kwargs)
-
-        for det in det_result.detections:
-            bb = det.bbox
-            x1, y1, x2, y2 = int(bb.x1), int(bb.y1), int(bb.x2), int(bb.y2)
-
-            # Warna berdasarkan class_id
-            color = self._get_color(det.class_id)
-
-            # Gambar bounding box
-            cv2.rectangle(result_img, (x1, y1), (x2, y2), color, 2)
-
-            # Label background + text
-            label_text = f"{det.label} {det.confidence:.2f}"
-            (tw, th), _ = cv2.getTextSize(label_text, cv2.FONT_HERSHEY_SIMPLEX, 0.6, 1)
-            cv2.rectangle(result_img, (x1, y1 - th - 8), (x1 + tw + 4, y1), color, -1)
-            cv2.putText(result_img, label_text, (x1 + 2, y1 - 4),
-                       cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 255, 255), 1)
-
-        return result_img
-
-    @staticmethod
-    def _get_color(class_id: int) -> tuple[int, int, int]:
-        """Generate warna konsisten per class_id."""
-        palette = [
-            (255, 56, 56), (255, 157, 151), (255, 112, 31), (255, 178, 29),
-            (207, 210, 49), (72, 249, 10), (146, 204, 23), (61, 219, 134),
-            (26, 147, 52), (0, 212, 187), (44, 153, 168), (0, 194, 255),
-            (52, 69, 147), (100, 115, 255), (0, 24, 236), (132, 56, 255),
-        ]
-        return palette[class_id % len(palette)]
-
-    @property
-    def available_classes(self) -> dict[int, str]:
-        """Return dict semua kelas yang bisa dideteksi."""
-        return self.model.names

cv/src/processors/image_preprocessor.py

@@ -1,154 +0,0 @@
-from __future__ import annotations
-
-import io
-import base64
-from pathlib import Path
-from typing import Union
-from dataclasses import dataclass, field
-
-import numpy as np
-from PIL import Image, ExifTags
-from loguru import logger
-
-
-@dataclass
-class ImageInput:
-    """Normalized image container — semua sumber dikonversi ke sini."""
-    pil_image: Image.Image
-    original_size: tuple[int, int]   # (width, height)
-    source: str = "unknown"
-    filename: str = ""
-    format: str = "RGB"
-    metadata: dict = field(default_factory=dict)
-
-    @property
-    def width(self) -> int:
-        return self.pil_image.width
-
-    @property
-    def height(self) -> int:
-        return self.pil_image.height
-
-    @property
-    def numpy(self) -> np.ndarray:
-        """Return as HWC uint8 numpy array (untuk OpenCV/YOLO)."""
-        return np.array(self.pil_image)
-
-    def to_base64(self) -> str:
-        """Konversi ke base64 string untuk response API."""
-        buf = io.BytesIO()
-        self.pil_image.save(buf, format="JPEG", quality=85)
-        return base64.b64encode(buf.getvalue()).decode()
-
-
-class ImagePreprocessor:
-    """
-    Handle semua bentuk input gambar:
-    - File path (str / Path)
-    - Raw bytes (dari upload)
-    - Base64 string
-    - URL (via HTTP)
-    - PIL Image langsung
-    """
-
-    MAX_SIZE = (1920, 1920)
-
-    @classmethod
-    def load(cls, source: Union[str, bytes, Path, Image.Image]) -> ImageInput:
-        """Auto-detect tipe input dan load sebagai ImageInput."""
-        if isinstance(source, Image.Image):
-            return cls._from_pil(source, source_name="pil_direct")
-
-        if isinstance(source, bytes):
-            return cls._from_bytes(source)
-
-        if isinstance(source, Path) or (isinstance(source, str) and not source.startswith(("http", "data:"))):
-            return cls._from_file(str(source))
-
-        if isinstance(source, str) and source.startswith("data:image"):
-            return cls._from_base64(source)
-
-        if isinstance(source, str) and source.startswith(("http://", "https://")):
-            return cls._from_url(source)
-
-        raise ValueError(f"Tipe input tidak dikenali: {type(source)}")
-
-    @classmethod
-    def _from_file(cls, path: str) -> ImageInput:
-        p = Path(path)
-        if not p.exists():
-            raise FileNotFoundError(f"Gambar tidak ditemukan: {path}")
-        img = Image.open(p)
-        img = cls._normalize(img)
-        logger.debug(f"Loaded image from file: {p.name} ({img.width}x{img.height})")
-        return ImageInput(
-            pil_image=img,
-            original_size=(img.width, img.height),
-            source="file",
-            filename=p.name,
-            metadata={"path": str(p), "format": p.suffix},
-        )
-
-    @classmethod
-    def _from_bytes(cls, data: bytes, filename: str = "upload") -> ImageInput:
-        img = Image.open(io.BytesIO(data))
-        original_size = (img.width, img.height)
-        img = cls._normalize(img)
-        return ImageInput(
-            pil_image=img,
-            original_size=original_size,
-            source="bytes",
-            filename=filename,
-            metadata={"size_bytes": len(data)},
-        )
-
-    @classmethod
-    def _from_base64(cls, b64_str: str) -> ImageInput:
-        # Strip data URI prefix jika ada
-        if "," in b64_str:
-            b64_str = b64_str.split(",", 1)[1]
-        data = base64.b64decode(b64_str)
-        return cls._from_bytes(data, filename="base64_input")
-
-    @classmethod
-    def _from_url(cls, url: str) -> ImageInput:
-        import httpx
-        logger.debug(f"Fetching image from URL: {url}")
-        r = httpx.get(url, timeout=15, follow_redirects=True)
-        r.raise_for_status()
-        img_input = cls._from_bytes(r.content, filename=url.split("/")[-1] or "url_image")
-        img_input.source = "url"
-        img_input.metadata["url"] = url
-        return img_input
-
-    @classmethod
-    def _from_pil(cls, img: Image.Image, source_name: str = "pil") -> ImageInput:
-        original_size = (img.width, img.height)
-        img = cls._normalize(img)
-        return ImageInput(pil_image=img, original_size=original_size, source=source_name)
-
-    @classmethod
-    def _normalize(cls, img: Image.Image) -> Image.Image:
-        """Convert ke RGB, fix EXIF rotation, resize jika terlalu besar."""
-        # Fix EXIF orientation
-        try:
-            exif = img._getexif()
-            if exif:
-                for tag, val in exif.items():
-                    if ExifTags.TAGS.get(tag) == "Orientation":
-                        rotations = {3: 180, 6: 270, 8: 90}
-                        if val in rotations:
-                            img = img.rotate(rotations[val], expand=True)
-        except Exception:
-            pass
-
-        # Convert ke RGB
-        if img.mode != "RGB":
-            img = img.convert("RGB")
-
-        # Resize jika melebihi batas
-        if img.width > cls.MAX_SIZE[0] or img.height > cls.MAX_SIZE[1]:
-            img.thumbnail(cls.MAX_SIZE, Image.LANCZOS)
-            logger.debug(f"Resized image to {img.width}x{img.height}")
-
-        return img

cv/src/processors/ocr_processor.py

@@ -1,235 +0,0 @@
-from __future__ import annotations
-
-from typing import List
-from dataclasses import dataclass, field
-from loguru import logger
-
-import numpy as np
-
-from ..config import get_cv_settings
-from ..processors.image_preprocessor import ImageInput
-
-
-@dataclass
-class OCRBox:
-    text: str
-    confidence: float
-    bbox: list   # [[x1,y1],[x2,y1],[x2,y2],[x1,y2]] format EasyOCR
-
-    def to_dict(self) -> dict:
-        return {
-            "text": self.text,
-            "confidence": round(self.confidence, 4),
-            "bbox": self.bbox,
-        }
-
-
-@dataclass
-class OCRResult:
-    full_text: str
-    boxes: List[OCRBox] = field(default_factory=list)
-    language: str = ""
-    engine: str = ""
-
-    @property
-    def word_count(self) -> int:
-        return len(self.full_text.split())
-
-
-class OCRProcessor:
-    """
-    OCR menggunakan EasyOCR dengan mode stabil (single-pass ringan).
-    Fokus: tidak crash di Docker + tetap improve akurasi.
-    """
-
-    MIN_CONFIDENCE = 0.10
-    MIN_OCR_DIM = 800
-
-    def __init__(self):
-        settings = get_cv_settings()
-        self.engine = settings.ocr_engine
-        self.languages = [l.strip() for l in settings.ocr_languages.split(",")]
-        logger.info(f"Loading OCR ({self.engine}) for languages: {self.languages}")
-
-        try:
-            import easyocr
-            self.reader = easyocr.Reader(
-                self.languages,
-                gpu=(settings.device == "cuda"),
-                model_storage_directory=settings.models_cache_dir,
-            )
-            logger.info("OCR processor ready.")
-        except Exception as e:
-            logger.error(f"Gagal init OCR: {e}")
-            raise
-
-    def _preprocess_for_ocr(self, img: np.ndarray) -> np.ndarray:
-        """
-        Preprocessing ringan:
-        - upscale (jika kecil)
-        - grayscale
-        - CLAHE contrast enhancement
-        - light sharpen
-        """
-        try:
-            import cv2
-
-            h, w = img.shape[:2]
-            if max(h, w) < self.MIN_OCR_DIM:
-                scale = self.MIN_OCR_DIM / max(h, w)
-                new_w, new_h = int(w * scale), int(h * scale)
-                img = cv2.resize(img, (new_w, new_h), interpolation=cv2.INTER_CUBIC)
-
-            if len(img.shape) == 3:
-                gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
-            else:
-                gray = img.copy()
-
-            clahe = cv2.createCLAHE(clipLimit=2.5, tileGridSize=(8, 8))
-            enhanced = clahe.apply(gray)
-
-            kernel = np.array([[0, -1, 0], [-1, 5, -1], [0, -1, 0]], dtype=np.float32)
-            sharpened = cv2.filter2D(enhanced, -1, kernel)
-
-            return cv2.cvtColor(sharpened, cv2.COLOR_GRAY2RGB)
-        except Exception as e:
-            logger.warning(f"OCR preprocessing fallback to original image: {e}")
-            return img
-
-    def _parse_results(self, raw_results: List) -> List[OCRBox]:
-        boxes = []
-        for item in raw_results:
-            if len(item) == 3:
-                bbox, text, confidence = item
-            elif len(item) == 2:
-                bbox, text = item
-                confidence = 0.8
-            else:
-                continue
-
-            text = str(text).strip()
-            if not text or confidence < self.MIN_CONFIDENCE:
-                continue
-
-            # Convert numpy scalars/arrays to native Python types for FastAPI/Pydantic serialization
-            safe_bbox = []
-            try:
-                for pt in bbox:
-                    if isinstance(pt, (list, tuple)) and len(pt) >= 2:
-                        safe_bbox.append([float(pt[0]), float(pt[1])])
-                    else:
-                        safe_bbox.append(pt)
-            except Exception:
-                safe_bbox = bbox
-
-            boxes.append(OCRBox(
-                text=text,
-                confidence=float(confidence),
-                bbox=safe_bbox,
-            ))
-        return boxes
-
-    def _boxes_to_text(self, boxes: List[OCRBox]) -> str:
-        if not boxes:
-            return ""
-
-        def cy(box: OCRBox) -> float:
-            try:
-                ys = [pt[1] for pt in box.bbox]
-                return sum(ys) / len(ys)
-            except Exception:
-                return 0
-
-        def cx(box: OCRBox) -> float:
-            try:
-                xs = [pt[0] for pt in box.bbox]
-                return sum(xs) / len(xs)
-            except Exception:
-                return 0
-
-        def h(box: OCRBox) -> float:
-            try:
-                ys = [pt[1] for pt in box.bbox]
-                return max(ys) - min(ys)
-            except Exception:
-                return 20
-
-        sorted_boxes = sorted(boxes, key=lambda b: (cy(b), cx(b)))
-        lines = []
-        current = [sorted_boxes[0]]
-        current_y = cy(sorted_boxes[0])
-
-        for box in sorted_boxes[1:]:
-            if abs(cy(box) - current_y) < max(h(box) * 0.5, 15):
-                current.append(box)
-            else:
-                current.sort(key=lambda b: cx(b))
-                lines.append(" ".join(b.text for b in current))
-                current = [box]
-                current_y = cy(box)
-
-        if current:
-            current.sort(key=lambda b: cx(b))
-            lines.append(" ".join(b.text for b in current))
-
-        return "\n".join(lines)
-
-    def extract_text(
-        self,
-        image: ImageInput,
-        detail: bool = True,
-        paragraph: bool = False,
-    ) -> OCRResult:
-        logger.debug(f"Running stable OCR on {image.width}x{image.height} image")
-
-        try:
-            processed = self._preprocess_for_ocr(image.numpy.copy())
-            raw_results = self.reader.readtext(
-                processed,
-                detail=1,
-                paragraph=False,
-                contrast_ths=0.1,
-                adjust_contrast=0.7,
-                text_threshold=0.5,
-                low_text=0.3,
-                link_threshold=0.3,
-                width_ths=0.7,
-                decoder="beamsearch",
-                beamWidth=10,
-            )
-            boxes = self._parse_results(raw_results)
-
-            if len(boxes) < 2:
-                raw2 = self.reader.readtext(
-                    image.numpy,
-                    detail=1,
-                    paragraph=False,
-                )
-                boxes2 = self._parse_results(raw2)
-                if len(boxes2) > len(boxes):
-                    boxes = boxes2
-
-            full_text = self._boxes_to_text(boxes)
-
-            return OCRResult(
-                full_text=full_text,
-                boxes=boxes,
-                language=",".join(self.languages),
-                engine=self.engine,
-            )
-
-        except Exception as e:
-            logger.error(f"OCR processing error: {e}")
-            raw_results = self.reader.readtext(image.numpy, detail=1, paragraph=False)
-            boxes = self._parse_results(raw_results)
-            full_text = self._boxes_to_text(boxes)
-            return OCRResult(
-                full_text=full_text,
-                boxes=boxes,
-                language=",".join(self.languages),
-                engine=self.engine,
-            )
-
-    def extract_text_simple(self, image: ImageInput) -> str:
-        result = self.extract_text(image, detail=True, paragraph=False)
-        return result.full_text

cv_module/src/api/routes.py

@@ -34,6 +34,10 @@ router = APIRouter()
 _pipeline: CVPipeline = None
 _pipeline_lock = threading.Lock()
 
+# Lock terpisah untuk trigger lazy-load — mencegah TOCTOU race kalau
+# beberapa request datang barengan untuk model yang sama.
+_trigger_lock = threading.Lock()
+
 
 def get_pipeline() -> CVPipeline:
     global _pipeline
@@ -57,29 +61,37 @@ def _trigger_and_wait(model_name: str):
     Thread-safe: hanya satu thread yang load, sisanya tunggu.
     """
     readiness = get_readiness()
-    status_info = readiness.get_status(model_name)
 
-    # Kalau sudah ready, langsung return.
+    # Atomic check-and-mark: hold _trigger_lock biar dua request ga sama-sama
+    # nge-spawn loader thread untuk model yang sama (CVPipeline punya per-model
+    # lock, tapi spawning extra thread tetep waste resource & log noise).
+    with _trigger_lock:
+        status_info = readiness.get_status(model_name)
+
+        # Kalau error, langsung raise.
+        if status_info.state.value == "error":
+            raise HTTPException(
+                status_code=503,
+                detail={
+                    "error": "model_failed_to_load",
+                    "model": model_name,
+                    "message": status_info.error_message or "Model gagal dimuat.",
+                    "hint": "Cek logs container untuk detail error.",
+                },
+            )
+
+        need_spawn = status_info.state.value in ("not_loaded",)
+        if need_spawn:
+            # Mark loading dulu — request berikutnya yang masuk akan lihat
+            # state="loading" dan langsung wait_for() tanpa spawn thread baru.
+            readiness.mark_loading(model_name)
+
+    # Kalau sudah ready, langsung return (tidak perlu wait).
     if status_info.state.value == "ready":
         return
 
-    # Kalau error, langsung raise.
-    if status_info.state.value == "error":
-        raise HTTPException(
-            status_code=503,
-            detail={
-                "error": "model_failed_to_load",
-                "model": model_name,
-                "message": status_info.error_message or "Model gagal dimuat.",
-                "hint": "Cek logs container untuk detail error.",
-            },
-        )
-
-    # Kalau belum loading (not_loaded) — trigger load via pipeline property access.
-    # ReadinessTracker akan di-update oleh pipeline lazy loader.
-    if status_info.state.value in ("not_loaded",):
-        readiness.mark_loading(model_name)
-        # Trigger load di thread baru supaya tidak block event loop.
+    # Spawn loader thread di luar lock supaya request lain bisa masuk.
+    if need_spawn:
         def _do_load():
             try:
                 p = get_pipeline()
@@ -104,6 +116,18 @@ def _trigger_and_wait(model_name: str):
     ok = readiness.wait_for(model_name, timeout=_MODEL_WAIT_TIMEOUT)
     if not ok:
         current = readiness.get_status(model_name).state.value
+        # Kalau state-nya error, kasih pesan error spesifik.
+        if current == "error":
+            err_msg = readiness.get_status(model_name).error_message
+            raise HTTPException(
+                status_code=503,
+                detail={
+                    "error": "model_failed_to_load",
+                    "model": model_name,
+                    "message": err_msg or f"Model '{model_name}' gagal dimuat.",
+                    "hint": "Cek logs container untuk traceback lengkap.",
+                },
+            )
         raise HTTPException(
             status_code=503,
             detail={
@@ -230,7 +254,8 @@ async def caption(url: str, prompt: str = None):
     except HTTPException:
         raise
     except Exception as e:
-        raise HTTPException(500, detail=str(e))
+        logger.error(f"Caption error: {e}")
+        raise HTTPException(500, detail=f"Caption gagal: {e}")
 
 
 @router.post("/detect", response_model=DetectionResponse, tags=["tasks"])
@@ -250,7 +275,8 @@ async def detect(url: str, conf: float = None):
     except HTTPException:
         raise
     except Exception as e:
-        raise HTTPException(500, detail=str(e))
+        logger.error(f"Detect error: {e}")
+        raise HTTPException(500, detail=f"Detect gagal: {e}")
 
 
 @router.post("/classify", response_model=ClassificationResponse, tags=["tasks"])
@@ -268,7 +294,8 @@ async def classify(req: ClassifyRequest):
     except HTTPException:
         raise
     except Exception as e:
-        raise HTTPException(500, detail=str(e))
+        logger.error(f"Classify error: {e}")
+        raise HTTPException(500, detail=f"Classify gagal: {e}")
 
 
 class OCRRequest(BaseModel):
@@ -293,7 +320,8 @@ async def ocr(req: OCRRequest):
     except HTTPException:
         raise
     except Exception as e:
-        raise HTTPException(500, detail=str(e))
+        logger.error(f"OCR error: {e}")
+        raise HTTPException(500, detail=f"OCR gagal: {e}")
 
 
 @router.post("/ocr/bytes", tags=["tasks"])

cv_module/src/cv_pipeline.py

@@ -69,38 +69,61 @@ class CVPipeline:
     Orchestrator untuk semua CV models.
     Lazy loading — model hanya di-load saat pertama kali dipakai.
     Support modular: bisa run satu atau semua model sekaligus.
+
+    Thread-safe: setiap model property pakai per-model lock supaya 2+ request
+    yang concurrent tidak load model yang sama secara duplicate (bisa OOM
+    di HF free tier yang RAM-nya cuma 16GB shared).
     """
 
+    # Cap ThreadPoolExecutor workers untuk analyze() — tanpa cap, default
+    # min(32, os.cpu_count()+4) bisa bikin OOM kalau semua model jalan paralel
+    # plus model loading butuh RAM. 2 cukup buat overlap I/O + compute.
+    _MAX_PARALLEL_TASKS = 2
+
     def __init__(self):
+        import threading
         self.settings = get_cv_settings()
         self._clip: Optional[CLIPModel] = None
         self._yolo: Optional[YOLODetector] = None
         self._captioner: Optional[ImageCaptioner] = None
         self._ocr: Optional[OCRProcessor] = None
-        logger.info("CVPipeline initialized (lazy loading).")
+        # Per-model locks — mencegah double-init kalau 2+ thread access barengan.
+        self._lock_clip = threading.Lock()
+        self._lock_yolo = threading.Lock()
+        self._lock_captioner = threading.Lock()
+        self._lock_ocr = threading.Lock()
+        logger.info("CVPipeline initialized (lazy loading, thread-safe).")
 
     @property
     def clip(self) -> CLIPModel:
         if self._clip is None:
-            self._clip = CLIPModel()
+            with self._lock_clip:
+                if self._clip is None:  # double-check after lock
+                    self._clip = CLIPModel()
         return self._clip
 
     @property
     def yolo(self) -> YOLODetector:
         if self._yolo is None:
-            self._yolo = YOLODetector()
+            with self._lock_yolo:
+                if self._yolo is None:
+                    self._yolo = YOLODetector()
         return self._yolo
 
     @property
     def captioner(self) -> ImageCaptioner:
         if self._captioner is None:
-            self._captioner = ImageCaptioner()
+            with self._lock_captioner:
+                if self._captioner is None:
+                    self._captioner = ImageCaptioner()
         return self._captioner
 
     @property
     def ocr(self) -> OCRProcessor:
         if self._ocr is None:
-            self._ocr = OCRProcessor()
+            with self._lock_ocr:
+                if self._ocr is None:
+                    self._ocr = OCRProcessor()
         return self._ocr
 
     # === Main analysis methods ===
@@ -139,9 +162,12 @@ class CVPipeline:
             source=image.source,
         )
 
-        # Jalankan semua model secara paralel untuk menghindari timeout
+        # Jalankan semua model secara paralel untuk menghindari timeout.
+        # max_workers=2 — cukup untuk overlap I/O & compute, ngga ngebebanin RAM.
+        # Tanpa cap, default workers bisa bikin 4 model jalan barengan + ngeload
+        # weight-nya barengan → OOM di HF free tier.
         tasks = {}
-        with concurrent.futures.ThreadPoolExecutor() as executor:
+        with concurrent.futures.ThreadPoolExecutor(max_workers=self._MAX_PARALLEL_TASKS) as executor:
             if run_caption:
                 tasks["caption"] = executor.submit(self.captioner.caption, image)
             if run_detection:

frontend/index.html

@@ -9,15 +9,54 @@
 // Anti-flash theme init. Key: 'ai-rag-theme'.
 // null/missing = follow device preference (re-read every load).
 // 'dark' or 'light' = user manually overrode.
+//
+// IMPORTANT: wrap in try/catch — beberapa browser (private mode, strict cookie
+// policy, embedded WebView) nge-block localStorage atau matchMedia → kalau script
+// throw, data-theme tidak pernah ke-set, CSS variables semua undefined, dan
+// tampilan keliatan "ilang temanya". Default ke 'light' kalau apa pun gagal.
 (function(){
-  var s = localStorage.getItem('ai-rag-theme');
-  var t = (s === 'dark' || s === 'light') ? s
-        : (window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light');
+  var t = 'light';
+  try {
+    var s = null;
+    try { s = window.localStorage.getItem('ai-rag-theme'); } catch (_) {}
+    if (s === 'dark' || s === 'light') {
+      t = s;
+    } else if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
+      t = 'dark';
+    }
+  } catch (_) { /* fall through to light */ }
   document.documentElement.setAttribute('data-theme', t);
 })();
 </script>
 <style>
 /* ── Theme tokens ───────────────────────────────────── */
+/* Fallback default (light) — applies when data-theme attribute is missing
+   or invalid. Without this, CSS vars all undefined → blank/white-on-white UI.
+   The inline script in <head> ALWAYS sets data-theme, but defense-in-depth
+   protects us against weird browsers / extensions / errors. */
+:root {
+  --bg0: #f4f5f7;
+  --bg1: #ffffff;
+  --bg2: #eef0f3;
+  --bg3: #e4e6eb;
+  --line: #d0d4db;
+  --line2: #bcc0c9;
+  --ink0: #1a1d24;
+  --ink1: #4a5263;
+  --ink2: #7a8296;
+  --ink3: #a8afc0;
+  --sage: #4a7a5e;
+  --sage-l: #357a52;
+  --sage-bg: #eaf4ee;
+  --amber: #8c6d3f;
+  --amber-l: #7a5820;
+  --amber-bg: #fdf3e3;
+  --sky: #2a5f80;
+  --sky-l: #1e5070;
+  --red: #8c3a3a;
+  --red-l: #9e2a2a;
+  --shadow: rgba(0,0,0,0.08);
+}
 :root[data-theme="dark"] {
   --bg0: #111214;
   --bg1: #16181c;
@@ -515,21 +554,25 @@ let queryCount=0;
 // === Theme ===
 function applyTheme(theme, manual){
   document.documentElement.setAttribute('data-theme', theme);
-  // Only persist if user manually toggled (not from system change or init)
+  // Only persist if user manually toggled (not from system change or init).
+  // Wrap in try/catch — localStorage can throw in private mode, embedded
+  // WebViews, or with strict cookie policies.
   if (manual) {
-    localStorage.setItem('ai-rag-theme', theme);
+    try { localStorage.setItem('ai-rag-theme', theme); } catch (_) {}
   }
   // icon-sun shown when dark (click -> go light)
   // icon-moon shown when light (click -> go dark)
   const isDark = theme === 'dark';
-  document.getElementById('icon-sun').style.display = isDark ? '' : 'none';
-  document.getElementById('icon-moon').style.display = isDark ? 'none' : '';
-  // Label shows current mode
-  document.getElementById('theme-label').textContent = theme;
+  const sun = document.getElementById('icon-sun');
+  const moon = document.getElementById('icon-moon');
+  const label = document.getElementById('theme-label');
+  if (sun) sun.style.display = isDark ? '' : 'none';
+  if (moon) moon.style.display = isDark ? 'none' : '';
+  if (label) label.textContent = theme;
 }
 
 function toggleTheme(){
-  const cur = document.documentElement.getAttribute('data-theme');
+  const cur = document.documentElement.getAttribute('data-theme') || 'light';
   applyTheme(cur === 'dark' ? 'light' : 'dark', true);
 }
 
@@ -540,14 +583,20 @@ function toggleTheme(){
   applyTheme(theme, false);
 })();
 
-// Listen for system theme changes (only if user hasn't manually overridden)
-// Follow system preference changes ONLY if user has no manual override
-window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', function(e){
-  var stored = localStorage.getItem('ai-rag-theme');
-  if (stored !== 'dark' && stored !== 'light') {
-    applyTheme(e.matches ? 'dark' : 'light', false);
-  }
-});
+// Listen for system theme changes (only if user hasn't manually overridden).
+// Wrap in try/catch — matchMedia.addEventListener missing in old browsers.
+try {
+  const mq = window.matchMedia('(prefers-color-scheme: dark)');
+  const handler = function(e){
+    var stored = null;
+    try { stored = localStorage.getItem('ai-rag-theme'); } catch (_) {}
+    if (stored !== 'dark' && stored !== 'light') {
+      applyTheme(e.matches ? 'dark' : 'light', false);
+    }
+  };
+  if (mq.addEventListener) mq.addEventListener('change', handler);
+  else if (mq.addListener) mq.addListener(handler);  // Safari < 14
+} catch (_) {}
 
 // === Utility ===
 function tick(){document.getElementById('clock').textContent=new Date().toLocaleTimeString('id-ID',{hour:'2-digit',minute:'2-digit',second:'2-digit'})}

rag-requirements.txt

@@ -1,34 +0,0 @@
-# ── LLM & Orchestration ──────────────────────────────────
-langchain==0.2.16
-langchain-groq==0.1.9
-langchain-community==0.2.16
-langchain-chroma==0.1.4
-
-# ── Vector Store ──────────────────────────────────────────
-chromadb==0.5.3
-
-# ── Embeddings ────────────────────────────────────────────
-sentence-transformers==2.7.0
-transformers==4.35.2
-numpy==1.26.4
-
-# ── Document Loaders ──────────────────────────────────────
-pypdf==4.3.1
-python-docx==1.1.2
-beautifulsoup4==4.12.3
-requests==2.32.3
-
-# ── API ───────────────────────────────────────────────────
-fastapi==0.112.0
-uvicorn[standard]==0.30.6
-python-multipart==0.0.9
-pydantic==2.8.2
-pydantic-settings==2.4.0
-
-# ── MLOps ─────────────────────────────────────────────────
-mlflow==2.15.1
-
-# ── Utils ─────────────────────────────────────────────────
-python-dotenv==1.0.1
-loguru==0.7.2
-httpx==0.27.0

rag/src/__init__.py

@@ -1 +0,0 @@
-# RAG Pipeline

rag/src/api/__init__.py

@@ -1 +0,0 @@
-

rag/src/api/main.py

@@ -1,57 +0,0 @@
-from fastapi import FastAPI
-from fastapi.middleware.cors import CORSMiddleware
-from loguru import logger
-import sys
-
-from .routes import router
-from ..config import get_settings
-
-# === Logging setup ===
-settings = get_settings()
-logger.remove()
-logger.add(sys.stderr, level=settings.log_level, colorize=True,
-           format="<green>{time:HH:mm:ss}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan> - {message}")
-logger.add(settings.log_file, rotation="10 MB", retention="7 days", level="DEBUG")
-
-# === App ===
-app = FastAPI(
-    title="RAG Pipeline API",
-    description="""
-## Multimodal AI Assistant — RAG Module
-
-Endpoint untuk indexing dan querying dokumen menggunakan:
-- **Groq** (LLM inference — cepat & gratis)
-- **Sentence Transformers** (local embeddings)
-- **ChromaDB** (vector store persistent)
-- **MLflow** (experiment tracking)
-
-### Supported document types
-PDF, TXT, Markdown, DOCX, JSON, JSONL, URL
-    """,
-    version="1.0.0",
-    docs_url="/docs",
-    redoc_url="/redoc",
-)
-
-# === CORS ===
-app.add_middleware(
-    CORSMiddleware,
-    allow_origins=["*"],  # Ganti dengan domain spesifik di production
-    allow_credentials=True,
-    allow_methods=["*"],
-    allow_headers=["*"],
-)
-
-# === Routes ===
-app.include_router(router, prefix="/api/v1")
-
-
-@app.on_event("startup")
-async def startup():
-    logger.info("RAG Pipeline API starting up...")
-    logger.info(f"Docs: http://{settings.api_host}:{settings.api_port}/docs")
-
-
-@app.on_event("shutdown")
-async def shutdown():
-    logger.info("RAG Pipeline API shutting down.")

rag/src/api/routes.py

@@ -1,137 +0,0 @@
-from fastapi import APIRouter, HTTPException, UploadFile, File
-from fastapi.responses import StreamingResponse
-from loguru import logger
-import tempfile
-import shutil
-from pathlib import Path
-
-from .schemas import (
-    IngestRequest, IngestResponse,
-    QueryRequest, QueryResponse,
-    SummarizeRequest, SummarizeResponse,
-    StatsResponse, DeleteResponse,
-)
-from ..retrieval.retriever import RAGRetriever
-
-router = APIRouter()
-
-# Singleton retriever — di-init sekali saat startup
-_retriever: RAGRetriever = None
-
-
-def get_retriever() -> RAGRetriever:
-    global _retriever
-    if _retriever is None:
-        _retriever = RAGRetriever()
-    return _retriever
-
-
-# === HEALTH ===
-
-@router.get("/health", tags=["system"])
-async def health_check():
-    return {"status": "ok", "service": "RAG Pipeline API"}
-
-
-# === STATS ===
-
-@router.get("/stats", response_model=StatsResponse, tags=["system"])
-async def get_stats():
-    """Info tentang vector store saat ini."""
-    return get_retriever().get_stats()
-
-
-# === INGEST ===
-
-@router.post("/ingest", response_model=IngestResponse, tags=["indexing"])
-async def ingest_documents(request: IngestRequest):
-    """
-    Index dokumen dari file path atau URL ke vector store.
-    Mendukung: PDF, TXT, MD, DOCX, JSON, JSONL, URL
-    """
-    try:
-        stats = get_retriever().ingest(request.sources)
-        return IngestResponse(status="success", **stats)
-    except Exception as e:
-        logger.error(f"Ingest error: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@router.post("/ingest/upload", tags=["indexing"])
-async def ingest_upload(file: UploadFile = File(...)):
-    """Upload dan index file langsung via multipart."""
-    allowed_exts = {".pdf", ".txt", ".md", ".docx", ".json", ".jsonl"}
-    ext = Path(file.filename).suffix.lower()
-
-    if ext not in allowed_exts:
-        raise HTTPException(
-            status_code=400,
-            detail=f"Ekstensi '{ext}' tidak didukung. Gunakan: {allowed_exts}"
-        )
-
-    # Simpan file sementara
-    with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as tmp:
-        shutil.copyfileobj(file.file, tmp)
-        tmp_path = tmp.name
-
-    try:
-        stats = get_retriever().ingest([tmp_path])
-        return IngestResponse(status="success", **stats)
-    except Exception as e:
-        raise HTTPException(status_code=500, detail=str(e))
-    finally:
-        Path(tmp_path).unlink(missing_ok=True)
-
-
-# === QUERY ===
-
-@router.post("/query", response_model=QueryResponse, tags=["querying"])
-async def query(request: QueryRequest):
-    """
-    Tanya jawab berdasarkan dokumen yang sudah di-index.
-    Mendukung multi-turn conversation via chat_history.
-    """
-    if request.stream:
-        # Streaming response
-        def generate():
-            yield from get_retriever().stream_query(
-                question=request.question,
-                chat_history=[m.model_dump() for m in (request.chat_history or [])],
-            )
-        return StreamingResponse(generate(), media_type="text/event-stream")
-
-    try:
-        result = get_retriever().query(
-            question=request.question,
-            chat_history=[m.model_dump() for m in (request.chat_history or [])],
-            top_k=request.top_k,
-            return_sources=request.return_sources,
-        )
-        return QueryResponse(**result)
-    except Exception as e:
-        logger.error(f"Query error: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-# === SUMMARIZE ===
-
-@router.post("/summarize", response_model=SummarizeResponse, tags=["querying"])
-async def summarize(request: SummarizeRequest):
-    """Buat ringkasan otomatis dari dokumen."""
-    try:
-        summary = get_retriever().summarize(request.source)
-        return SummarizeResponse(summary=summary, source=request.source)
-    except Exception as e:
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-# === DELETE ===
-
-@router.delete("/collection", response_model=DeleteResponse, tags=["system"])
-async def delete_collection():
-    """Hapus semua dokumen dari vector store. HATI-HATI: tidak bisa di-undo."""
-    get_retriever().vector_store.delete_collection()
-    return DeleteResponse(
-        status="success",
-        message="Semua dokumen berhasil dihapus dari vector store."
-    )

rag/src/api/schemas.py

@@ -1,67 +0,0 @@
-from pydantic import BaseModel, Field
-from typing import List, Optional
-
-
-class IngestRequest(BaseModel):
-    sources: List[str] = Field(
-        ...,
-        description="List of file paths atau URLs untuk di-index",
-        example=["./docs/laporan.pdf", "https://example.com/artikel"],
-    )
-
-
-class IngestResponse(BaseModel):
-    status: str
-    documents_loaded: int
-    chunks_indexed: int
-    total_docs_in_store: int
-    elapsed_seconds: float
-    sources: List[str]
-
-
-class ChatMessage(BaseModel):
-    role: str = Field(..., pattern="^(user|assistant)$")
-    content: str
-
-
-class QueryRequest(BaseModel):
-    question: str = Field(..., min_length=1, max_length=2000)
-    chat_history: Optional[List[ChatMessage]] = Field(default=[], description="Riwayat chat untuk multi-turn")
-    top_k: Optional[int] = Field(default=None, ge=1, le=20)
-    return_sources: bool = Field(default=True)
-    stream: bool = Field(default=False, description="Gunakan streaming response")
-
-
-class SourceChunk(BaseModel):
-    content: str
-    metadata: dict
-    relevance_score: float
-
-
-class QueryResponse(BaseModel):
-    answer: str
-    question: str
-    latency_seconds: float
-    chunks_retrieved: int
-    sources: Optional[List[SourceChunk]] = None
-
-
-class SummarizeRequest(BaseModel):
-    source: str = Field(..., description="File path atau URL untuk diringkas")
-
-
-class SummarizeResponse(BaseModel):
-    summary: str
-    source: str
-
-
-class StatsResponse(BaseModel):
-    total_chunks: int
-    collection_name: str
-    embedding_model: str
-    llm_model: str
-
-
-class DeleteResponse(BaseModel):
-    status: str
-    message: str

rag/src/config.py

@@ -1,56 +0,0 @@
-from pydantic_settings import BaseSettings
-from pydantic import Field
-from functools import lru_cache
-from pathlib import Path
-
-
-class Settings(BaseSettings):
-    # LLM
-    groq_api_key: str = Field(..., env="GROQ_API_KEY")
-    groq_model: str = Field("llama-3.3-70b-versatile", env="GROQ_MODEL")  # updated from 3.1
-
-    # Embeddings
-    embedding_model: str = Field("all-MiniLM-L6-v2", env="EMBEDDING_MODEL")
-    embedding_device: str = Field("cpu", env="EMBEDDING_DEVICE")
-
-    # Vector Store
-    chroma_persist_dir: str = Field("./chroma_db", env="CHROMA_PERSIST_DIR")
-    chroma_collection_name: str = Field("rag_documents", env="CHROMA_COLLECTION_NAME")
-
-    # RAG Settings
-    chunk_size: int = Field(1000, env="CHUNK_SIZE")
-    chunk_overlap: int = Field(200, env="CHUNK_OVERLAP")
-    top_k_retrieval: int = Field(5, env="TOP_K_RETRIEVAL")
-    max_tokens: int = Field(2048, env="MAX_TOKENS")
-    temperature: float = Field(0.1, env="TEMPERATURE")
-
-    # API
-    api_host: str = Field("0.0.0.0", env="API_HOST")
-    api_port: int = Field(8000, env="API_PORT")
-    api_reload: bool = Field(True, env="API_RELOAD")
-
-    # MLflow
-    mlflow_tracking_uri: str = Field("./mlruns", env="MLFLOW_TRACKING_URI")
-    mlflow_experiment_name: str = Field("rag_pipeline", env="MLFLOW_EXPERIMENT_NAME")
-
-    # Logging
-    log_level: str = Field("INFO", env="LOG_LEVEL")
-    log_file: str = Field("./logs/app.log", env="LOG_FILE")
-
-    class Config:
-        env_file = ".env"
-        env_file_encoding = "utf-8"
-
-    def ensure_dirs(self):
-        """Buat direktori yang dibutuhkan jika belum ada."""
-        Path(self.chroma_persist_dir).mkdir(parents=True, exist_ok=True)
-        Path(self.log_file).parent.mkdir(parents=True, exist_ok=True)
-        Path(self.mlflow_tracking_uri).mkdir(parents=True, exist_ok=True)
-
-
-@lru_cache()
-def get_settings() -> Settings:
-    """Singleton settings — di-cache supaya tidak re-parse tiap request."""
-    settings = Settings()
-    settings.ensure_dirs()
-    return settings

rag/src/embeddings/__init__.py

@@ -1 +0,0 @@
-

rag/src/embeddings/embedder.py

@@ -1,60 +0,0 @@
-from typing import List
-from loguru import logger
-from langchain.text_splitter import RecursiveCharacterTextSplitter
-from langchain_community.embeddings import HuggingFaceEmbeddings
-
-from ..config import get_settings
-from ..loaders.base_loader import Document
-
-
-class DocumentEmbedder:
-    """
-    Bertanggung jawab untuk:
-    1. Chunking dokumen panjang jadi potongan yang bisa di-embed
-    2. Membuat embedding vektor pakai model lokal (no API cost!)
-    """
-
-    def __init__(self):
-        settings = get_settings()
-        logger.info(f"Loading embedding model: {settings.embedding_model}")
-
-        self.embeddings = HuggingFaceEmbeddings(
-            model_name=settings.embedding_model,
-            model_kwargs={"device": settings.embedding_device},
-            encode_kwargs={"normalize_embeddings": True},
-        )
-
-        self.splitter = RecursiveCharacterTextSplitter(
-            chunk_size=settings.chunk_size,
-            chunk_overlap=settings.chunk_overlap,
-            separators=["\n\n", "\n", ". ", " ", ""],
-        )
-
-        logger.info("Embedder ready.")
-
-    def chunk_documents(self, documents: List[Document]) -> List[Document]:
-        """
-        Split dokumen panjang jadi chunks.
-        Metadata dari dokumen asli diwarisi ke setiap chunk.
-        """
-        chunks = []
-        for doc in documents:
-            texts = self.splitter.split_text(doc.content)
-            for i, text in enumerate(texts):
-                chunk_metadata = {
-                    **doc.metadata,
-                    "chunk_index": i,
-                    "total_chunks": len(texts),
-                    "parent_doc_id": doc.doc_id,
-                }
-                chunks.append(Document(
-                    content=text,
-                    metadata=chunk_metadata,
-                ))
-
-        logger.info(f"Chunked {len(documents)} docs → {len(chunks)} chunks")
-        return chunks
-
-    def get_embeddings_model(self):
-        """Return LangChain-compatible embeddings object untuk ChromaDB."""
-        return self.embeddings

rag/src/llm/__init__.py

@@ -1 +0,0 @@
-

rag/src/llm/groq_client.py

@@ -1,62 +0,0 @@
-from typing import List, Optional, Iterator
-from loguru import logger
-from langchain_groq import ChatGroq
-from langchain.schema import BaseMessage, HumanMessage, AIMessage
-from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
-
-from ..config import get_settings
-
-
-class GroqClient:
-    """
-    Wrapper di atas ChatGroq dari LangChain.
-    Mendukung regular call dan streaming.
-    """
-
-    def __init__(self, streaming: bool = False):
-        settings = get_settings()
-        callbacks = [StreamingStdOutCallbackHandler()] if streaming else []
-
-        self.llm = ChatGroq(
-            api_key=settings.groq_api_key,
-            model_name=settings.groq_model,
-            temperature=settings.temperature,
-            max_tokens=settings.max_tokens,
-            streaming=streaming,
-            callbacks=callbacks,
-        )
-        self.model_name = settings.groq_model
-        logger.info(f"Groq client initialized. Model: {settings.groq_model}")
-
-    def invoke(self, messages: List[BaseMessage]) -> str:
-        """Kirim messages dan return response string."""
-        response = self.llm.invoke(messages)
-        return response.content
-
-    def stream(self, messages: List[BaseMessage]) -> Iterator[str]:
-        """Streaming response — yield token per token."""
-        for chunk in self.llm.stream(messages):
-            if chunk.content:
-                yield chunk.content
-
-    def get_langchain_llm(self):
-        """Return raw LangChain LLM object untuk dipakai di chain."""
-        return self.llm
-
-    @staticmethod
-    def build_messages(
-        question: str,
-        chat_history: Optional[List[dict]] = None,
-    ) -> List[BaseMessage]:
-        """
-        Convert chat history format ke LangChain messages.
-        chat_history format: [{"role": "user"/"assistant", "content": "..."}]
-        """
-        messages = []
-        for msg in (chat_history or []):
-            if msg["role"] == "user":
-                messages.append(HumanMessage(content=msg["content"]))
-            elif msg["role"] == "assistant":
-                messages.append(AIMessage(content=msg["content"]))
-        messages.append(HumanMessage(content=question))
-        return messages

rag/src/llm/prompt_templates.py

@@ -1,36 +0,0 @@
-from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
-
-# === RAG QA Prompt ===
-RAG_PROMPT = ChatPromptTemplate.from_messages([
-    ("system", """Kamu adalah AI assistant yang menjawab pertanyaan berdasarkan konteks dokumen yang diberikan.
-
-ATURAN:
-- Jawab HANYA berdasarkan konteks yang disediakan
-- Jika jawaban tidak ada di konteks, katakan "Informasi ini tidak tersedia dalam dokumen yang diberikan"
-- Selalu sebutkan sumber dokumen jika relevan
-- Jawab dalam bahasa yang sama dengan pertanyaan pengguna
-- Berikan jawaban yang ringkas, akurat, dan terstruktur
-
-KONTEKS DOKUMEN:
-{context}
-"""),
-    MessagesPlaceholder(variable_name="chat_history", optional=True),
-    ("human", "{question}"),
-])
-
-# === Standalone Question Prompt (untuk rephrase pertanyaan follow-up) ===
-CONDENSE_QUESTION_PROMPT = ChatPromptTemplate.from_messages([
-    ("system", """Diberikan riwayat percakapan dan pertanyaan terbaru dari pengguna, 
-reformulasikan pertanyaan menjadi pertanyaan mandiri yang bisa dipahami tanpa konteks percakapan sebelumnya.
-Jangan jawab pertanyaannya, cukup reformulasikan jika perlu. Jika tidak perlu, kembalikan apa adanya."""),
-    MessagesPlaceholder(variable_name="chat_history"),
-    ("human", "{question}"),
-])
-
-# === Summary Prompt ===
-SUMMARY_PROMPT = ChatPromptTemplate.from_messages([
-    ("system", """Buat ringkasan dari dokumen berikut. 
-Sertakan: poin-poin utama, informasi kunci, dan kesimpulan.
-Format: gunakan bullet points untuk keterbacaan yang baik."""),
-    ("human", "Dokumen:\n{document}\n\nBuat ringkasan:"),
-])

rag/src/loaders/__init__.py

@@ -1,69 +0,0 @@
-from typing import List
-from pathlib import Path
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-from .pdf_loader import PDFLoader
-from .text_loader import TextLoader
-from .docx_loader import DocxLoader
-from .web_loader import WebLoader
-from .json_loader import JSONLoader
-
-
-class LoaderFactory:
-    """
-    Auto-detect loader yang tepat berdasarkan ekstensi file atau URL.
-    Pattern: Factory Method — client tidak perlu tahu loader mana yang dipakai.
-    """
-
-    _loaders: dict[str, BaseLoader] = {
-        ".pdf": PDFLoader(),
-        ".txt": TextLoader(),
-        ".md": TextLoader(),
-        ".markdown": TextLoader(),
-        ".docx": DocxLoader(),
-        ".doc": DocxLoader(),
-        ".json": JSONLoader(),
-        ".jsonl": JSONLoader(),
-    }
-
-    @classmethod
-    def get_loader(cls, source: str) -> BaseLoader:
-        """Pilih loader yang sesuai untuk source."""
-        # URL
-        if source.startswith(("http://", "https://")):
-            return WebLoader()
-
-        # File
-        ext = Path(source).suffix.lower()
-        loader = cls._loaders.get(ext)
-        if loader is None:
-            raise ValueError(
-                f"Tidak ada loader untuk ekstensi '{ext}'. "
-                f"Didukung: {list(cls._loaders.keys())} + URL"
-            )
-        return loader
-
-    @classmethod
-    def load(cls, source: str) -> List[Document]:
-        """One-liner: auto-detect loader dan langsung load."""
-        loader = cls.get_loader(source)
-        logger.info(f"Using {loader.__class__.__name__} for: {source}")
-        return loader.load(source)
-
-    @classmethod
-    def load_many(cls, sources: List[str]) -> List[Document]:
-        """Load multiple sources sekaligus."""
-        all_docs = []
-        for source in sources:
-            try:
-                docs = cls.load(source)
-                all_docs.extend(docs)
-                logger.info(f"Loaded {len(docs)} docs from {source}")
-            except Exception as e:
-                logger.error(f"Gagal load {source}: {e}")
-        logger.info(f"Total loaded: {len(all_docs)} documents")
-        return all_docs
-
-
-__all__ = ["LoaderFactory", "Document", "BaseLoader"]

rag/src/loaders/base_loader.py

@@ -1,39 +0,0 @@
-from abc import ABC, abstractmethod
-from dataclasses import dataclass, field
-from typing import List, Optional
-from pathlib import Path
-
-
-@dataclass
-class Document:
-    """Representasi satu dokumen atau chunk yang sudah diload."""
-    content: str
-    metadata: dict = field(default_factory=dict)
-    doc_id: Optional[str] = None
-
-    def __post_init__(self):
-        if self.doc_id is None:
-            import hashlib
-            self.doc_id = hashlib.md5(self.content.encode()).hexdigest()[:12]
-
-
-class BaseLoader(ABC):
-    """Abstract base class untuk semua document loaders."""
-
-    @abstractmethod
-    def load(self, source: str) -> List[Document]:
-        """
-        Load dokumen dari source (path file atau URL).
-        Returns list of Document objects.
-        """
-        pass
-
-    def validate_source(self, source: str) -> bool:
-        """Validasi apakah source bisa di-handle loader ini."""
-        return True
-
-    @property
-    @abstractmethod
-    def supported_extensions(self) -> List[str]:
-        """Daftar ekstensi file yang didukung loader ini."""
-        pass

rag/src/loaders/docx_loader.py

@@ -1,52 +0,0 @@
-from typing import List
-from pathlib import Path
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-
-
-class DocxLoader(BaseLoader):
-    """Loader untuk file .docx menggunakan python-docx."""
-
-    @property
-    def supported_extensions(self) -> List[str]:
-        return [".docx", ".doc"]
-
-    def load(self, source: str) -> List[Document]:
-        try:
-            from docx import Document as DocxDocument
-        except ImportError:
-            raise ImportError("Install python-docx: pip install python-docx")
-
-        path = Path(source)
-        if not path.exists():
-            raise FileNotFoundError(f"File tidak ditemukan: {source}")
-
-        logger.info(f"Loading DOCX: {path.name}")
-        doc = DocxDocument(str(path))
-
-        # Ambil semua paragraf yang tidak kosong
-        paragraphs = [p.text.strip() for p in doc.paragraphs if p.text.strip()]
-        content = "\n\n".join(paragraphs)
-
-        # Ambil teks dari tabel juga
-        table_texts = []
-        for table in doc.tables:
-            for row in table.rows:
-                row_text = " | ".join(cell.text.strip() for cell in row.cells if cell.text.strip())
-                if row_text:
-                    table_texts.append(row_text)
-
-        if table_texts:
-            content += "\n\n[Tables]\n" + "\n".join(table_texts)
-
-        return [Document(
-            content=content,
-            metadata={
-                "source": str(path),
-                "filename": path.name,
-                "type": "docx",
-                "paragraphs": len(paragraphs),
-                "tables": len(doc.tables),
-            }
-        )]

rag/src/loaders/json_loader.py

@@ -1,103 +0,0 @@
-import json
-from typing import List
-from pathlib import Path
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-
-
-class JSONLoader(BaseLoader):
-    """
-    Loader untuk file JSON.
-    Bisa flatten nested JSON menjadi teks untuk di-embed.
-    """
-
-    def __init__(self, text_key: str = None, jq_schema: str = None):
-        """
-        text_key: key spesifik yang jadi konten utama (e.g. 'content', 'text')
-        jq_schema: opsional — filter JSON pakai jq-style path
-        """
-        self.text_key = text_key
-        self.jq_schema = jq_schema
-
-    @property
-    def supported_extensions(self) -> List[str]:
-        return [".json", ".jsonl"]
-
-    def load(self, source: str) -> List[Document]:
-        path = Path(source)
-        if not path.exists():
-            raise FileNotFoundError(f"File tidak ditemukan: {source}")
-
-        logger.info(f"Loading JSON: {path.name}")
-
-        # Handle JSONL (JSON Lines)
-        if path.suffix == ".jsonl":
-            return self._load_jsonl(path)
-
-        with open(path, "r", encoding="utf-8") as f:
-            data = json.load(f)
-
-        # Jika list of records
-        if isinstance(data, list):
-            documents = []
-            for i, record in enumerate(data):
-                content = self._extract_content(record)
-                documents.append(Document(
-                    content=content,
-                    metadata={
-                        "source": str(path),
-                        "filename": path.name,
-                        "type": "json",
-                        "record_index": i,
-                    }
-                ))
-            return documents
-
-        # Single object
-        content = self._extract_content(data)
-        return [Document(
-            content=content,
-            metadata={
-                "source": str(path),
-                "filename": path.name,
-                "type": "json",
-            }
-        )]
-
-    def _load_jsonl(self, path: Path) -> List[Document]:
-        documents = []
-        with open(path, "r", encoding="utf-8") as f:
-            for i, line in enumerate(f):
-                line = line.strip()
-                if not line:
-                    continue
-                record = json.loads(line)
-                content = self._extract_content(record)
-                documents.append(Document(
-                    content=content,
-                    metadata={
-                        "source": str(path),
-                        "filename": path.name,
-                        "type": "jsonl",
-                        "line": i + 1,
-                    }
-                ))
-        return documents
-
-    def _extract_content(self, data: dict) -> str:
-        """Konversi dict/list ke string yang bisa di-embed."""
-        if self.text_key and isinstance(data, dict) and self.text_key in data:
-            return str(data[self.text_key])
-
-        # Fallback: flatten semua key-value pair
-        if isinstance(data, dict):
-            parts = []
-            for k, v in data.items():
-                if isinstance(v, (str, int, float, bool)):
-                    parts.append(f"{k}: {v}")
-                elif isinstance(v, (list, dict)):
-                    parts.append(f"{k}: {json.dumps(v, ensure_ascii=False)}")
-            return "\n".join(parts)
-
-        return json.dumps(data, ensure_ascii=False, indent=2)

rag/src/loaders/pdf_loader.py

@@ -1,46 +0,0 @@
-from typing import List
-from pathlib import Path
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-
-
-class PDFLoader(BaseLoader):
-    """Loader untuk file PDF menggunakan pypdf."""
-
-    @property
-    def supported_extensions(self) -> List[str]:
-        return [".pdf"]
-
-    def load(self, source: str) -> List[Document]:
-        try:
-            from pypdf import PdfReader
-        except ImportError:
-            raise ImportError("Install pypdf: pip install pypdf")
-
-        path = Path(source)
-        if not path.exists():
-            raise FileNotFoundError(f"File tidak ditemukan: {source}")
-
-        logger.info(f"Loading PDF: {path.name}")
-        reader = PdfReader(str(path))
-        documents = []
-
-        for i, page in enumerate(reader.pages):
-            text = page.extract_text()
-            if not text or not text.strip():
-                continue
-
-            documents.append(Document(
-                content=text.strip(),
-                metadata={
-                    "source": str(path),
-                    "filename": path.name,
-                    "page": i + 1,
-                    "total_pages": len(reader.pages),
-                    "type": "pdf",
-                }
-            ))
-
-        logger.info(f"Loaded {len(documents)} pages from {path.name}")
-        return documents

rag/src/loaders/text_loader.py

@@ -1,31 +0,0 @@
-from typing import List
-from pathlib import Path
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-
-
-class TextLoader(BaseLoader):
-    """Loader untuk file .txt dan .md."""
-
-    @property
-    def supported_extensions(self) -> List[str]:
-        return [".txt", ".md", ".markdown"]
-
-    def load(self, source: str) -> List[Document]:
-        path = Path(source)
-        if not path.exists():
-            raise FileNotFoundError(f"File tidak ditemukan: {source}")
-
-        logger.info(f"Loading text file: {path.name}")
-        content = path.read_text(encoding="utf-8")
-
-        return [Document(
-            content=content,
-            metadata={
-                "source": str(path),
-                "filename": path.name,
-                "type": path.suffix.lstrip("."),
-                "size_chars": len(content),
-            }
-        )]

rag/src/loaders/web_loader.py

@@ -1,57 +0,0 @@
-from typing import List
-from loguru import logger
-
-from .base_loader import BaseLoader, Document
-
-
-class WebLoader(BaseLoader):
-    """Loader untuk URL — scrape konten teks dari halaman web."""
-
-    @property
-    def supported_extensions(self) -> List[str]:
-        return []  # Tidak berbasis ekstensi, berbasis URL
-
-    def validate_source(self, source: str) -> bool:
-        return source.startswith(("http://", "https://"))
-
-    def load(self, source: str) -> List[Document]:
-        try:
-            import requests
-            from bs4 import BeautifulSoup
-        except ImportError:
-            raise ImportError("Install: pip install requests beautifulsoup4")
-
-        logger.info(f"Fetching URL: {source}")
-        headers = {"User-Agent": "Mozilla/5.0 (compatible; RAG-Pipeline/1.0)"}
-
-        response = requests.get(source, headers=headers, timeout=15)
-        response.raise_for_status()
-
-        soup = BeautifulSoup(response.text, "html.parser")
-
-        # Hapus tag yang tidak relevan
-        for tag in soup(["script", "style", "nav", "footer", "header", "aside"]):
-            tag.decompose()
-
-        # Ambil judul
-        title = soup.find("title")
-        title_text = title.get_text(strip=True) if title else ""
-
-        # Ambil konten utama
-        main = soup.find("main") or soup.find("article") or soup.find("body")
-        content = main.get_text(separator="\n", strip=True) if main else soup.get_text(separator="\n", strip=True)
-
-        # Bersihkan baris kosong berulang
-        lines = [line for line in content.splitlines() if line.strip()]
-        content = "\n".join(lines)
-
-        return [Document(
-            content=content,
-            metadata={
-                "source": source,
-                "title": title_text,
-                "type": "web",
-                "status_code": response.status_code,
-                "content_length": len(content),
-            }
-        )]

rag/src/retrieval/__init__.py

@@ -1 +0,0 @@
-

rag/src/retrieval/retriever.py

@@ -1,211 +0,0 @@
-from typing import List, Optional, Iterator
-from loguru import logger
-import mlflow
-import time
-
-from langchain.schema import HumanMessage, AIMessage
-
-from ..config import get_settings
-from ..retrieval.vector_store import VectorStore
-from ..llm.groq_client import GroqClient
-from ..llm.prompt_templates import RAG_PROMPT, SUMMARY_PROMPT
-from ..loaders import LoaderFactory, Document
-
-
-class RAGRetriever:
-    """
-    Core class yang menyatukan semua komponen RAG:
-    Document Loading → Chunking → Embedding → Retrieval → Generation
-    """
-
-    def __init__(self):
-        self.settings = get_settings()
-        self.vector_store = VectorStore()
-        self.groq = GroqClient()
-        self._setup_mlflow()
-        logger.info("RAGRetriever initialized.")
-
-    def _setup_mlflow(self):
-        mlflow.set_tracking_uri(self.settings.mlflow_tracking_uri)
-        mlflow.set_experiment(self.settings.mlflow_experiment_name)
-
-    # === INDEXING ===
-
-    def ingest(self, sources: List[str]) -> dict:
-        """
-        Load, chunk, embed, dan index dokumen dari berbagai sources.
-        
-        Args:
-            sources: List of file paths atau URLs
-            
-        Returns:
-            dict berisi stats indexing
-        """
-        logger.info(f"Ingesting {len(sources)} sources...")
-        start = time.time()
-
-        with mlflow.start_run(run_name="ingest"):
-            mlflow.log_params({
-                "sources_count": len(sources),
-                "chunk_size": self.settings.chunk_size,
-                "chunk_overlap": self.settings.chunk_overlap,
-                "embedding_model": self.settings.embedding_model,
-            })
-
-            # Load semua dokumen
-            documents = LoaderFactory.load_many(sources)
-
-            # Index ke vector store
-            chunks_indexed = self.vector_store.add_documents(documents)
-
-            elapsed = time.time() - start
-            stats = {
-                "documents_loaded": len(documents),
-                "chunks_indexed": chunks_indexed,
-                "sources": sources,
-                "elapsed_seconds": round(elapsed, 2),
-                "total_docs_in_store": self.vector_store.count(),
-            }
-
-            mlflow.log_metrics({
-                "documents_loaded": len(documents),
-                "chunks_indexed": chunks_indexed,
-                "elapsed_seconds": elapsed,
-            })
-
-        logger.info(f"Ingestion selesai: {stats}")
-        return stats
-
-    # === QUERYING ===
-
-    def query(
-        self,
-        question: str,
-        chat_history: Optional[List[dict]] = None,
-        top_k: Optional[int] = None,
-        return_sources: bool = True,
-    ) -> dict:
-        """
-        Jawab pertanyaan menggunakan RAG.
-        
-        Args:
-            question: Pertanyaan user
-            chat_history: Riwayat chat [{"role": "user"/"assistant", "content": "..."}]
-            top_k: Jumlah chunks yang diretrieve
-            return_sources: Sertakan source chunks di response
-            
-        Returns:
-            dict dengan 'answer', 'sources', dan 'metadata'
-        """
-        start = time.time()
-        logger.info(f"Query: '{question[:80]}...'")
-
-        with mlflow.start_run(run_name="query"):
-            mlflow.log_param("question", question[:250])
-            mlflow.log_param("model", self.settings.groq_model)
-
-            # Retrieve relevant chunks
-            k = top_k or self.settings.top_k_retrieval
-
-            # Guard: jika store kosong, langsung jawab tanpa retrieval
-            store_count = self.vector_store.count()
-            if store_count == 0:
-                retrieved = []
-            else:
-                try:
-                    retrieved = self.vector_store.similarity_search_with_score(question, k=k)
-                except Exception as e:
-                    logger.warning(f"Retrieval failed (mungkin store kosong): {e}")
-                    retrieved = []
-
-            # Format context
-            context_parts = []
-            sources = []
-            for i, (doc, score) in enumerate(retrieved):
-                source_info = doc.metadata.get("filename", doc.metadata.get("source", "Unknown"))
-                page_info = f" (hal. {doc.metadata['page']})" if "page" in doc.metadata else ""
-                context_parts.append(
-                    f"[Sumber {i+1}: {source_info}{page_info} | Relevansi: {1-score:.2f}]\n{doc.page_content}"
-                )
-                sources.append({
-                    "content": doc.page_content[:300] + "..." if len(doc.page_content) > 300 else doc.page_content,
-                    "metadata": doc.metadata,
-                    "relevance_score": round(1 - score, 4),
-                })
-
-            context = "\n\n---\n\n".join(context_parts) if context_parts else "(Tidak ada dokumen yang di-index. Silakan upload dokumen terlebih dahulu.)"
-
-            # Build chat history with correct message types
-            history_messages = []
-            for m in (chat_history or []):
-                if m["role"] == "user":
-                    history_messages.append(HumanMessage(content=m["content"]))
-                elif m["role"] == "assistant":
-                    history_messages.append(AIMessage(content=m["content"]))
-
-            # Build prompt dan generate
-            formatted_prompt = RAG_PROMPT.format_messages(
-                context=context,
-                question=question,
-                chat_history=history_messages,
-            )
-
-            answer = self.groq.invoke(formatted_prompt)
-            elapsed = time.time() - start
-
-            mlflow.log_metrics({
-                "chunks_retrieved": len(retrieved),
-                "answer_length": len(answer),
-                "latency_seconds": elapsed,
-            })
-
-        result = {
-            "answer": answer,
-            "question": question,
-            "latency_seconds": round(elapsed, 2),
-            "chunks_retrieved": len(retrieved),
-        }
-        if return_sources:
-            result["sources"] = sources
-
-        return result
-
-    def stream_query(
-        self,
-        question: str,
-        chat_history: Optional[List[dict]] = None,
-    ) -> Iterator[str]:
-        """Streaming version dari query — yield token per token."""
-        retrieved = self.vector_store.similarity_search(question)
-        context = "\n\n---\n\n".join(
-            f"[Sumber: {doc.metadata.get('filename', 'Unknown')}]\n{doc.page_content}"
-            for doc in retrieved
-        )
-        formatted = RAG_PROMPT.format_messages(
-            context=context,
-            question=question,
-            chat_history=[],
-        )
-        groq_stream = GroqClient(streaming=True)
-        yield from groq_stream.stream(formatted)
-
-    def summarize(self, source: str) -> str:
-        """Buat ringkasan dari satu dokumen."""
-        documents = LoaderFactory.load(source)
-        full_text = "\n\n".join(doc.content for doc in documents)
-
-        # Truncate jika terlalu panjang
-        if len(full_text) > 12000:
-            full_text = full_text[:12000] + "\n...[dokumen dipotong untuk efisiensi]"
-
-        messages = SUMMARY_PROMPT.format_messages(document=full_text)
-        return self.groq.invoke(messages)
-
-    def get_stats(self) -> dict:
-        """Statistik vector store saat ini."""
-        return {
-            "total_chunks": self.vector_store.count(),
-            "collection_name": self.settings.chroma_collection_name,
-            "embedding_model": self.settings.embedding_model,
-            "llm_model": self.settings.groq_model,
-        }

rag/src/retrieval/vector_store.py

@@ -1,93 +0,0 @@
-from typing import List, Optional
-from loguru import logger
-from langchain_chroma import Chroma
-from langchain.schema import Document as LCDocument
-
-from ..config import get_settings
-from ..loaders.base_loader import Document
-from ..embeddings.embedder import DocumentEmbedder
-
-
-class VectorStore:
-    """
-    Wrapper di atas ChromaDB.
-    Menangani indexing, persistence, dan similarity search.
-    """
-
-    def __init__(self, embedder: Optional[DocumentEmbedder] = None):
-        settings = get_settings()
-        self.embedder = embedder or DocumentEmbedder()
-        self.settings = settings
-
-        self.db = Chroma(
-            collection_name=settings.chroma_collection_name,
-            embedding_function=self.embedder.get_embeddings_model(),
-            persist_directory=settings.chroma_persist_dir,
-        )
-        logger.info(
-            f"VectorStore ready. Collection: '{settings.chroma_collection_name}' "
-            f"| Docs: {self.db._collection.count()}"
-        )
-
-    def add_documents(self, documents: List[Document]) -> int:
-        """
-        Chunk dan index dokumen ke ChromaDB.
-        Returns: jumlah chunks yang berhasil di-index.
-        """
-        chunks = self.embedder.chunk_documents(documents)
-
-        # Konversi ke format LangChain
-        lc_docs = [
-            LCDocument(page_content=chunk.content, metadata=chunk.metadata)
-            for chunk in chunks
-        ]
-
-        self.db.add_documents(lc_docs)
-        logger.info(f"Indexed {len(chunks)} chunks ke ChromaDB.")
-        return len(chunks)
-
-    def similarity_search(
-        self,
-        query: str,
-        k: Optional[int] = None,
-        filter: Optional[dict] = None,
-    ) -> List[LCDocument]:
-        """Cari dokumen paling relevan berdasarkan query."""
-        k = k or self.settings.top_k_retrieval
-        results = self.db.similarity_search(query, k=k, filter=filter)
-        logger.debug(f"Retrieved {len(results)} chunks for query: '{query[:60]}...'")
-        return results
-
-    def similarity_search_with_score(
-        self,
-        query: str,
-        k: Optional[int] = None,
-    ) -> List[tuple]:
-        """Sama seperti similarity_search tapi return (doc, score)."""
-        k = k or self.settings.top_k_retrieval
-        return self.db.similarity_search_with_score(query, k=k)
-
-    def reset_collection(self):
-        """
-        Hapus semua dokumen TANPA mematikan collection.
-        Pakai reset_collection() dari langchain-chroma — collection tetap hidup
-        dan langsung siap untuk ingest berikutnya.
-        """
-        self.db.reset_collection()
-        logger.warning(
-            f"Collection '{self.settings.chroma_collection_name}' di-reset. "
-            "Semua dokumen dihapus, collection siap dipakai kembali."
-        )
-
-    def delete_collection(self):
-        """Alias ke reset_collection() — collection tidak dimatikan, aman."""
-        self.reset_collection()
-
-    def count(self) -> int:
-        """Jumlah chunks yang tersimpan."""
-        return self.db._collection.count()
-
-    def get_retriever(self, search_kwargs: Optional[dict] = None):
-        """Return LangChain retriever untuk dipakai di chain."""
-        search_kwargs = search_kwargs or {"k": self.settings.top_k_retrieval}
-        return self.db.as_retriever(search_kwargs=search_kwargs)

rag_pipeline/src/api/routes.py

@@ -71,6 +71,12 @@ async def readiness_combined():
             models={"rag": {"state": rag_state, "error": str(e)[:300]}},
         )
 
+    # Cek apakah GROQ_API_KEY ada — penting untuk endpoint /query dan /summarize.
+    # Tidak fail readiness, tapi UI dapat info untuk munculin warning.
+    from ..config import get_settings
+    settings = get_settings()
+    has_groq_key = bool((settings.groq_api_key or "").strip())
+
     # Cek status CV API (best-effort — kalau CV down, RAG tetep bisa pake
     # text-based loaders, cuma OCR fallback yang ga jalan).
     cv_url = os.getenv("CV_API_URL", "http://127.0.0.1:8001")
@@ -85,7 +91,9 @@ async def readiness_combined():
     except Exception as e:
         logger.debug(f"CV /ready unreachable: {e}")
 
-    all_models = {"rag": {"state": rag_state}}
+    all_models = {
+        "rag": {"state": rag_state, "groq_api_key": "set" if has_groq_key else "missing"},
+    }
     for name, info in cv_models.items():
         all_models[f"cv.{name}"] = info
 
@@ -220,6 +228,20 @@ async def query(request: QueryRequest):
             source_ids=request.source_ids,
         )
         return QueryResponse(**result)
+    except RuntimeError as e:
+        # GROQ_API_KEY missing → 503 Service Unavailable dengan pesan jelas.
+        msg = str(e)
+        if "GROQ_API_KEY" in msg:
+            logger.warning(f"Query rejected (no API key): {msg}")
+            raise HTTPException(
+                status_code=503,
+                detail={
+                    "error": "groq_api_key_missing",
+                    "message": msg,
+                },
+            )
+        logger.error(f"Query runtime error: {e}")
+        raise HTTPException(status_code=500, detail=str(e))
     except Exception as e:
         logger.error(f"Query error: {e}")
         raise HTTPException(status_code=500, detail=str(e))
@@ -233,6 +255,14 @@ async def summarize(request: SummarizeRequest):
     try:
         summary = get_retriever().summarize(request.source)
         return SummarizeResponse(summary=summary, source=request.source)
+    except RuntimeError as e:
+        msg = str(e)
+        if "GROQ_API_KEY" in msg:
+            raise HTTPException(
+                status_code=503,
+                detail={"error": "groq_api_key_missing", "message": msg},
+            )
+        raise HTTPException(status_code=500, detail=str(e))
     except Exception as e:
         raise HTTPException(status_code=500, detail=str(e))
 
@@ -242,8 +272,15 @@ async def summarize(request: SummarizeRequest):
 @router.delete("/collection", response_model=DeleteResponse, tags=["system"])
 async def delete_collection():
     """Hapus semua dokumen dari vector store."""
-    get_retriever().vector_store.delete_collection()
-    return DeleteResponse(
-        status="success",
-        message="Semua dokumen berhasil dihapus dari vector store.",
-    )
+    try:
+        get_retriever().vector_store.delete_collection()
+        return DeleteResponse(
+            status="success",
+            message="Semua dokumen berhasil dihapus dari vector store.",
+        )
+    except Exception as e:
+        logger.error(f"Clear collection error: {e}")
+        raise HTTPException(
+            status_code=500,
+            detail=f"Gagal hapus collection: {e}",
+        )

rag_pipeline/src/config.py

@@ -6,7 +6,11 @@ from pathlib import Path
 
 class Settings(BaseSettings):
     # LLM
-    groq_api_key: str = Field(..., env="GROQ_API_KEY")
+    # groq_api_key sengaja TIDAK required (default ""). Kalau env var ga di-set,
+    # service tetap bisa start dan endpoint /stats /sources /ingest masih jalan.
+    # Cuma /query dan /summarize yang gagal dengan pesan jelas — ini lebih bagus
+    # daripada seluruh container crash di startup.
+    groq_api_key: str = Field(default="", env="GROQ_API_KEY")
     groq_model: str = Field("llama-3.3-70b-versatile", env="GROQ_MODEL")  # updated from 3.1
 
     # Embeddings

rag_pipeline/src/llm/groq_client.py

@@ -1,6 +1,20 @@
+"""
+Groq LLM client.
+
+Perubahan: ChatGroq di-init LAZY — bukan saat constructor jalan, tapi saat
+pertama kali invoke/stream dipanggil. Alasannya:
+
+- RAGRetriever.__init__ sekarang ngga crash kalau GROQ_API_KEY ngga di-set.
+- Endpoint yang ngga butuh LLM (/stats, /sources, /ingest, /collection DELETE)
+  tetep jalan walau API key ngga ada.
+- /query dan /summarize gagal dengan pesan yang jelas: "GROQ_API_KEY belum
+  di-set" — bukan error pydantic / startup crash yang bikin user bingung.
+"""
+
+from __future__ import annotations
+
 from typing import List, Optional, Iterator
 from loguru import logger
-from langchain_groq import ChatGroq
 from langchain.schema import BaseMessage, HumanMessage, AIMessage
 from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
 
@@ -11,37 +25,70 @@ class GroqClient:
     """
     Wrapper di atas ChatGroq dari LangChain.
     Mendukung regular call dan streaming.
+
+    LAZY: ChatGroq instance baru dibuat saat pertama kali invoke/stream.
     """
 
     def __init__(self, streaming: bool = False):
-        settings = get_settings()
-        callbacks = [StreamingStdOutCallbackHandler()] if streaming else []
-
-        self.llm = ChatGroq(
-            api_key=settings.groq_api_key,
-            model_name=settings.groq_model,
-            temperature=settings.temperature,
-            max_tokens=settings.max_tokens,
-            streaming=streaming,
+        self.settings = get_settings()
+        self.streaming = streaming
+        self.model_name = self.settings.groq_model
+        self._llm = None  # akan diisi di _ensure_llm()
+        logger.info(
+            f"Groq client constructed (lazy). Model: {self.model_name} "
+            f"| API key: {'SET' if self.settings.groq_api_key else 'NOT SET'}"
+        )
+
+    def _ensure_llm(self):
+        """Buat ChatGroq instance kalau belum ada. Validasi API key di sini."""
+        if self._llm is not None:
+            return self._llm
+
+        api_key = (self.settings.groq_api_key or "").strip()
+        if not api_key:
+            raise RuntimeError(
+                "GROQ_API_KEY belum di-set. "
+                "Tambahkan di Hugging Face Space → Settings → Variables and secrets, "
+                "atau set environment variable di host. "
+                "Endpoint /query dan /summarize butuh API key ini."
+            )
+
+        try:
+            from langchain_groq import ChatGroq
+        except ImportError as e:
+            raise RuntimeError(
+                f"langchain-groq tidak terinstall: {e}. "
+                "Cek requirements.txt."
+            )
+
+        callbacks = [StreamingStdOutCallbackHandler()] if self.streaming else []
+        self._llm = ChatGroq(
+            api_key=api_key,
+            model_name=self.model_name,
+            temperature=self.settings.temperature,
+            max_tokens=self.settings.max_tokens,
+            streaming=self.streaming,
             callbacks=callbacks,
         )
-        self.model_name = settings.groq_model
-        logger.info(f"Groq client initialized. Model: {settings.groq_model}")
+        logger.info(f"Groq client initialized lazily. Model: {self.model_name}")
+        return self._llm
 
     def invoke(self, messages: List[BaseMessage]) -> str:
         """Kirim messages dan return response string."""
-        response = self.llm.invoke(messages)
+        llm = self._ensure_llm()
+        response = llm.invoke(messages)
         return response.content
 
     def stream(self, messages: List[BaseMessage]) -> Iterator[str]:
         """Streaming response — yield token per token."""
-        for chunk in self.llm.stream(messages):
+        llm = self._ensure_llm()
+        for chunk in llm.stream(messages):
             if chunk.content:
                 yield chunk.content
 
     def get_langchain_llm(self):
         """Return raw LangChain LLM object untuk dipakai di chain."""
-        return self.llm
+        return self._ensure_llm()
 
     @staticmethod
     def build_messages(

rag_pipeline/src/retrieval/vector_store.py

@@ -203,26 +203,69 @@ class VectorStore:
         return list(bucket.values())
 
     def reset_collection(self):
-        """Hapus semua dokumen tanpa mematikan collection."""
+        """
+        Hapus semua dokumen dari collection.
+
+        Strategi (paling kompatibel):
+        1. Coba `db.reset_collection()` (ada di langchain_chroma versi baru).
+        2. Fallback: ambil semua IDs lalu hapus via `_collection.delete(ids=...)`.
+           Ini bekerja di semua versi langchain_chroma + chromadb 0.5.x.
+        3. Last resort: nuke via `_client.delete_collection()` + re-init Chroma.
+
+        Kenapa option 2 jadi default fallback (bukan option 3):
+        - Re-init Chroma kadang triggers ChromaDB re-create error kalau
+          ada race condition dengan delete_collection di backend.
+        - Delete by IDs jauh lebih atomic.
+        """
+        # Option 1: high-level method (newer langchain_chroma)
         try:
             self.db.reset_collection()
+            logger.warning(
+                f"Collection '{self.settings.chroma_collection_name}' di-reset (high-level)."
+            )
+            return
+        except AttributeError:
+            logger.debug("db.reset_collection() tidak tersedia, fallback ke delete-by-ids.")
         except Exception as e:
-            # Fallback: kalau reset_collection tidak tersedia/gagal, pakai delete + recreate.
-            logger.warning(f"reset_collection() gagal: {e} — fallback ke delete + recreate.")
-            try:
-                self.db._client.delete_collection(self.settings.chroma_collection_name)
-            except Exception:
-                pass
-            # Re-init dengan re-attach embedding function.
+            logger.debug(f"db.reset_collection() gagal: {e} — fallback ke delete-by-ids.")
+
+        # Option 2: delete semua document by IDs
+        try:
+            collection = self.db._collection
+            data = collection.get(include=[])  # cuma butuh ids
+            ids = data.get("ids") or []
+            if ids:
+                collection.delete(ids=ids)
+                logger.warning(
+                    f"Collection '{self.settings.chroma_collection_name}' di-reset "
+                    f"({len(ids)} chunks dihapus via delete-by-ids)."
+                )
+            else:
+                logger.info(
+                    f"Collection '{self.settings.chroma_collection_name}' sudah kosong, ngga ada yg di-delete."
+                )
+            return
+        except Exception as e:
+            logger.warning(f"Delete-by-ids gagal: {e} — fallback ke delete + re-init.")
+
+        # Option 3: nuke collection lalu re-init Chroma
+        try:
+            self.db._client.delete_collection(self.settings.chroma_collection_name)
+        except Exception as e:
+            logger.warning(f"_client.delete_collection() gagal: {e}")
+
+        try:
             self.db = Chroma(
                 collection_name=self.settings.chroma_collection_name,
                 embedding_function=self.embedder.get_embeddings_model(),
                 persist_directory=self.settings.chroma_persist_dir,
             )
-        logger.warning(
-            f"Collection '{self.settings.chroma_collection_name}' di-reset. "
-            "Semua dokumen dihapus, collection siap dipakai kembali."
-        )
+            logger.warning(
+                f"Collection '{self.settings.chroma_collection_name}' nuked & re-init."
+            )
+        except Exception as e:
+            logger.error(f"Re-init Chroma gagal setelah delete: {e}")
+            raise
 
     def delete_collection(self):
         """Alias ke reset_collection() — collection tidak dimatikan, aman."""

start.sh

@@ -1,16 +1,41 @@
 #!/bin/bash
 set -e
 
-echo "=== Multimodal AI Platform — Starting ==="
-echo "GROQ_API_KEY: ${GROQ_API_KEY:+SET (hidden)}${GROQ_API_KEY:-NOT SET — RAG queries will fail!}"
+echo "========================================================"
+echo "  Multimodal AI Platform — Starting"
+echo "========================================================"
 
-# Export GROQ_API_KEY so child processes (supervisord → uvicorn) can access it
+# --- Diagnostics ---
+echo ""
+echo "[diag] Python: $(python3 --version 2>&1)"
+echo "[diag] Working dir: $(pwd)"
+echo ""
+echo "[diag] Source layout:"
+ls -la /app/rag/src/api/main.py 2>/dev/null && echo "  ✓ /app/rag/src/api/main.py" || echo "  ✗ /app/rag/src/api/main.py MISSING"
+ls -la /app/cv/src/api/main.py 2>/dev/null && echo "  ✓ /app/cv/src/api/main.py" || echo "  ✗ /app/cv/src/api/main.py MISSING"
+ls -la /app/frontend/index.html 2>/dev/null && echo "  ✓ /app/frontend/index.html" || echo "  ✗ /app/frontend/index.html MISSING"
+ls -la /app/cv/model_cache/yolov8n.onnx 2>/dev/null && echo "  ✓ YOLOv8n ONNX model" || echo "  ✗ YOLOv8n ONNX MISSING (CV /detect akan gagal)"
+echo ""
+
+# --- Secrets / env ---
+if [ -z "${GROQ_API_KEY}" ]; then
+  echo "[warn] GROQ_API_KEY tidak di-set."
+  echo "       /api/v1/query dan /api/v1/summarize akan return 503."
+  echo "       Endpoint lain (/ingest /sources /stats) tetap jalan normal."
+  echo "       Set di Hugging Face Space → Settings → Variables and secrets."
+else
+  echo "[ok]   GROQ_API_KEY: SET (${#GROQ_API_KEY} chars)"
+fi
+echo ""
+
+# Export GROQ_API_KEY supaya child processes (supervisord → uvicorn) bisa akses.
 export GROQ_API_KEY="${GROQ_API_KEY:-}"
 
-# Ensure directories exist
+# Pastikan direktori yang dibutuhkan ada (mount point HF Space sometimes resets).
 mkdir -p /app/rag/chroma_db /app/rag/mlruns /app/rag/logs \
          /app/cv/model_cache /app/cv/mlruns /app/cv/logs /app/cv/uploads \
          /var/log/supervisor /run
 
-echo "Starting supervisord (nginx + rag-api + cv-api)..."
+echo "[boot] Starting supervisord (nginx + rag-api + cv-api)..."
+echo "========================================================"
 exec /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf