Spaces:
Paused
Paused
Commit History
-
Add enforce_eager=True to fix KV cache memory issue
torch.compile was consuming ~3-4GB overhead, causing negative KV cache memory (-1.13 GiB). enforce_eager skips compilation, reducing memory overhead and enabling successful model startup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1b7fbd7 -
Reduce vLLM memory for A100 24GB compatibility
- gpu_memory_utilization: 0.80 → 0.55 (leaves ~10GB for embedding+reranker) - max_model_len: 16384 → 8192 (reduces KV cache memory) Fixes OOM error when loading all 3 models on 22.3GB usable VRAM. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>b85b1e0 -
Fix SessionState field name in samples.py
Replace deprecated tab1/2/3_complete fields with input_complete to match simplified SessionState model. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>6fc2368 -
Fix Gradio 6.x Chatbot API: remove deprecated 'type' parameter
The 'type' parameter was removed in Gradio 6.x - messages format is now default. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>fbf7e70 -
Switch to Qwen3-VL-4B-Thinking for single-GPU simplicity
Architecture change: - Vision: 30B MoE (4 GPU TP) → 4B Dense (single GPU) - tensor_parallel_size: 4 → 1 - gpu_memory_utilization: 0.50 → 0.80 - Removed NCCL workarounds (not needed for single GPU) Why: 30B MoE model failed on 4xL4 due to vLLM V1 + NCCL issues (L4s lack NVLINK). 4B dense model fits on single L4 (22GB), eliminates all multi-GPU coordination problems. Memory: ~18GB total (10GB vision + 4GB embed + 4GB rerank) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>14c59e5 -
Reduce context/memory to minimize NCCL overhead on L4s
Changes: - max_model_len: 16384 → 8192 (half context length) - gpu_memory_utilization: 0.70 → 0.50 (less memory pressure) L4 GPUs lack NVLINK, making tensor parallelism communication slow via PCIe. Reducing these values minimizes NCCL overhead. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>7d5c713 -
Align vLLM config with official Qwen3-VL model card
Changes: - Remove VLLM_USE_V1=0 (V0 is removed in vLLM 0.11+, must use V1) - Remove dtype="float16" (FP8 model should auto-detect) - Change gpu_memory_utilization 0.90→0.70 (official recommendation) - Remove enforce_eager=True (let vLLM default per official) Root cause: V0 engine doesn't exist in vLLM >=0.11.0. Qwen3-VL requires vLLM >=0.11.0. Must make V1 work with official config. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>b2fe3f4 -
Pin vLLM <0.13.0 to fix V1 engine hang on multi-GPU
Root cause: vLLM 0.13.x deprecated V0 engine, VLLM_USE_V1=0 not honored. Logs showed V1 engine initializing despite env var, causing 4-GPU sync hang. Test: Deploy and verify logs show "V0 LLM engine" instead of "V1". Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>a65b765 -
Fix vLLM multi-GPU init: explicit dtype + higher mem util + eager mode
Based on successful RTX 4090D config from HF discussions: - Force env vars with os.environ[] (not setdefault) - Add dtype="float16" explicitly - Increase gpu_memory_utilization to 0.90 - Add enforce_eager=True for multi-GPU stability Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>71a896b -
Force vLLM V0 engine + reduce max_model_len for stability
- VLLM_USE_V1=0: Force stable V0 engine instead of V1 - Reduce max_model_len from 32768 to 16384 for memory safety - Keep NCCL and spawn settings for multi-GPU reliability Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3c9a722 -
Fix vLLM Engine core initialization failed on multi-GPU
Add environment variables before vLLM import to fix tensor parallelism issues: - VLLM_WORKER_MULTIPROC_METHOD=spawn (fixes process spawn issues) - NCCL_P2P_DISABLE=1, NCCL_IB_DISABLE=1 (multi-GPU reliability) References: - https://github.com/vllm-project/vllm/issues/17618 - https://github.com/vllm-project/vllm/issues/19002 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>ed575b1 -
Frontend simplification (4→2 tabs) + lazy imports for HF Spaces
Major changes: - Consolidate 4 tabs into 2: Input (room+images+observations) and Results+Chat - Add chat interface for Q&A and document modifications (pipeline/chat.py) - Lazy imports throughout to defer chromadb until needed - Auto-build RAG index on startup if empty - Default mock_models=False for production Files added: - ui/tabs/input_tab.py: Combined input tab with accordions - ui/tabs/results_tab.py: Results display with chat interface - pipeline/chat.py: Chat handler using vision model in text-only mode Import chain fixes (HF Spaces compatibility): - pipeline/__init__.py: __getattr__ lazy imports - rag/__init__.py: __getattr__ lazy imports - pipeline/main.py, dispositions.py, generator.py: TYPE_CHECKING + deferred imports - rag/retriever.py: Lazy vectorstore import in __init__ FDAM methodology fixes: - Public/Childcare lead threshold: 4.3 → 0.54 µg/100cm² (FDAM §1.4) - Sample density tiers for large areas (FDAM §2.3) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>78caafb -
Replace dual 8B with single 30B-A3B FP8 vision model
Simplify pipeline architecture: - Vision: Qwen3-VL-30B-A3B-Thinking-FP8 (~30-35GB) replaces dual 8B - Embedding: 2B model (2048-dim) replaces 8B - Reranker: 2B model replaces 8B - Total VRAM: ~38-43GB (was ~68GB), 45GB+ headroom on 4xL4 Key changes: - vLLM with FP8 quantization (built-in, no autoawq needed) - Proper Qwen3-VL chat template formatting via processor - Removed dual-model Thinking→Instruct pipeline - Single model handles analysis + structured JSON output Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>706520f -
Reduce thinking model max_new_tokens to fix slow inference
Root cause: max_new_tokens=32768 caused ~27 min inference time, appearing as a "hang". Reduced to 8192 for ~7 min thinking stage. The device mismatch warning is expected behavior - transformers handles routing internally for device_map="auto" models. Also includes ruff auto-fixes for unused imports. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>0699c5f -
Replace 30B MoE with dual 8B models (Thinking + Instruct)
Architecture change: - Vision: Qwen3-VL-30B-A3B-Instruct → dual Qwen3-VL-8B-Thinking + 8B-Instruct - Two-stage pipeline: Thinking (deep analysis) → Instruct (JSON formatting) - VRAM: 90GB → 68GB (~22GB savings, 20GB headroom on 4xL4) Key changes: - models/real.py: New DualVisionModel with token-based </think> parsing - config/settings.py: Dual model paths (vision_model_thinking, vision_model_instruct) - config/inference.py: ThinkingInferenceConfig (temp=0.6, max_tokens=32768) - Removed all lazy loading code (load_vision/unload_vision/load_rag) - All 4 models now load simultaneously at startup Per Qwen3-VL GitHub recommended hyperparameters for thinking models. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>333c083
-
MVP UI simplification: single room, 4 tabs
- Remove Project Info tab and ProjectFormData - Change from multi-room to single room (session.room) - Move facility_classification and construction_era to RoomFormData - Simplify from 5 tabs to 4: Room, Images, Observations, Results - Delete all test files (testing on HuggingFace due to GPU/ChromaDB) - Update pipeline files to use session.room - Update samples and documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>3b08f11 -
Implement lazy model loading to prevent CUDA OOM on 4xL4 GPUs
Problem: All 3 models (~92GB) loaded at startup exceeded 88GB VRAM. Solution: Sequential loading - vision model during Stage 2, RAG models during Stage 3+. Vision is unloaded before RAG loads. Peak: ~60GB. Changes: - models/real.py: Add load_vision(), unload_vision(), load_rag() with proper hook removal per HuggingFace accelerate docs - models/loader.py: Real models now use lazy loading (no load_all) - pipeline/main.py: Load/unload at appropriate pipeline stages - rag/vectorstore.py: Use SharedEmbeddingFunction (no duplicate load) - rag/retriever.py: Use SharedReranker (no duplicate load) - models/mock.py: Add is_vision_loaded(), is_rag_loaded() for API parity Memory profile: - Phase A (Vision): 30B model ~60GB - Transition: Unload + gc + empty_cache - Phase B (RAG): 8B + 8B ~32GB - Peak never exceeds 60GB (fits in 88GB) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>5f0db1e -
Add property accessors to RealModelStack for interface parity
RealModelStack stored models in self.models dict but lacked property accessors that pipeline code expected. MockModelStack exposes models as direct attributes (self.vision, etc.) but RealModelStack did not. This fix adds @property accessors for vision, embedding, and reranker that wrap the raw models in their respective wrapper classes, making RealModelStack API-compatible with MockModelStack. Fixes: 'RealModelStack' object has no attribute 'vision' error Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>c190082 -
Fix multi-GPU compatibility issues (6 locations)
1. Meta tensor error in reranker (BLOCKING): - Use .clone() to force weight materialization before copying - Create Linear layer with correct device/dtype from lm_head 2. Remove all .to(device) calls that break distributed models: - scripts/qwen3_vl/qwen3_vl_embedding.py:388 - scripts/qwen3_vl/qwen3_vl_reranker.py:109, 374 - models/real.py:259 - rag/vectorstore.py:130 - rag/retriever.py:149 With device_map="auto", transformers handles device routing internally. Manual .to(device) calls cause device mismatches on multi-GPU setups. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>d1901ae -
Fix multi-GPU support in vendored Qwen3-VL scripts
The original scripts used .to(device) which moves the entire model to a single GPU. With vision model already on GPU 0 (~22GB), this caused OOM when loading embedding model. Changes: - Remove .to(device) calls in both Qwen3VLEmbedder and Qwen3VLReranker - Add device_map="auto" to from_pretrained() for multi-GPU distribution - Update device references to use model's distributed device This allows the embedding and reranker models to be distributed across available GPUs alongside the vision model. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>c4bfdfa -
Fix embedding/reranker loading with official Qwen3-VL classes
Root cause: AutoModel.from_pretrained() loads base transformer instead of specialized embedding/reranking variants. Changes: - Vendor official scripts from QwenLM/Qwen3-VL-Embedding repo - Replace AutoModel with Qwen3VLEmbedder for embedding model - Replace AutoModel with Qwen3VLReranker for reranker model - Update embed()/rerank() methods to use official process() API The official loaders handle: - Proper last-token pooling and L2 normalization (embedding) - Yes/no binary scoring from LM head weights (reranker) This eliminates the fallback L2 norm heuristic scoring that was producing "less accurate" results. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>455c786 -
Fix critical model implementations and add sample scenarios
Model Fixes (verified against official Qwen3-VL-Embedding repo): - Embedding: Use last-token pooling instead of mean pooling - Embedding: Fix dimension from 768/384 to 4096 - Reranker: Use yes/no LM head weights + sigmoid instead of CLS norm - Apply fixes in both models/real.py and rag/vectorstore.py Additional Changes: - Add sample scenarios with real fire damage images - Add E2E tests with Playwright - Fix Gradio 6.x tab navigation tests - Add logging configuration - Improve UI form handling All 179 unit tests pass. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>f3ebc82 -
Trigger rebuild
8771f89 -
Initial commit: FDAM AI Pipeline v4.0.1
- Gradio-based fire damage assessment application - Qwen3-VL vision model integration (mock + real) - RAG-based knowledge retrieval with ChromaDB - FDAM-compliant calculations (ACH, sample density) - PDF generation with WeasyPrint - Session persistence via localStorage - 151 passing tests Ready for HuggingFace Spaces deployment on 4xL4 (96GB VRAM) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>88bdcff