Spaces:
Paused
Paused
Commit History
-
chore: clean up dead code, stale comments, and misleading names
Zhen Ye +1: Claude Opus 4.6 (1M context)- Rename gpt_data -> metadata on STrack (tracker.py) - Remove dead inject_metadata method and phantom METADATA_SYNC_KEYS - Fix _sync_data to sync depth_rel (the key actually written) - Remove dead _build_display_label and gpt_distance_m checks - Remove no-op add_no_cache_header middleware - Remove unused get_segmenter_detector import - Cache index.html at module load instead of reading per request - Extract _parse_queries helper to deduplicate query parsing - Remove empty set_track_data calls in segmentation writer loop - Remove sentence-transformers semantic matching from coco_classes - Clean stale GPT/mission references from docstrings - Update CLAUDE.md to reflect stripped-down architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>2e2a601
-
Eliminate redundant JPEG frame loading via shared frame store
Zhen Yec97a5f9
-
feat(ttfs): instrument segmentation time-to-first-stream
Zhen Ye301e154 -
fix: download yolo11m.pt via huggingface_hub to writable /tmp path
Zhen Ye +1: Claude Opus 4.6ultralytics YOLO() tries to download to CWD which is read-only in the container. Use hf_hub_download to fetch weights to /tmp/yolo_weights then load from that path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a4e3c2b -
fix: use YOLO("yolo11m.pt") with writable config dir
Zhen Ye +1: Claude Opus 4.6hf:// prefix not supported in installed ultralytics version. Use standard YOLO("yolo11m.pt") which auto-downloads from CDN. Set YOLO_CONFIG_DIR=/tmp/ultralytics so weights download to a writable path in read-only containers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>e745021 -
refactor: rename hf_yolov8 → yolo11 across codebase
Zhen Ye +1: Claude Opus 4.6Rename file yolov8.py → yolov11.py, class HuggingFaceYoloV8Detector → Yolo11Detector, and registry key "hf_yolov8" → "yolo11" in all 11 files. Loads YOLO11m COCO-pretrained via hf://Ultralytics/YOLO11. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>f89fa0b -
feat: replace custom YOLOv8m_defence with YOLO11m COCO-pretrained
Zhen Ye +1: Claude Opus 4.6The custom defence weights (spencercdz/YOLOv8m_defence) hallucinated on standard traffic scenes—detecting road markings as cars while missing actual vehicles. Switch to ultralytics YOLO11m with COCO pretrained weights for reliable general-purpose detection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>53922f5 -
feat(segmentation): add YSAM2 variants and mission-aware detector mapping
Zhen Ye6268ac2 -
chore: remove dead code and unused frontend modules
Zhen Yed74c718 -
refactor(gsam2): make SAM2 detector-agnostic
Zhen Ye21c29ae
-
fix: SAM2 _prep_prompts() signature
Zhen Ye74105c1 -
perf: defer GPU->CPU mask transfer to render thread via LazyFrameObjects
Zhen Ye +1: Claude Opus 4.6Replace bulk segment_output.to_object_dicts() CPU transfer with lazy materialization — only the last frame per segment is materialized for IoU tracking continuity, while render frames use LazyFrameObjects that defer GPU->CPU copy until the render thread needs them. Reduces CUDA syncs from 100+ per segment to ~3. Also removes deployment_logs.txt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a1ae44f -
fix: correct benchmark metrics — remove double-counting and add missing timers
Zhen Ye +1: Claude Opus 4.6- Fix sam_video_total_ms double-counting in single-GPU process_video() path (propagate_segment already accumulates, outer timer was adding it twice) - Fix id_reconciliation_ms in multi-GPU path to measure actual IoU work instead of wall-clock queue waits - Add model_load_ms and init_state_ms metrics for both single/multi-GPU paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>2f284f5
-
perf: pipeline GSAM2 tracking + rendering with startup buffer
Zhen Ye +1: Claude Opus 4.6Pipeline tracking and rendering so segments stream as they're tracked instead of waiting for all tracking to complete before rendering begins. - Add on_segment callback to process_video() for incremental feeding - Hoist render/writer infrastructure above tracking branch - Single-GPU: callback-based incremental feeding via on_segment - Multi-GPU: streaming reconciliation with segment_buffer pattern - Writer startup buffer (60 frames) before streaming begins - 3x frame duplication for 18 FPS stream (6 FPS processing throughput) - Safety threshold: pause streaming when buffer < 20 frames - try/finally sentinel safety for render worker shutdown - Increase render_out queue from 64 to 128 for buffer headroom Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>7124ca1 -
feat: add num_maskmem parameter to /benchmark endpoint
Zhen Ye +1: Claude Opus 4.6Thread num_maskmem override through the full pipeline so benchmark runs can test reduced memory bank sizes (e.g. 3 instead of default 7). Patches the SAM2 video predictor's maskmem_tpos_enc at runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1c6c619 -
revert: remove torch.compile — runtime failures on HF Space
Zhen Ye +1: Claude Opus 4.6Reverts all torch.compile changes (b17bd6d..10eb3c6). The HF Space container lacks Triton/inductor support, causing 500 errors. The GPU-resident tensor pipeline from 5aec47c is retained. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>5749bd6 -
fix: drop fullgraph=True from torch.compile for broader compatibility
Zhen Ye +1: Claude Opus 4.6fullgraph=True requires no graph breaks, which fails without SAM2VideoPredictorVOS's .clone() patches. Removing it allows torch.compile to handle graph breaks gracefully via partial compilation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>10eb3c6 -
fix: add runtime fallback for torch.compile inductor/triton failures
Zhen Ye +1: Claude Opus 4.6torch.compile wrapping succeeds immediately but actual compilation happens lazily on first forward pass. Now catches runtime errors on the first propagate_in_video yield, reverts all compiled forwards to eager originals, and retries the propagation transparently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>4c36e1e -
fix: manual torch.compile with graceful fallback for SAM2 components
Zhen Ye +1: Claude Opus 4.6Replace vos_optimized=True (caused 500 error) with manual per-component torch.compile matching Facebook's official recipe. Compiles image_encoder, memory_encoder, memory_attention (dynamic=True), sam_prompt_encoder, and sam_mask_decoder with max-autotune. Falls back to eager mode silently if PyTorch version or CUDA compiler is unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>63684e4 -
perf: enable torch.compile for SAM2 via vos_optimized flag
Zhen Ye +1: Claude Opus 4.6Uses Facebook's official SAM2VideoPredictorVOS which compiles all five model components (image_encoder, memory_encoder, memory_attention, sam_prompt_encoder, sam_mask_decoder) with torch.compile max-autotune. First inference has ~30s warmup cost; subsequent frames benefit from fused kernels and reduced memory round-trips. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>b17bd6d -
perf: GPU-resident tensor pipeline for SAM2 video propagation
Zhen Ye +1: Claude Opus 4.6Eliminate all CUDA synchronization from propagate_segment() by keeping masks, bboxes, and validity flags on GPU in pre-allocated buffers. CPU materialization is deferred to a single bulk transfer via SegmentOutput.to_object_dicts() at the consumer. - Add _bbox_gpu() for zero-sync bounding box computation on GPU - Add SegmentOutput dataclass for GPU-resident segment results - Rewrite propagate_segment() with inline bbox + pre-allocated tensors - Refactor process_video() to reuse propagate_segment() - Update Phase 4 reconciliation: 3 CUDA syncs per segment vs 100+ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>5aec47c
-
perf: eliminate CUDA sync points in SAM2 video propagation hot-path
Zhen Ye +1: Claude Opus 4.6Replace per-object-per-frame torch.nonzero + .item() calls (7 CUDA syncs each) with batched GPU-native argmax reductions (2 syncs per segment). Move deepcopy from per-frame to once per segment on last frame only. Reduces total CUDA sync points from ~28,700 to ~82 per pipeline run. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>64f68de -
feat: add detailed timing metrics to GroundedSAM2 segmenter
Zhen Ye74ae00c
-
Fix segmentation mission routing and first-frame GPT alignment
Zhen Yed73eff6
-
Fix cross-GPU mask device mismatch in GSAM2 reconciliation
Zhen Ye968c327 -
Harden GSAM2 parallel pipeline and tracking reconciliation
Zhen Yec90fe44 -
Refine Grounded-SAM2 tracking behavior and video frame handling
Zhen Ye032b60f -
Remove SAM3 and standardize segmentation on Grounded-SAM2
Zhen Ye3fde4e4 -
Handle set-typed missing_keys in Grounding DINO loader
Zhen Ye04d4562 -
Fix Grounding DINO load reliability and writer error handling
Zhen Yea7f0600
-
fix: replaced batch with sequential
Zhen Yeaf0f84f -
Fix: Grounding DINO batch inference
Zhen Ye8bc4370 -
Add diagnostic logging to pinpoint tracker dropping detections
Zhen Ye +1: Claude Opus 4.64 targeted log points across inference.py and utils/tracker.py to diagnose why first-frame preview shows bounding boxes but the processed video has none — the tracker is the prime suspect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a70bcf5
-
Fix Grounding DINO detections dropped by ByteTracker threshold mismatch
Zhen Ye +1: Claude Opus 4.6Lower ByteTracker new_track_thresh from 0.40 to 0.30 so DINO scores (≥0.35) can spawn tracks. Harden DINO _post_process to remove bare fallback that silently dropped to HuggingFace default threshold of 0.25. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>0d3be57
-
Fix GroundingDino loading: manually materialize meta tensors to support tied weights
Zhen Yefe2ace8 -
Refine GPT enrichment logic, add relevance badges to UI, and fix minor model/logging issues
Zhen Ye2c4431d
-
feat: Radar historic traces and fix DETR crash
Zhen Yefe670b9
-
Use DETR no_timm revision to avoid timm dependency
Zhen Yed52f7cc -
Fix BGR to RGB conversion for DETR and GroundingDino inference
Zhen Yef78d96f -
fix: Resolve IndentationError in DroneYoloDetector due to displaced import
Zhen Ye4e93b33 -
feat: Implement SAHI Tiling for 4K video detection
Zhen Ye- Added utils/tiling.py for image slicing and NMS - Updated DroneYolo and HFYoloV8 to auto-tile images > 3000px width - Uses 1280x1280 tiles with 20% overlap for maximum small object recallaf29397 -
optimize: Force high-res (1280px) inference for YOLO models to improve small object detection
Zhen Ye284ce20 -
perf: Tune batch sizes and queues for A10 GPUs
Zhen Ye- Increased YOLO (HF/Drone) batch size to 32 - Increased Depth Anything batch size to 16 - Increased SAM3 batch size to 8 - Increased DETR batch size to 8 - Expanded inference.py queue_out to handle 32-frame batch burstsa5f8d15
-
Simplify drone YOLO loading with ultralytics native HF Hub support
Zhen Ye2392266 -
Update Lite model to use spencercdz/YOLOv8m_defence
Zhen Ye38a60a8 -
Remove InternVL2 logic as requested
Zhen Ye29e1c2b -
Fix SAM3 batch prediction shape mismatch and add InternVL2 to frontend
Zhen Ye3de3df3 -
feat: Integrate InternVL2 and fix SAM3 segmentation batch size issue
Zhen Ye3d32b4a
-
Implement Batch Inference & Queue Backpressure Fixes
Zhen Ye5c36daa
-
Fix YOLO models to support specific GPU assignment
Zhen Yeb5e6df2