Spaces:
Paused
Paused
Commit History
-
chore: clean up dead code, stale comments, and misleading names
Zhen Ye +1: Claude Opus 4.6 (1M context)- Rename gpt_data -> metadata on STrack (tracker.py) - Remove dead inject_metadata method and phantom METADATA_SYNC_KEYS - Fix _sync_data to sync depth_rel (the key actually written) - Remove dead _build_display_label and gpt_distance_m checks - Remove no-op add_no_cache_header middleware - Remove unused get_segmenter_detector import - Cache index.html at module load instead of reading per request - Extract _parse_queries helper to deduplicate query parsing - Remove empty set_track_data calls in segmentation writer loop - Remove sentence-transformers semantic matching from coco_classes - Clean stale GPT/mission references from docstrings - Update CLAUDE.md to reflect stripped-down architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>2e2a601
-
Eliminate redundant JPEG frame loading via shared frame store
Zhen Yec97a5f9
-
feat(ttfs): instrument segmentation time-to-first-stream
Zhen Ye301e154 -
refactor: rename hf_yolov8 → yolo11 across codebase
Zhen Ye +1: Claude Opus 4.6Rename file yolov8.py → yolov11.py, class HuggingFaceYoloV8Detector → Yolo11Detector, and registry key "hf_yolov8" → "yolo11" in all 11 files. Loads YOLO11m COCO-pretrained via hf://Ultralytics/YOLO11. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>f89fa0b -
feat(segmentation): add YSAM2 variants and mission-aware detector mapping
Zhen Ye6268ac2 -
chore: remove dead code and unused frontend modules
Zhen Yed74c718 -
refactor(gsam2): make SAM2 detector-agnostic
Zhen Ye21c29ae
-
fix: SAM2 _prep_prompts() signature
Zhen Ye74105c1 -
perf: defer GPU->CPU mask transfer to render thread via LazyFrameObjects
Zhen Ye +1: Claude Opus 4.6Replace bulk segment_output.to_object_dicts() CPU transfer with lazy materialization — only the last frame per segment is materialized for IoU tracking continuity, while render frames use LazyFrameObjects that defer GPU->CPU copy until the render thread needs them. Reduces CUDA syncs from 100+ per segment to ~3. Also removes deployment_logs.txt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a1ae44f -
fix: correct benchmark metrics — remove double-counting and add missing timers
Zhen Ye +1: Claude Opus 4.6- Fix sam_video_total_ms double-counting in single-GPU process_video() path (propagate_segment already accumulates, outer timer was adding it twice) - Fix id_reconciliation_ms in multi-GPU path to measure actual IoU work instead of wall-clock queue waits - Add model_load_ms and init_state_ms metrics for both single/multi-GPU paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>2f284f5
-
perf: pipeline GSAM2 tracking + rendering with startup buffer
Zhen Ye +1: Claude Opus 4.6Pipeline tracking and rendering so segments stream as they're tracked instead of waiting for all tracking to complete before rendering begins. - Add on_segment callback to process_video() for incremental feeding - Hoist render/writer infrastructure above tracking branch - Single-GPU: callback-based incremental feeding via on_segment - Multi-GPU: streaming reconciliation with segment_buffer pattern - Writer startup buffer (60 frames) before streaming begins - 3x frame duplication for 18 FPS stream (6 FPS processing throughput) - Safety threshold: pause streaming when buffer < 20 frames - try/finally sentinel safety for render worker shutdown - Increase render_out queue from 64 to 128 for buffer headroom Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>7124ca1 -
feat: add num_maskmem parameter to /benchmark endpoint
Zhen Ye +1: Claude Opus 4.6Thread num_maskmem override through the full pipeline so benchmark runs can test reduced memory bank sizes (e.g. 3 instead of default 7). Patches the SAM2 video predictor's maskmem_tpos_enc at runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1c6c619 -
revert: remove torch.compile — runtime failures on HF Space
Zhen Ye +1: Claude Opus 4.6Reverts all torch.compile changes (b17bd6d..10eb3c6). The HF Space container lacks Triton/inductor support, causing 500 errors. The GPU-resident tensor pipeline from 5aec47c is retained. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>5749bd6 -
fix: drop fullgraph=True from torch.compile for broader compatibility
Zhen Ye +1: Claude Opus 4.6fullgraph=True requires no graph breaks, which fails without SAM2VideoPredictorVOS's .clone() patches. Removing it allows torch.compile to handle graph breaks gracefully via partial compilation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>10eb3c6 -
fix: add runtime fallback for torch.compile inductor/triton failures
Zhen Ye +1: Claude Opus 4.6torch.compile wrapping succeeds immediately but actual compilation happens lazily on first forward pass. Now catches runtime errors on the first propagate_in_video yield, reverts all compiled forwards to eager originals, and retries the propagation transparently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>4c36e1e -
fix: manual torch.compile with graceful fallback for SAM2 components
Zhen Ye +1: Claude Opus 4.6Replace vos_optimized=True (caused 500 error) with manual per-component torch.compile matching Facebook's official recipe. Compiles image_encoder, memory_encoder, memory_attention (dynamic=True), sam_prompt_encoder, and sam_mask_decoder with max-autotune. Falls back to eager mode silently if PyTorch version or CUDA compiler is unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>63684e4 -
perf: enable torch.compile for SAM2 via vos_optimized flag
Zhen Ye +1: Claude Opus 4.6Uses Facebook's official SAM2VideoPredictorVOS which compiles all five model components (image_encoder, memory_encoder, memory_attention, sam_prompt_encoder, sam_mask_decoder) with torch.compile max-autotune. First inference has ~30s warmup cost; subsequent frames benefit from fused kernels and reduced memory round-trips. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>b17bd6d -
perf: GPU-resident tensor pipeline for SAM2 video propagation
Zhen Ye +1: Claude Opus 4.6Eliminate all CUDA synchronization from propagate_segment() by keeping masks, bboxes, and validity flags on GPU in pre-allocated buffers. CPU materialization is deferred to a single bulk transfer via SegmentOutput.to_object_dicts() at the consumer. - Add _bbox_gpu() for zero-sync bounding box computation on GPU - Add SegmentOutput dataclass for GPU-resident segment results - Rewrite propagate_segment() with inline bbox + pre-allocated tensors - Refactor process_video() to reuse propagate_segment() - Update Phase 4 reconciliation: 3 CUDA syncs per segment vs 100+ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>5aec47c
-
perf: eliminate CUDA sync points in SAM2 video propagation hot-path
Zhen Ye +1: Claude Opus 4.6Replace per-object-per-frame torch.nonzero + .item() calls (7 CUDA syncs each) with batched GPU-native argmax reductions (2 syncs per segment). Move deepcopy from per-frame to once per segment on last frame only. Reduces total CUDA sync points from ~28,700 to ~82 per pipeline run. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>64f68de -
feat: add detailed timing metrics to GroundedSAM2 segmenter
Zhen Ye74ae00c
-
Fix segmentation mission routing and first-frame GPT alignment
Zhen Yed73eff6
-
Fix cross-GPU mask device mismatch in GSAM2 reconciliation
Zhen Ye968c327 -
Harden GSAM2 parallel pipeline and tracking reconciliation
Zhen Yec90fe44 -
Refine Grounded-SAM2 tracking behavior and video frame handling
Zhen Ye032b60f -
Remove SAM3 and standardize segmentation on Grounded-SAM2
Zhen Ye3fde4e4
-
perf: Tune batch sizes and queues for A10 GPUs
Zhen Ye- Increased YOLO (HF/Drone) batch size to 32 - Increased Depth Anything batch size to 16 - Increased SAM3 batch size to 8 - Increased DETR batch size to 8 - Expanded inference.py queue_out to handle 32-frame batch burstsa5f8d15
-
Fix SAM3 batch prediction shape mismatch and add InternVL2 to frontend
Zhen Ye3de3df3 -
feat: Integrate InternVL2 and fix SAM3 segmentation batch size issue
Zhen Ye3d32b4a
-
Implement Batch Inference & Queue Backpressure Fixes
Zhen Ye5c36daa
-
feat(inference): enable full multi-GPU support for all models
Zhen Ye- Update inference.py to parallelize detection, segmentation, and depth estimation across all available GPUs - Update detectors (YOLOv8, DETR, GroundingDINO, DroneYolo) to accept device argument - Update SAM3 and DepthAnythingV2 to accept device argument - Add device-specific model loading to all model loaders - Remove OwlV2 support45eb65b
-
inital commit
Zhen Yeb30e7a3