Spaces:
Runtime error
Runtime error
Commit History
-
refactor: clean up backend utilities and background job handler
- Simplify background.py exception handling (merge two except blocks) - Remove unused imports and dead code in models.py, isr/utils.py - Use get_pil_rgb() in frame_store adapter instead of manual BGR conversion - Remove redundant dd benchmark variables in hardware_info.py - Remove unused decode timing vars in profiler.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>c1e17cf -
refactor(tracker): clean up ByteTracker implementation
- Extract _xyah_from_xyxy to module-level function (was duplicated in KalmanFilter and STrack) - Remove unused _sync_data method - Simplify linear_assignment unmatched index computation - Use sets for duplicate track removal - Remove dead code and stale comments Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>fc033d4 -
refactor: anonymize LLM labels, merge fragmented tracks, and tune defaults
Replace GPT-4o/Claude/Gemini branding with generic Analyst 1/2/3 labels in both frontend and backend. Add track merging to deduplicate fragmented IDs for the same physical object. Filter non-mission-relevant objects from timeline events. Bump frame store budget to 24 GiB, increase default segmentation step to 12, auto-detect API base URL, and fix video playback reliability (blob MIME type, canplay wait, timeout increase). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>fec2ee8
-
fix: clean up video display — remove track ID labels, smaller fonts, masks-only for segmentation
Zhen Ye +1: Claude Opus 4.6 (1M context)- Remove "T#" prefix from track IDs everywhere (backend + frontend) - Remove SVG overlay label tags (redundant with backend-rendered labels) - Track cards show class label only instead of "T24 · truck" - Segmentation mode (GSAM/YSAM) renders masks with labels only, no bboxes - Reduce label font scale from 1.0 to 0.6 for cleaner video output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>672057c
-
refactor: remove first-frame tab and simplify to video-only UI
Zhen Ye +1: Claude Opus 4.6Strip Tab 1 (first-frame detection preview), GPT toggle, chat panel, and all supporting backend endpoints (/detect/first-frame, depth). Delete process_first_frame pipeline, unused utils (enrichment, gpt_reasoning, mission_parser, relevance, schemas). ~2800 lines removed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>f8976b4
-
refactor: remove remaining dead HEL/threat code remnants
Zhen Ye +1: Claude Opus 4.6Strip leftover HEL fields (aim, baseRange, pkill, dwellAccum, etc.) from main.js, tracker.js, physics.js, state.js, video.js. Remove dead UI elements (btnRecompute, chipPolicy, chipBeam, chipHz, dwell bar) from index.html. Update schema/parser ASSESS_THREAT → EVALUATE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>be9d440 -
feat: add per-object satisfaction reasoning pipeline
Zhen Ye +1: Claude Opus 4.6Replace hardcoded threat assessment with generic per-object satisfaction reasoning. GPT now answers a yes/no question derived from the mission query for each detected object. - Add satisfaction_condition field to MissionSpecification - Update mission parser to extract satisfaction_condition from NL queries (e.g. "red cars" → detect cars, ask "Is this car red?" per object) - Rewrite gpt_reasoning.py: evaluate_satisfaction_gpt() draws highlighted bboxes on frame copy for GPT, returns {satisfies, reason} per object - Fast path: when satisfaction_condition is null, skip GPT entirely - Update enrichment.py to wire new function and map satisfies→mission_relevant Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>6e7b2b3 -
refactor: strip threat fields from all frontend and backend code
Zhen Ye +1: Claude Opus 4.6Remove all references to threat_level_score, threat_classification, weapon_readiness, gpt_distance_m, gpt_direction, gpt_description, HEL imports, syncKnobDisplays, recomputeHEL, /chat/threat endpoint, and chatAboutThreats client function across 14 files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>50d50b0 -
refactor: delete dead threat code (threat_chat.py, hel.js)
Zhen Ye +1: Claude Opus 4.6Remove 100% threat-specific modules with no reusable logic: - utils/threat_chat.py: military Q&A endpoint - frontend/js/core/hel.js: High-Energy Laser weapons simulator Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>21d90f0
-
feat: replace drone_yolo with YOLOv8-VisDrone detector
Zhen Ye +1: Claude Opus 4.6Remove broken drone_yolo detector (rujutashashikanjoshi repo not found) and replace with Mahadih534/YoloV8-VisDrone. Use hf_hub_download for reliable weight fetching. Allow drone_detection mode to use user-selected detector instead of hardcoding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>dda4696
-
feat: add GPT vision-based mission relevance assessment
Zhen Ye +1: Claude Opus 4.6Piggyback mission_relevant + relevance_reason onto existing GPT-4o threat assessment call (zero additional API cost). GPT can now demote false positives the deterministic gate missed. Preserve GPT decisions in the detection writer loop so subsequent frames don't overwrite them. Also remove unused helicopter demo data files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>c5f1a37
-
Eliminate redundant JPEG frame loading via shared frame store
Zhen Yef013501
-
chore(yolo11): update yolo naming comments
Zhen Ye6cf41fb -
refactor: rename hf_yolov8 → yolo11 across codebase
Zhen Ye +1: Claude Opus 4.6Rename file yolov8.py → yolov11.py, class HuggingFaceYoloV8Detector → Yolo11Detector, and registry key "hf_yolov8" → "yolo11" in all 11 files. Loads YOLO11m COCO-pretrained via hf://Ultralytics/YOLO11. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>02d40a4 -
feat(segmentation): add YSAM2 variants and mission-aware detector mapping
Zhen Ye7d48e41 -
chore: remove dead code and unused frontend modules
Zhen Ye5a72443 -
refactor(gsam2): make SAM2 detector-agnostic
Zhen Yed23c4ea -
perf: increase production keyframe step from 20 to 60
Zhen Ye +1: Claude Opus 4.6Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a0fe24f
-
feat: expose num_maskmem parameter in benchmark profile/analysis endpoints
Zhen Ye +1: Claude Opus 4.6Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>72a0c0b -
feat: add benchmark profiler & roofline analysis system
Zhen Ye +1: Claude Opus 4.6Add hardware extraction, per-frame GPU/CPU profiling with CUDA events, and automated roofline analysis for detection and segmentation modes. New endpoints: - GET /benchmark/hardware — cached hardware specs - POST /benchmark/profile — per-frame timing breakdown - POST /benchmark/analysis — full roofline with bottleneck ID Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>927cf02
-
feat(frontend): improve UI for detection and chat interaction
Zhen Yeff48b97 -
feat(backend): enhance inference pipeline with GLM logic and structured outputs
Zhen Ye05c8c5b
-
Handle GPT refusal paths and preserve assessment status
Zhen Ye733ca2c
-
Refine Grounded-SAM2 tracking behavior and video frame handling
Zhen Ye89e1672 -
Remove SAM3 and standardize segmentation on Grounded-SAM2
Zhen Yea637bd3
-
Add diagnostic logging to pinpoint tracker dropping detections
Zhen Ye +1: Claude Opus 4.64 targeted log points across inference.py and utils/tracker.py to diagnose why first-frame preview shows bounding boxes but the processed video has none — the tracker is the prime suspect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>235e34b
-
Fix Grounding DINO detections dropped by ByteTracker threshold mismatch
Zhen Ye +1: Claude Opus 4.6Lower ByteTracker new_track_thresh from 0.40 to 0.30 so DINO scores (≥0.35) can spawn tracks. Harden DINO _post_process to remove bare fallback that silently dropped to HuggingFace default threshold of 0.25. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>8f30f48
-
Refine GPT enrichment logic, add relevance badges to UI, and fix minor model/logging issues
Zhen Ye47750a9 -
Add StreamingVideoWriter (ffmpeg pipe), reuse first-frame GPT results in inference pipeline
Zhen Ye014fc47 -
GPT reasoning: add in-memory frame encoding, image_b64 support, use gpt-4o-mini for relevance
Zhen Ye3c4f14e
-
Refactor ByteTracker: add min_hits confirmation, split thresholds, associate unconfirmed tracks
Zhen Ye1932c96 -
Tune ByteTracker: lower track_thresh to 0.3, increase track_buffer to 60
Zhen Ye0c1b533 -
Add vision-grounded mission parsing: send first frame to GPT for object class refinement
Zhen Ye569aa92 -
Update overlays, tracker, GPT reasoning, and threat chat improvements
Zhen Ye1425674 -
feat: Add LLM post-processing relevance gate for broad detection
Zhen Ye +1: Claude Opus 4.6Move LLM's role from pre-processing (choosing detector classes) to post-processing (filtering detected objects against mission intent). Fast-path (comma-separated labels) is unchanged. LLM-path now detects broadly (all COCO classes or domain-expanded queries), then asks GPT once on frame 0 which labels are mission-relevant, caching the result for deterministic filtering on all subsequent frames. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>495a767 -
feat: Add mission-driven object relevance abstractions
Zhen Ye +1: Claude Opus 4.6Introduces MissionSpecification extraction, object relevance gating, domain-aware GPT prompts, assessment provenance/staleness tracking, and UNASSESSED status distinct from score 0. Mission text is now parsed into structured intent before reaching the detector, and only mission-relevant objects are sent to GPT for threat assessment. Key changes: - MissionSpecification + RelevanceCriteria schemas (utils/schemas.py) - Mission text parser with fast-path and LLM extraction (utils/mission_parser.py) - Deterministic relevance gate between detection and GPT (utils/relevance.py) - Domain-aware GPT system prompts with mission context injection - Temporal validity tracking (ASSESSED/UNASSESSED/STALE) in tracker - LEGACY mode when no mission text provided (GPT auto-disabled) - Frontend: deterministic sort, UNASSESSED/STALE badges, range qualifiers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>ec42e2b
-
fix: Remove chat emojis, improve tracker GPT metadata sync
Zhen Ye- Remove emojis from threat chat UI (use text labels YOU/TAC/SYS) - Fix GPT reasoning to use track_id instead of index-based IDs - Add inject_metadata() to ByteTracker for post-update GPT sync - Add GPT_SYNC_KEYS for all threat intelligence fields - Reset track ID counter on new ByteTracker instance - Minor fixes to overlays, cards, and main.js8954d50 -
feat: Add threat chat feature for Q&A about detected threats
Zhen Ye- Add utils/threat_chat.py with chat_about_threats() function - Add POST /chat/threat endpoint to app.py - Add frontend chat panel to Tab 1 (index.html) - Add js/ui/chat.js UI module - Add chatAboutThreats() API function to client.js - Add chat panel CSS styles to style.css689b163
-
feat(threat-assessment): implement naval threat analysis with GPT-4o\n\n- Rename utils/gpt_distance.py to utils/gpt_reasoning.py and update logic for 15 naval threat features\n- Add Pydantic schemas for NavalThreatAssessment in utils/schemas.py\n- Update backend (app.py, inference.py) to use new threat estimation and pass full metadata\n- refactor(frontend): render threat level badges and detailed feature table in UI
Zhen Yeb2c37ab
-
Fix: IndexError in ByteTracker
Zhen Ye6d57a30 -
feat: replace SimpleTracker with ByteTrack
Zhen Yee2ab4dc
-
Add granular logging for frame processing and ffmpeg
Zhen Yea4f6ac4
-
fix: Switch to gpt-4o-mini and verified queue safety
Zhen Yef9fb35b
-
feat: Implement SAHI Tiling for 4K video detection
Zhen Ye- Added utils/tiling.py for image slicing and NMS - Updated DroneYolo and HFYoloV8 to auto-tile images > 3000px width - Uses 1280x1280 tiles with 20% overlap for maximum small object recallb121a08 -
feat: Implement AsyncVideoReader for parallel video decoding
Zhen YeAddresses CPU bottleneck during inference by moving video decoding to a background thread with a prefetch buffer. Integrated into inference pipeline.2ec5dcb
-
fix: Handle empty GPT responses and log outputs
Zhen Ye645cc72
-
Feature: GPT distance integration, Radar Map, and optional legacy depth
Zhen Yeaf004bd
-
update:inference pipeline optimization
Zhen Ye7f8fcb7
-
inital commit
Zhen Yef297b77