File size: 1,038 Bytes
722bda8 4fca835 722bda8 4fca835 722bda8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | """
Scene-level multimodal indexing for Search-UI.
This package builds *scene descriptions* — short paragraphs that fuse visual
content, dialogue, and (optionally) non-speech audio cues for a fixed window
of a video. Descriptions are embedded with a text model and stored alongside
existing subtitle embeddings so the search layer can answer narrative queries
like "people who disagree and then forgive each other" that the literal
subtitle text would miss.
The VLM step itself runs via OpenRouter (scripts/openrouter_scene_processor.py);
this package provides the local building blocks (frame sampling, windowing,
embedding, DB) that the prepare/persist stages use.
Submodules:
frames - ffmpeg frame sampling + perceptual-hash dedup.
scenes - VTT-aligned scene window construction.
embed - bge-m3 text embedding wrapper.
scene_db - scene_index.db schema + atomic upserts.
index_filter / exclusions - which catalog videos are in scope.
audio_tags - CLAP zero-shot audio event tagging.
"""
|