| --- |
| license: apache-2.0 |
| tags: |
| - pytorch |
| - multimodal |
| - video-temporal-grounding |
| - moment-detr |
| - cctv |
| - surveillance |
| - video-understanding |
| - temporal-grounding |
| library_name: pytorch |
| pipeline_tag: other |
| language: |
| - en |
| --- |
| |
| # MMVTG: Checkpoints for CCTV-Based Multi-Modal Video Temporal Grounding |
|
|
| This repository hosts the **official model checkpoints** accompanying our paper: |
|
|
| > **Cross-Attention-Based Intelligent Video Temporal Grounding for CCTV-Based Crime Identification in Smart Cities** |
|
|
| The released checkpoints are intended to support research reproducibility, benchmarking, and future work in multi-modal video understanding and intelligent surveillance. |
|
|
| --- |
|
|
| # Overview |
|
|
| This work extends the excellent **Moment-DETR** framework with a reproducible workflow tailored for Multi-Modal Video Temporal Grounding (MMVTG) in CCTV crime investigation scenarios. |
|
|
| Our contributions include: |
|
|
| * Weakly-supervised pretraining using ASR captions |
| * Fine-tuning on the QVHighlights benchmark |
| * Inference-time contextual grounding using retrieved FIR (First Information Report) documents |
| * End-to-end reproducible training and inference workflow |
| * A full-stack deployment application for interactive inference |
|
|
| **This repository contains only the released model checkpoints.** |
|
|
| The accompanying GitHub repository contains: |
|
|
| * Source code |
| * Training scripts |
| * Colab notebooks |
| * Deployment application |
| * Documentation |
| * Reproducibility guide |
| * Paper companion resources |
|
|
| --- |
|
|
| # Available Checkpoints |
|
|
| | Checkpoint | Filename | Purpose | |
| | ----------------------- | ----------------------------------- | ---------------------------------------------------------------- | |
| | ASR Pretrained | `mmvtg-pretrained-asr.ckpt` | Weakly-supervised pretraining using ASR captions | |
| | QVHighlights Fine-tuned | `mmvtg-finetuned-qvhighlights.ckpt` | Fine-tuned model for downstream MMVTG tasks | |
| | Best Validation Model | `mmvtg-best-val.ckpt` | Recommended checkpoint for evaluation, inference, and deployment | |
|
|
| --- |
|
|
| # Training Workflow |
|
|
| The released checkpoints correspond to the following pipeline: |
|
|
| ```text |
| ASR Caption Pretraining |
| │ |
| ▼ |
| Fine-tuning on QVHighlights |
| │ |
| ▼ |
| Validation |
| │ |
| ▼ |
| Best Checkpoint Selection |
| │ |
| ▼ |
| Inference |
| │ |
| ▼ |
| Grounded Inference using Retrieved FIR Reports |
| ``` |
|
|
| --- |
|
|
| # Intended Use |
|
|
| These checkpoints are released for: |
|
|
| * Academic research |
| * Reproducibility of the accompanying paper |
| * Benchmarking |
| * Educational purposes |
| * Experimental extensions of Moment-DETR |
|
|
| --- |
|
|
| # Limitations |
|
|
| This work **extends** the Moment-DETR framework. |
|
|
| The retrieval component is performed **only during inference** by augmenting the input query with retrieved FIR reports to provide additional contextual information. |
|
|
| This repository **does not implement a complete Retrieval-Augmented Generation (RAG) architecture** or an end-to-end retrieval-training pipeline. |
|
|
| --- |
|
|
| # Repository Structure |
|
|
| This Hugging Face repository contains only the released model checkpoints. |
|
|
| The complete project is organized as: |
|
|
| * Research implementation |
| * Deployment application |
| * Paper companion repository |
| * Documentation |
| * Reproducibility resources |
|
|
| --- |
|
|
| # Citation |
|
|
| If you use these checkpoints in your research, please cite our paper. |
|
|
| BibTeX will be added once the final publication metadata becomes available. |
|
|
| ```bibtex |
| % Citation coming soon |
| ``` |
|
|
| --- |
|
|
| # Acknowledgements |
|
|
| This work builds upon the excellent open-source implementation of **Moment-DETR** developed by Lei et al. |
|
|
| We gratefully acknowledge the original authors for making their implementation publicly available and for providing the foundation upon which this work was developed. |
|
|
| --- |
|
|
| # Related Resources |
|
|
| | Resource | Link | |
| | ------------------------------- | --------------------------------------- | |
| | Research Companion Repository | https://github.com/RA7AN/MMVTG | |
| | Research Paper | https://ieeexplore.ieee.org/document/11548957| |
| | Original Moment-DETR Repository | https://github.com/jayleicn/moment_detr | |
| |
| --- |
| |
| # License |
| |
| These checkpoints are released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). See the accompanying [GitHub repository](https://github.com/RA7AN/MMVTG) for full licensing details and usage terms. |
| |
