---
license: apache-2.0
tags:
- pytorch
- multimodal
- video-temporal-grounding
- moment-detr
- cctv
- surveillance
- video-understanding
- temporal-grounding
library_name: pytorch
pipeline_tag: other
language:
- en
---

# MMVTG: Checkpoints for CCTV-Based Multi-Modal Video Temporal Grounding

This repository hosts the **official model checkpoints** accompanying our paper:

> **Cross-Attention-Based Intelligent Video Temporal Grounding for CCTV-Based Crime Identification in Smart Cities**

The released checkpoints are intended to support research reproducibility, benchmarking, and future work in multi-modal video understanding and intelligent surveillance.

---

# Overview

This work extends the excellent **Moment-DETR** framework with a reproducible workflow tailored for Multi-Modal Video Temporal Grounding (MMVTG) in CCTV crime investigation scenarios.

Our contributions include:

* Weakly-supervised pretraining using ASR captions
* Fine-tuning on the QVHighlights benchmark
* Inference-time contextual grounding using retrieved FIR (First Information Report) documents
* End-to-end reproducible training and inference workflow
* A full-stack deployment application for interactive inference

**This repository contains only the released model checkpoints.**

The accompanying GitHub repository contains:

* Source code
* Training scripts
* Colab notebooks
* Deployment application
* Documentation
* Reproducibility guide
* Paper companion resources

---

# Available Checkpoints

| Checkpoint              | Filename                            | Purpose                                                          |
| ----------------------- | ----------------------------------- | ---------------------------------------------------------------- |
| ASR Pretrained          | `mmvtg-pretrained-asr.ckpt`         | Weakly-supervised pretraining using ASR captions                 |
| QVHighlights Fine-tuned | `mmvtg-finetuned-qvhighlights.ckpt` | Fine-tuned model for downstream MMVTG tasks                      |
| Best Validation Model   | `mmvtg-best-val.ckpt`               | Recommended checkpoint for evaluation, inference, and deployment |

---

# Training Workflow

The released checkpoints correspond to the following pipeline:

```text
ASR Caption Pretraining
        │
        ▼
Fine-tuning on QVHighlights
        │
        ▼
Validation
        │
        ▼
Best Checkpoint Selection
        │
        ▼
Inference
        │
        ▼
Grounded Inference using Retrieved FIR Reports
```

---

# Intended Use

These checkpoints are released for:

* Academic research
* Reproducibility of the accompanying paper
* Benchmarking
* Educational purposes
* Experimental extensions of Moment-DETR

---

# Limitations

This work **extends** the Moment-DETR framework.

The retrieval component is performed **only during inference** by augmenting the input query with retrieved FIR reports to provide additional contextual information.

This repository **does not implement a complete Retrieval-Augmented Generation (RAG) architecture** or an end-to-end retrieval-training pipeline.

---

# Repository Structure

This Hugging Face repository contains only the released model checkpoints.

The complete project is organized as:

* Research implementation
* Deployment application
* Paper companion repository
* Documentation
* Reproducibility resources

---

# Citation

If you use these checkpoints in your research, please cite our paper.

BibTeX will be added once the final publication metadata becomes available.

```bibtex
% Citation coming soon
```

---

# Acknowledgements

This work builds upon the excellent open-source implementation of **Moment-DETR** developed by Lei et al.

We gratefully acknowledge the original authors for making their implementation publicly available and for providing the foundation upon which this work was developed.

---

# Related Resources

| Resource                        | Link                                    |
| ------------------------------- | --------------------------------------- |
| Research Companion Repository   | https://github.com/RA7AN/MMVTG          |
| Research Paper                  | https://ieeexplore.ieee.org/document/11548957|
| Original Moment-DETR Repository | https://github.com/jayleicn/moment_detr |

---

# License

These checkpoints are released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). See the accompanying [GitHub repository](https://github.com/RA7AN/MMVTG) for full licensing details and usage terms.