Video-Text-to-Text
Transformers
Safetensors
causalvlm
text-generation
dense-video-captioning
causal-reasoning
video-understanding
Instructions to use causalvlm/CausalVLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use causalvlm/CausalVLM with Transformers:
# Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("causalvlm/CausalVLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
| license: cc-by-nc-nd-4.0 | |
| library_name: transformers | |
| pipeline_tag: video-text-to-text | |
| tags: | |
| - dense-video-captioning | |
| - causal-reasoning | |
| - video-understanding | |
| # CausalVLM | |
| **Dense Causal Captioning (DCC): joint dense event captioning + causal-graph prediction for video.** | |
| CausalVLM produces timestamped dense event descriptions for a video and predicts a causal (prerequisite) graph over those events. | |
| ## Usage | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| d = snapshot_download("causalvlm/CausalVLM") | |
| import sys; sys.path.insert(0, d) | |
| from modeling_causalvlm import CausalVLM | |
| model = CausalVLM(d, device="cuda") | |
| out = model.run("my_video.mp4") | |
| for t, desc in out["events"]: | |
| print(f"[{t:.1f}s] {desc}") | |
| print("causal edges:", out["causal_edges"]) | |
| ``` | |
| ## Example output (illustrative) | |
| ``` | |
| [t1s] <event 1 description> | |
| [t2s] <event 2 description> | |
| [t3s] <event 3 description> | |
| causal edges: [(1, 2), (2, 3)] | |
| ``` | |
| Each `(i, j)` edge means event *i* is a prerequisite of event *j*. | |
| ## License | |
| CC BY-NC-ND 4.0. | |
| ## Citation | |
| ``` | |
| (anonymous — under review) | |
| ``` | |