CausalVLM

Dense Causal Captioning (DCC): joint dense event captioning + causal-graph prediction for video.

CausalVLM produces timestamped dense event descriptions for a video and predicts a causal (prerequisite) graph over those events.

Usage

from huggingface_hub import snapshot_download
d = snapshot_download("causalvlm/CausalVLM")
import sys; sys.path.insert(0, d)
from modeling_causalvlm import CausalVLM

model = CausalVLM(d, device="cuda")
out = model.run("my_video.mp4")
for t, desc in out["events"]:
    print(f"[{t:.1f}s] {desc}")
print("causal edges:", out["causal_edges"])

Example output (illustrative)

[t1s] <event 1 description>
[t2s] <event 2 description>
[t3s] <event 3 description>
causal edges: [(1, 2), (2, 3)]

Each (i, j) edge means event i is a prerequisite of event j.

License

CC BY-NC-ND 4.0.

Citation

(anonymous — under review)