Spaces:
Running on CPU Upgrade
Clarification on test corpus overlap, use of released annotations, and TOP_K
Hi organizers, a few clarifications would help us stay within the rules:
Test corpus overlap. Are the ~600 test videos disjoint from the 2,066 clips released in CASCADE cascade-v1.0 (train/val)? If not, roughly what share overlaps?
Using released annotations at inference. If test videos can include released clips, may a submission use the publicly released CASCADE annotations or retrieval labels for those clips at inference time, for example by looking them up by clip ID? Or must systems predict everything from the video pixels and query text alone?
Training data. Can you confirm that training on the released CASCADE annotations, and on PhysicalAI-AV sensor data such as egomotion, is permitted, given that the test container receives only queries.jsonl and the mp4 files? Is it fine for the submitted Apache-2.0 weights to be trained on those datasets?
Result list length. The devkit validator warns "every query returns exactly TOP_K videos; TOP_K is a cap, not a target." Is it acceptable to return a full ranked list of TOP_K videos per query, or are submissions expected to return only predicted matches?
You can use the public CASCADE annotations. However, I don't think that looking by clip ID will produce a competitive solution.
Training data. Can you confirm that training on the released CASCADE annotations, and on PhysicalAI-AV sensor data such as egomotion, is permitted, given that the test container receives only queries.jsonl and the mp4 files? Is it fine for the submitted Apache-2.0 weights to be trained on those datasets? --> Yes, you can train on it (that is the whole point).
Yes, returning a full ranked list of up to TOP_K (100) videos per query is acceptable and recommended. TOP_K is an upper bound: lists longer than 100 are rejected.