| --- |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| base_model: Qwen/Qwen3-VL-8B-Instruct |
| datasets: |
| - KlingTeam/TimePLE-Dataset |
| language: |
| - en |
| tags: |
| - video |
| - video-temporal-grounding |
| - qwen3-vl |
| - timeple |
| - arxiv:2607.23951 |
| --- |
| |
| # TimePLE-8B |
|
|
| TimePLE-8B is a Qwen3-VL-8B-Instruct model fine-tuned for video temporal grounding. Given a video and a natural-language event description, it represents and predicts the event's start/end span through a duration-adaptive TimePLE codec. |
|
|
| This is an inference release: it contains the selected stage-2 model weights and tokenizer/processor assets. The TimePLE model implementation is maintained in the public TimePLE source repository instead of being duplicated in this model repository. Optimizer, scheduler, RNG, and distributed-training states are intentionally excluded. |
|
|
| ## Requirements and loading |
|
|
| Install the TimePLE package from its public GitHub repository first, then install the pinned runtime dependencies from this directory: |
|
|
| ```bash |
| pip install "timeple @ git+https://github.com/KlingAIResearch/TimePLE.git" |
| pip install -r requirements.txt |
| ``` |
|
|
| Importing `timeple` registers the TimePLE configuration, model, and processor with Transformers. The model repository contains no executable Python code and does not require `trust_remote_code=True`. |
|
|
| ```python |
| import torch |
| import timeple # registers TimePLE with the Transformers AutoClasses |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| |
| # A downloaded local model directory also works. |
| model_id = "KlingTeam/TimePLE" |
| |
| processor = AutoProcessor.from_pretrained(model_id) |
| model = AutoModelForImageTextToText.from_pretrained( |
| model_id, |
| dtype=torch.bfloat16, |
| device_map="auto", |
| ).eval() |
| ``` |
|
|
| The TimePLE source repository provides the video preparation, temporal decoding, and benchmark runners. For example: |
|
|
| ```bash |
| SUITE=charades_sta bash scripts/eval/run_suite.sh --models timeple_8b |
| ``` |
|
|
| The plain Transformers load above verifies and exposes the model; use the repository's TimePLE inference/evaluation path for decoded temporal intervals because it supplies video durations and the task-specific post-processing contract. |
|
|
| ## Model details |
|
|
| - Base model: Qwen/Qwen3-VL-8B-Instruct |
| - Architecture: `Qwen3VLForConditionalGenerationWithTimePLECodec` |
| - Task: video temporal grounding / moment retrieval |
| - Weight dtype: bfloat16 |
| - Serialization: four sharded safetensors files |
| - Vision tower: frozen during stage-2 SFT |
| - Language model, TimePLE codec, and MLP interface adapter: trainable during stage-2 SFT |
| - License: Apache-2.0 |
|
|
| The tokenizer vocabulary has 151,671 entries. It preserves all 13 Qwen3-VL additional special tokens and appends: |
|
|
| | Token | ID | Purpose | |
| |---|---:|---| |
| | `<|TIMESTAMP|>` | 151669 | Temporal point marker | |
| | `<|TIMESPAN|>` | 151670 | Temporal interval marker | |
|
|
| The TimePLE codec uses a 128×128 canonical span grid, a duration-adaptive residual decoder, and a learned MLP interface adapter to the 4096-dimensional language-model representation. |
|
|
| ## Intended use |
|
|
| This model is intended for research and evaluation of temporal localization in videos from a textual event query. Suitable uses include moment retrieval experiments, temporal-grounding ablations, and further task-specific fine-tuning. |
|
|
| ## Limitations |
|
|
| - Temporal quality depends on frame sampling, video duration metadata, and the inference pipeline's video preprocessing. |
| - The training data and prompts are primarily English; other languages and unseen domains have not been established here. |
| - Long, ambiguous, repeated, or extremely brief events may yield unstable boundaries. |
| - The reported validation result comes from the training-time validation mixture and should not be compared directly with benchmark test results produced under different sampling or evaluation protocols. |
| - The model inherits limitations and potential biases of Qwen3-VL and of the stage-1/stage-2 training data. |
|
|
| ## Release contents and integrity |
|
|
| Model parameters—including all TimePLE codec and adapter parameters—are stored in the indexed safetensors shards. No pickle-based `.bin`, `.pt`, or `.pth` weight file is required. The model directory contains no executable Python implementation; AutoClass registration is provided by the separately installed `timeple` package. |
|
|
| See `LICENSE` for the release license. Users must also comply with the terms applicable to the base model, input videos, and any datasets they use. |
|
|
| ## Citation |
|
|
| Paper: [TimePLE: Rethinking Temporal Representation for Video Temporal Grounding](https://arxiv.org/abs/2607.23951) |
|
|
