README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3-VL-8B-Instruct
datasets:
  - KlingTeam/TimePLE-Dataset
language:
  - en
tags:
  - video
  - video-temporal-grounding
  - qwen3-vl
  - timeple
  - arxiv:2607.23951
---

# TimePLE-8B

TimePLE-8B is a Qwen3-VL-8B-Instruct model fine-tuned for video temporal grounding. Given a video and a natural-language event description, it represents and predicts the event's start/end span through a duration-adaptive TimePLE codec.

This is an inference release: it contains the selected stage-2 model weights and tokenizer/processor assets. The TimePLE model implementation is maintained in the public TimePLE source repository instead of being duplicated in this model repository. Optimizer, scheduler, RNG, and distributed-training states are intentionally excluded.

## Requirements and loading

Install the TimePLE package from its public GitHub repository first, then install the pinned runtime dependencies from this directory:

```bash
pip install "timeple @ git+https://github.com/KlingAIResearch/TimePLE.git"
pip install -r requirements.txt
```

Importing `timeple` registers the TimePLE configuration, model, and processor with Transformers. The model repository contains no executable Python code and does not require `trust_remote_code=True`.

```python
import torch
import timeple  # registers TimePLE with the Transformers AutoClasses
from transformers import AutoModelForImageTextToText, AutoProcessor

# A downloaded local model directory also works.
model_id = "KlingTeam/TimePLE"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()
```

The TimePLE source repository provides the video preparation, temporal decoding, and benchmark runners. For example:

```bash
SUITE=charades_sta bash scripts/eval/run_suite.sh --models timeple_8b
```

The plain Transformers load above verifies and exposes the model; use the repository's TimePLE inference/evaluation path for decoded temporal intervals because it supplies video durations and the task-specific post-processing contract.

## Model details

- Base model: Qwen/Qwen3-VL-8B-Instruct
- Architecture: `Qwen3VLForConditionalGenerationWithTimePLECodec`
- Task: video temporal grounding / moment retrieval
- Weight dtype: bfloat16
- Serialization: four sharded safetensors files
- Vision tower: frozen during stage-2 SFT
- Language model, TimePLE codec, and MLP interface adapter: trainable during stage-2 SFT
- License: Apache-2.0

The tokenizer vocabulary has 151,671 entries. It preserves all 13 Qwen3-VL additional special tokens and appends:

| Token | ID | Purpose |
|---|---:|---|
| `<|TIMESTAMP|>` | 151669 | Temporal point marker |
| `<|TIMESPAN|>` | 151670 | Temporal interval marker |

The TimePLE codec uses a 128×128 canonical span grid, a duration-adaptive residual decoder, and a learned MLP interface adapter to the 4096-dimensional language-model representation.

## Intended use

This model is intended for research and evaluation of temporal localization in videos from a textual event query. Suitable uses include moment retrieval experiments, temporal-grounding ablations, and further task-specific fine-tuning.

## Limitations

- Temporal quality depends on frame sampling, video duration metadata, and the inference pipeline's video preprocessing.
- The training data and prompts are primarily English; other languages and unseen domains have not been established here.
- Long, ambiguous, repeated, or extremely brief events may yield unstable boundaries.
- The reported validation result comes from the training-time validation mixture and should not be compared directly with benchmark test results produced under different sampling or evaluation protocols.
- The model inherits limitations and potential biases of Qwen3-VL and of the stage-1/stage-2 training data.

## Release contents and integrity

Model parameters—including all TimePLE codec and adapter parameters—are stored in the indexed safetensors shards. No pickle-based `.bin`, `.pt`, or `.pth` weight file is required. The model directory contains no executable Python implementation; AutoClass registration is provided by the separately installed `timeple` package.

See `LICENSE` for the release license. Users must also comply with the terms applicable to the base model, input videos, and any datasets they use.

## Citation

Paper: [TimePLE: Rethinking Temporal Representation for Video Temporal Grounding](https://arxiv.org/abs/2607.23951)