README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen3-VL-Reranker-2B
library_name: transformers
---

# Qwen3-VL-Reranker-0.6B-small

This is a small version of [Qwen/Qwen3-VL-Reranker-2B](https://huggingface.co/Qwen/Qwen3-VL-Reranker-2B) created for testing and development. It preserves the full architecture characteristics of the base (Qwen3-VL multimodal: dense text tower with QK-norms and interleaved mrope, vision tower with deepstack feature injection, spatial-merge projector, tied embeddings) with fewer layers.

## Model Details

- **Base Model**: Qwen/Qwen3-VL-Reranker-2B
- **Architecture**: qwen3_vl (Qwen3VLForConditionalGeneration)
- **Total Parameters**: 0.59B (591,990,784; dense model — activated == total)

## Configuration Changes

The following parameters were reduced from the original model (these are the ONLY changes — all other hyperparameters are identical to the base):

| Parameter | Original | Small |
|-----------|----------|-------|
| `text_config.num_hidden_layers` | 28 | 4 |
| `vision_config.depth` | 24 | 2 |
| `vision_config.deepstack_visual_indexes` | [5, 11, 17] | [1] |

The deepstack index list was trimmed consistently with the new vision depth (index 1 is a valid block index for depth 2), so the deepstack visual feature injection into the first text layers stays exercised (one deepstack merger, as in the base's per-index merger list).

## Checkpoint Structure

Single-file `model.safetensors` (bf16, 85 tensors). Key naming, per-layer key sets, and tensor shapes match the base checkpoint exactly modulo layer counts (verified against the base checkpoint's safetensors header: 0 extra/missing tensor patterns; text layers 28→4, vision blocks 24→2, deepstack mergers 3→1). `lm_head` is tied to the embedding (`tie_word_embeddings=true`) and absent from the checkpoint, as in the base.

## Usage

```python
from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "inference-optimization/Qwen3-VL-Reranker-0.6B-small",
    device_map="auto",
    dtype="bfloat16",
)
processor = AutoProcessor.from_pretrained("inference-optimization/Qwen3-VL-Reranker-0.6B-small")

inputs = processor(text="According to all known laws", return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=20)
print(processor.decode(output[0], skip_special_tokens=True))
```

Multimodal (image + text) inputs are supported through the standard Qwen3-VL vision markup (`<|vision_start|>...<|vision_end|>`); the image+text path is smoke-tested below.

## Creation Process

This model was created using the llm-compressor `create-tiny-model` skill:

1. Base config inspected; the base config JSON was edited directly (text layer count, vision depth, deepstack index list), then reloaded and verified equivalent to the base config modulo the intended changes.
2. Model built with `skip_weights_download` (no base weights downloaded), re-initialized, NaN fixup, bf16 cast.
3. Fine-tuned (text-only) on the skill's copypasta toy dataset (bf16, lr 5e-5, batch 2, early-stopped) to perplexity 1.0.
4. Checkpoint structure validated against the base checkpoint header (see above).
5. Perplexity + generation + multimodal forward validated.

## Validation Output

```
Success: perplexity 1.0007 <= 10.0

Generating sample text:
According to all known laws of aviation, there is no way a bee should be able to fly. Its wings are too small to get its fat little body off the ground. The bee, of course

Multimodal forward smoke test:
inputs: {'input_ids': (1, 22), 'attention_mask': (1, 22), 'mm_token_type_ids': (1, 22), 'pixel_values': (64, 1536), 'image_grid_thw': (1, 3)}
logits shape: (1, 22, 151936) finite: True
multimodal forward OK
```

## Notes

- All hyperparameters other than the table above are identical to the base (text hidden 2048, intermediate 6144, 16 heads / 8 kv heads, head_dim 128, interleaved mrope section [24, 20, 20], rope_theta 5e6, vocab 151936, tied embeddings, vision hidden 1024 / out 2048, patch 16, spatial merge 2, axial vision rope).
- The base is a generative reranker: its `Qwen3VLForConditionalGeneration` weights are a standard causal LM; reranker scoring compares the logits of the `true`/`false` tokens (ids 9693/2152 in the base's `1_LogitScore` sentence-transformers module). The sentence-transformers wrapper files (`modules.json`, `1_LogitScore/`, `config_sentence_transformers.json`, `scripts/`) from the base repo are not part of the transformers checkpoint and are not included; the default and reranker chat templates are included (`chat_template.jinja`, `additional_chat_templates/reranker.jinja`).
- `use_cache` stays `false` in the text config, matching the base.
- This model was fine-tuned to memorize a toy copypasta dataset; it is not intended to produce meaningful relevance scores or content.