Upload README.md with huggingface_hub

README.md

@@ -17,31 +17,31 @@ tags:
 
 A 4-bit [MLX](https://github.com/ml-explore/mlx) conversion of
 [omni-research/Tarsier2-Recap-7b](https://huggingface.co/omni-research/Tarsier2-Recap-7b),
-so it runs natively and fast on Apple Silicon.
+prepared for fast, native inference on Apple Silicon.
 
-Tarsier2 is one of the most detailed open video captioners available, but no
-official MLX build exists. This repo fills that gap: same model, quantized to
-4-bit, ~5.3 GB, running on the Mac GPU through MLX.
+Tarsier2 is among the most detailed open video captioners available, yet no
+official MLX build exists. This repository provides one: the same model,
+quantized to 4-bit (~5.3 GB) and accelerated on the Mac GPU through MLX.
 
 - **Base model:** omni-research/Tarsier2-Recap-7b (Apache-2.0)
 - **Format:** MLX, 4-bit affine quantization (group size 64)
 - **Size on disk:** ~5.3 GB
 - **Underlying architecture:** Qwen2-VL 7B (see conversion notes below)
-- **Runs on:** Apple Silicon (tested on an M4 Pro, 24 GB unified memory)
+- **Tested on:** Apple M4 Pro, 24 GB unified memory
 
-## Why this exists
+## Purpose
 
-Tarsier2-Recap-7b produces unusually specific, grounded video descriptions —
-it reads small details (a sail number, an object in someone's hand,
-architectural features) rather than generic scene summaries. On Apple Silicon
-the only practical way to run a 7B vision-language model at usable speed is
-MLX 4-bit; the full bf16 weights via `transformers` on MPS are far too slow
-(hundreds of seconds per clip). This conversion gets it to roughly
-**7 seconds per short clip** on an M4 Pro.
+Tarsier2-Recap-7b produces unusually specific, grounded video descriptions. It
+identifies small details — a sail number, an object held in a subject's hand,
+architectural features — rather than generic scene summaries. On Apple Silicon,
+the practical way to run a 7B vision-language model at usable speed is MLX
+4-bit; the full bf16 weights under `transformers` on MPS are prohibitively slow
+(hundreds of seconds per clip). This conversion reduces inference to
+approximately **7 seconds per short clip** on an M4 Pro.
 
 ## Usage
 
-Run with [`mlx-vlm`](https://github.com/Blaizzy/mlx-vlm) (0.6.3 or newer).
+Run with [`mlx-vlm`](https://github.com/Blaizzy/mlx-vlm) (0.6.3 or later).
 
 ```python
 from mlx_vlm import load, generate
@@ -49,7 +49,7 @@ from mlx_vlm.prompt_utils import apply_chat_template
 
 model, processor = load("viavicdev/Tarsier2-Recap-7b-MLX-4bit")
 
-# NOTE: feed video as a list of extracted frame images (see gotcha #1 below)
+# Provide the video as a list of extracted frame images (see note 1)
 frames = ["frame_00.jpg", "frame_01.jpg", "frame_02.jpg", "frame_03.jpg"]
 prompt = apply_chat_template(
     processor, model.config,
@@ -58,38 +58,39 @@ prompt = apply_chat_template(
 )
 out = generate(
     model, processor, prompt,
-    image=frames,                 # note: image= (not images=)
+    image=frames,                 # parameter is image= (not images=)
     max_tokens=300,
     temperature=0.3,
-    repetition_penalty=1.3,       # see gotcha #3
+    repetition_penalty=1.3,       # see note 3
     repetition_context_size=40,
 )
 print(out)
 ```
 
-### Gotchas (learned the hard way)
+### Practical notes
 
-1. **Feed frames as images, not a video file.** The `mlx-vlm` 0.6.3 video
-   decoder path is unreliable and can hallucinate. Extract frames with ffmpeg
-   and pass them as a list of images (`image=[...]` with
-   `num_images=N`). This delivers the real frames to the model.
-2. **The chat template is included** in this repo (`chat_template.json` /
-   `chat_template.jinja`). Some MLX repacks ship without it, which breaks
-   `apply_chat_template`.
-3. **Use a repetition penalty.** At 4-bit the model can occasionally loop.
-   `repetition_penalty=1.3`, `repetition_context_size=40`, `temperature=0.3`
-   keeps output clean. In testing, 1 of 22 clips degenerated without it; none with it.
+1. **Provide frames as images rather than a video file.** The `mlx-vlm` 0.6.3
+   video decoder path is unreliable and may produce hallucinated output. Extract
+   frames with ffmpeg and pass them as a list of images (`image=[...]` with
+   `num_images=N`); this delivers the actual frames to the model.
+2. **The chat template is included** in this repository (`chat_template.json`
+   and `chat_template.jinja`). Some MLX repackings omit it, which causes
+   `apply_chat_template` to fail.
+3. **Apply a repetition penalty.** At 4-bit precision the model can occasionally
+   loop. The settings `repetition_penalty=1.3`, `repetition_context_size=40`,
+   and `temperature=0.3` keep the output stable. In internal testing, 1 of 22
+   clips degenerated without a penalty and none did with it.
 
-## How it was converted
+## Conversion notes
 
-Tarsier2 ships as `TarsierForConditionalGeneration` with `model_type: llava`,
-but its text and vision towers **are Qwen2-VL weights under different key
-names**. The `mlx-vlm` llava handler does not support Qwen2-VL's mRoPE
-(it errors with `rope_scaling type only supports linear`); the `qwen2_vl`
-handler does. So the conversion is just a key-remap into native Qwen2-VL layout,
-followed by a standard MLX quantize.
+Tarsier2 is distributed as `TarsierForConditionalGeneration` with
+`model_type: llava`, but its text and vision towers are Qwen2-VL weights under
+different key names. The `mlx-vlm` llava handler does not support Qwen2-VL's
+mRoPE (it raises `rope_scaling type only supports linear`), whereas the
+`qwen2_vl` handler does. The conversion therefore consists of a key-remap into
+the native Qwen2-VL layout, followed by a standard MLX quantization.
 
-Only three key prefixes need remapping:
+Only three key prefixes require remapping:
 
 ```
 language_model.model.*    ->  model.*
@@ -97,35 +98,35 @@ language_model.lm_head.*  ->  lm_head.*
 vision_tower.*            ->  visual.*
 ```
 
-Then a flat `qwen2_vl` config is built from Tarsier's `text_config` +
+A flat `qwen2_vl` configuration is then built from Tarsier's `text_config` and
 `vision_config` (preserving `rope_scaling: mrope`, the vision/image/video token
-ids, etc.), and the remapped weights are quantized:
+ids, and related fields), and the remapped weights are quantized:
 
 ```bash
 mlx_vlm.convert --hf-path ./tarsier2-qwen2vl-hf -q --q-bits 4
 ```
 
-This is a general pattern: **custom Qwen2-VL-based models (Tarsier and
-similar) that lack an official MLX or HF remote-code path can often be run on
-MLX via a key-remap**, as long as you route them through the `qwen2_vl` handler
-so mRoPE is respected.
+This reflects a general pattern: custom Qwen2-VL-based models such as Tarsier,
+which lack an official MLX or HF remote-code path, can often be run on MLX
+through a key-remap, provided they are routed through the `qwen2_vl` handler so
+that mRoPE is respected.
 
 ## Limitations
 
-- 4-bit quantization trades some fidelity for size/speed; for maximum quality
-  use the original bf16 weights on a CUDA machine.
-- English captioning model — it describes in English regardless of spoken
-  language in the clip.
-- Vision only (no audio). It captions what is *seen*, not heard.
-- Follows free-form description better than rigid output schemas; best used as
-  a captioner rather than a strict classifier.
+- 4-bit quantization trades some fidelity for reduced size and higher speed.
+  For maximum quality, use the original bf16 weights on a CUDA device.
+- This is an English captioning model; it describes in English regardless of any
+  spoken language in the clip.
+- Vision only (no audio). It captions what is seen, not what is heard.
+- It follows free-form description more reliably than rigid output schemas, and
+  is best used as a captioner rather than a strict classifier.
 
 ## License and attribution
 
 This conversion inherits the **Apache-2.0** license of the base model. All
-credit for the model itself goes to the Tarsier2 authors
+credit for the model itself belongs to the Tarsier2 authors
 ([omni-research/Tarsier2-Recap-7b](https://huggingface.co/omni-research/Tarsier2-Recap-7b)).
-This repository only provides a format conversion (weight key-remap + MLX 4-bit
-quantization) for the Apple Silicon community.
+This repository provides only a format conversion — a weight key-remap and MLX
+4-bit quantization — for the Apple Silicon community.
 
-If you cite Tarsier2, cite the original authors.
+When citing Tarsier2, please cite the original authors.