Video-Text-to-Text
Transformers
Safetensors
molmo2
image-text-to-text
video
object-tracking
ecology
custom_code
Instructions to use tidalove/Molmo2Fish with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tidalove/Molmo2Fish with Transformers:
# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("tidalove/Molmo2Fish", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -52,33 +52,13 @@ corrupted correction, correction of real model predictions at two quality levels
|
|
| 52 |
[dataset card](https://huggingface.co/datasets/tidalove/cfc-track-instruction) for what each
|
| 53 |
config contains.
|
| 54 |
|
| 55 |
-
## Usage
|
| 56 |
-
|
| 57 |
-
```bash
|
| 58 |
-
git clone https://github.com/tidalove/molmo2fish.git && cd molmo2fish
|
| 59 |
-
pip install torchcodec && pip install -e .[all]
|
| 60 |
-
|
| 61 |
-
export MOLMO_DATA_DIR=./data
|
| 62 |
-
python -m scripts.download_datasets cfc --n-procs 8
|
| 63 |
-
|
| 64 |
-
hf download tidalove/Molmo2Fish --exclude "*.tar" --local-dir Molmo2Fish-HF/step420-hf
|
| 65 |
-
python launch_scripts/hf_eval.py Molmo2Fish-HF/step420-hf \
|
| 66 |
-
cfc_hf_correction_molmo_low_full_eval_2fps
|
| 67 |
-
```
|
| 68 |
-
|
| 69 |
-
Correction tasks report `HOTA_before` (the tracks the model was handed), `HOTA_after` (what
|
| 70 |
-
it returned), and `norm_delta_HOTA` (the fraction of available headroom it closed), alongside
|
| 71 |
-
a per-river breakdown and the directional net-count error `nMAE`.
|
| 72 |
-
|
| 73 |
-
Evaluation runs on 6 fps clips with tracks annotated at 2 fps, sliced from the original CFC
|
| 74 |
-
videos — metrics are not comparable to those computed on the original CFC release.
|
| 75 |
-
|
| 76 |
## Citation
|
| 77 |
|
| 78 |
```bibtex
|
| 79 |
@article{molmo2fish,
|
| 80 |
title={Teach a Molmo2Fish: Towards interactive fish tracking with natural language guidance},
|
| 81 |
author={Kai van Brunt and Justin Kay and Sara Beery},
|
| 82 |
-
year={2026}
|
|
|
|
| 83 |
}
|
| 84 |
```
|
|
|
|
| 52 |
[dataset card](https://huggingface.co/datasets/tidalove/cfc-track-instruction) for what each
|
| 53 |
config contains.
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
## Citation
|
| 56 |
|
| 57 |
```bibtex
|
| 58 |
@article{molmo2fish,
|
| 59 |
title={Teach a Molmo2Fish: Towards interactive fish tracking with natural language guidance},
|
| 60 |
author={Kai van Brunt and Justin Kay and Sara Beery},
|
| 61 |
+
year={2026},
|
| 62 |
+
url={https://arxiv.org/abs/2608.18602}
|
| 63 |
}
|
| 64 |
```
|