File size: 3,303 Bytes
4883b53 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# FISHER: Gradient-Decoupled Hierarchical Multi-Task Learning for Fine-Grained Aquatic Species Recognition
This repository contains the official implementation of **FISHER**, a novel framework designed for the fine-grained recognition of aquatic species, specifically addressing the challenges of long-tailed distributions where ultra-rare species are poorly represented.
FISHER aligns network optimization with the natural biological hierarchy of aquatic species by breaking down recognition into three interrelated sub-tasks: semantic segmentation of anatomical parts, morphological trait prediction, and species classification.
## ✨ Key Features
* **Detached Hierarchical Architecture:** Enforces a unidirectional information flow (Segmentation → Traits → Species) and applies gradient detachment at task boundaries to prevent high-level classification objectives from corrupting lower-level morphological representations.
* **Prototype-Based Segmentation:** Replaces conventional decoders with learnable prototypes and orthogonality regularization, enabling compact, disentangled, and interpretable delineations of subtle anatomical structures.
* **Dynamic Task Balancing:** Employs homoscedastic uncertainty weighting to dynamically balance the contributions of dense tasks (segmentation) with higher-level tasks during training.
* **High Performance:** Achieves state-of-the-art results on the large-scale Fish-Vista dataset, including 97.7% mAP for unseen trait identification and a 13.4% accuracy improvement for ultra-rare species compared to strong baselines.
## ⚙️ Installation
1. **Clone the repository:**
```bash
git clone https://github.com/phucngvinuni/MTL
cd MTL
```
2. **Install dependencies:**
```bash
pip install torch torchvision pandas numpy tqdm scikit-learn torchmetrics opencv-python matplotlib
```
## 🚀 Usage
### 1. Training (Model)
To train the FISHER model from scratch:
```bash
python train_detached.py
```
**Config:** Batch size 32 (default evaluated in paper), Learning Rate 1e-4 using AdamW, 50 Epochs with Cosine Annealing.
* **Output:** Checkpoints will be automatically saved to the `checkpoints_detached/` directory.
### 2. Evaluation
To evaluate the trained model across all three hierarchical tasks (Species Classification, Trait Identification, and Semantic Segmentation) and generate the final metrics JSON:
```bash
python evaluationdetached.py
```
* **Note:** Ensure you update the `CHECKPOINT_PATH` inside `evaluationdetached.py` to point to your best saved model (e.g., `best_model.pth`) before running.
## 💾 Checkpoints
Pre-trained model checkpoints can be found at: `[Insert Link to HuggingFace / Google Drive / Release Assets Here]`
## 📖 Citation
If you find this code or our research helpful in your work, please cite our paper:
```bibtex
@misc{nguyen2026fishergradientdecoupledhierarchicalmultitask,
title={FISHER: Gradient-Decoupled Hierarchical Multi-Task Learning for Fine-Grained Aquatic Species Recognition},
author={Phuc H. Nguyen and Ba Hung Ngo and Mai Phuong Tran and Cuong D. Do and Van-Dinh Nguyen},
year={2026},
eprint={2607.20523},
archivePrefix={arXiv},
primaryClass={q-bio.QM},
url={https://arxiv.org/abs/2607.20523},
}
```
***
|