Add AgriScope model card and project assets

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+images/annotation_pipeline.png filter=lfs diff=lfs merge=lfs -text
+images/architecture.png filter=lfs diff=lfs merge=lfs -text
+images/overview.png filter=lfs diff=lfs merge=lfs -text
+images/qualitative_gcg.png filter=lfs diff=lfs merge=lfs -text
+images/qualitative_tasks.png filter=lfs diff=lfs merge=lfs -text
+images/referring_segmentation_comparison.png filter=lfs diff=lfs merge=lfs -text
+images/task_examples_v3.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,20 @@
+__pycache__/
+*.py[cod]
+*.so
+
+.DS_Store
+Thumbs.db
+
+.venv/
+venv/
+env/
+.env
+
+checkpoints/
+weights/
+outputs/
+results/
+wandb/
+
+*.log
+*.tmp

CITATION.cff

@@ -0,0 +1,20 @@
+cff-version: 1.2.0
+message: "If you use AgriScope in your research, please cite the accompanying paper."
+title: "AgriScope: Pixel-Grounded Multimodal Understanding for Agriculture Images"
+type: software
+authors:
+  - family-names: Boudiaf
+    given-names: Abderrahmene
+  - family-names: Alanssari
+    given-names: Mohamad
+  - family-names: Hussain
+    given-names: Irfan
+  - family-names: Javed
+    given-names: Sajid
+repository-code: "https://github.com/boudiafA/AgriScope"
+url: "https://github.com/boudiafA/AgriScope"
+license: Apache-2.0
+date-released: 2026-07-17
+abstract: >-
+  AgriScope is a unified pixel-grounded multimodal framework for image-level,
+  region-level, and pixel-level understanding of agricultural imagery.

LICENSE

@@ -0,0 +1,201 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+   "License" shall mean the terms and conditions for use, reproduction,
+   and distribution as defined by Sections 1 through 9 of this document.
+
+   "Licensor" shall mean the copyright owner or entity authorized by
+   the copyright owner that is granting the License.
+
+   "Legal Entity" shall mean the union of the acting entity and all
+   other entities that control, are controlled by, or are under common
+   control with that entity. For the purposes of this definition,
+   "control" means (i) the power, direct or indirect, to cause the
+   direction or management of such entity, whether by contract or
+   otherwise, or (ii) ownership of fifty percent (50%) or more of the
+   outstanding shares, or (iii) beneficial ownership of such entity.
+
+   "You" (or "Your") shall mean an individual or Legal Entity
+   exercising permissions granted by this License.
+
+   "Source" form shall mean the preferred form for making modifications,
+   including but not limited to software source code, documentation
+   source, and configuration files.
+
+   "Object" form shall mean any form resulting from mechanical
+   transformation or translation of a Source form, including but
+   not limited to compiled object code, generated documentation,
+   and conversions to other media types.
+
+   "Work" shall mean the work of authorship, whether in Source or
+   Object form, made available under the License, as indicated by a
+   copyright notice that is included in or attached to the work
+   (an example is provided in the Appendix below).
+
+   "Derivative Works" shall mean any work, whether in Source or Object
+   form, that is based on (or derived from) the Work and for which the
+   editorial revisions, annotations, elaborations, or other modifications
+   represent, as a whole, an original work of authorship. For the purposes
+   of this License, Derivative Works shall not include works that remain
+   separable from, or merely link (or bind by name) to the interfaces of,
+   the Work and Derivative Works thereof.
+
+   "Contribution" shall mean any work of authorship, including
+   the original version of the Work and any modifications or additions
+   to that Work or Derivative Works thereof, that is intentionally
+   submitted to Licensor for inclusion in the Work by the copyright owner
+   or by an individual or Legal Entity authorized to submit on behalf of
+   the copyright owner. For the purposes of this definition, "submitted"
+   means any form of electronic, verbal, or written communication sent
+   to the Licensor or its representatives, including but not limited to
+   communication on electronic mailing lists, source code control systems,
+   and issue tracking systems that are managed by, or on behalf of, the
+   Licensor for the purpose of discussing and improving the Work, but
+   excluding communication that is conspicuously marked or otherwise
+   designated in writing by the copyright owner as "Not a Contribution."
+
+   "Contributor" shall mean Licensor and any individual or Legal Entity
+   on behalf of whom a Contribution has been received by Licensor and
+   subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   copyright license to reproduce, prepare Derivative Works of,
+   publicly display, publicly perform, sublicense, and distribute the
+   Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   (except as stated in this section) patent license to make, have made,
+   use, offer to sell, sell, import, and otherwise transfer the Work,
+   where such license applies only to those patent claims licensable
+   by such Contributor that are necessarily infringed by their
+   Contribution(s) alone or by combination of their Contribution(s)
+   with the Work to which such Contribution(s) was submitted. If You
+   institute patent litigation against any entity (including a
+   cross-claim or counterclaim in a lawsuit) alleging that the Work
+   or a Contribution incorporated within the Work constitutes direct
+   or contributory patent infringement, then any patent licenses
+   granted to You under this License for that Work shall terminate
+   as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+   Work or Derivative Works thereof in any medium, with or without
+   modifications, and in Source or Object form, provided that You
+   meet the following conditions:
+
+   (a) You must give any other recipients of the Work or
+       Derivative Works a copy of this License; and
+
+   (b) You must cause any modified files to carry prominent notices
+       stating that You changed the files; and
+
+   (c) You must retain, in the Source form of any Derivative Works
+       that You distribute, all copyright, patent, trademark, and
+       attribution notices from the Source form of the Work,
+       excluding those notices that do not pertain to any part of
+       the Derivative Works; and
+
+   (d) If the Work includes a "NOTICE" text file as part of its
+       distribution, then any Derivative Works that You distribute must
+       include a readable copy of the attribution notices contained
+       within such NOTICE file, excluding those notices that do not
+       pertain to any part of the Derivative Works, in at least one
+       of the following places: within a NOTICE text file distributed
+       as part of the Derivative Works; within the Source form or
+       documentation, if provided along with the Derivative Works; or,
+       within a display generated by the Derivative Works, if and
+       wherever such third-party notices normally appear. The contents
+       of the NOTICE file are for informational purposes only and
+       do not modify the License. You may add Your own attribution
+       notices within Derivative Works that You distribute, alongside
+       or as an addendum to the NOTICE text from the Work, provided
+       that such additional attribution notices cannot be construed
+       as modifying the License.
+
+   You may add Your own copyright statement to Your modifications and
+   may provide additional or different license terms and conditions
+   for use, reproduction, or distribution of Your modifications, or
+   for any such Derivative Works as a whole, provided Your use,
+   reproduction, and distribution of the Work otherwise complies with
+   the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+   Notwithstanding the above, nothing herein shall supersede or modify
+   the terms of any separate license agreement you may have executed
+   with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+   names, trademarks, service marks, or product names of the Licensor,
+   except as required for reasonable and customary use in describing the
+   origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+   agreed to in writing, Licensor provides the Work (and each
+   Contributor provides its Contributions) on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+   implied, including, without limitation, any warranties or conditions
+   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+   PARTICULAR PURPOSE. You are solely responsible for determining the
+   appropriateness of using or redistributing the Work and assume any
+   risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+   whether in tort (including negligence), contract, or otherwise,
+   unless required by applicable law (such as deliberate and grossly
+   negligent acts) or agreed to in writing, shall any Contributor be
+   liable to You for damages, including any direct, indirect, special,
+   incidental, or consequential damages of any character arising as a
+   result of this License or out of the use or inability to use the
+   Work (including but not limited to damages for loss of goodwill,
+   work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses), even if such Contributor
+   has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+   the Work or Derivative Works thereof, You may choose to offer,
+   and charge a fee for, acceptance of support, warranty, indemnity,
+   or other liability obligations and/or rights consistent with this
+   License. However, in accepting such obligations, You may act only
+   on Your own behalf and on Your sole responsibility, not on behalf
+   of any other Contributor, and only if You agree to indemnify,
+   defend, and hold each Contributor harmless for any liability
+   incurred by, or claims asserted against, such Contributor by reason
+   of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work.
+
+   To apply the Apache License to your work, attach the following
+   boilerplate notice, with the fields enclosed by brackets "[]"
+   replaced with your own identifying information. (Don't include
+   the brackets.) The text should be enclosed in the appropriate
+   comment syntax for the file format. We also recommend that a
+   file or class name and description of purpose be included on the
+   same "printed page" as the copyright notice for easier
+   identification within third-party archives.
+
+Copyright [yyyy] [name of copyright owner]
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

README.md

@@ -1,3 +1,277 @@
 ---
 license: apache-2.0
+language:
+  - en
+pipeline_tag: image-text-to-text
+tags:
+  - agriculture
+  - multimodal
+  - vision-language
+  - visual-grounding
+  - image-segmentation
+  - grounded-caption-generation
+  - referring-expression-segmentation
+  - sam2
+  - bioclip
+  - dinov3
+inference: false
 ---
+
+# AgriScope
+
+<p align="center">
+  <strong>Pixel-Grounded Multimodal Understanding for Agriculture Images</strong>
+</p>
+
+<p align="center">
+  <strong>Abderrahmene Boudiaf, Mohamad Alanssari, Irfan Hussain, Sajid Javed</strong><br>
+  Khalifa University of Science and Technology, Abu Dhabi, UAE
+</p>
+
+<p align="center">
+  <a href="https://github.com/boudiafA/AgriScope"><img src="https://img.shields.io/badge/GitHub-Repository-181717" alt="GitHub repository"></a>
+  <img src="https://img.shields.io/badge/Paper-Coming%20soon-lightgrey" alt="Paper coming soon">
+  <img src="https://img.shields.io/badge/Weights-Coming%20soon-lightgrey" alt="Weights coming soon">
+  <img src="https://img.shields.io/badge/Dataset-Coming%20soon-lightgrey" alt="Dataset coming soon">
+</p>
+
+> **Release status:** This repository currently provides the AgriScope model card, project figures, task definitions, and reported manuscript results. Model weights, configuration files, processors, training code, and AgriGround annotations are coming soon.
+
+## Model Description
+
+<p align="center">
+  <img src="images/overview.png" alt="AgriScope and AgriGround overview" width="100%">
+</p>
+
+**AgriScope** is a pixel-grounded multimodal model for agricultural image understanding. It supports image-level, region-level, and pixel-level interaction within one framework, connecting generated agricultural concepts to segmentation masks and spatial annotations.
+
+The model is designed to identify, describe, localize, and segment agricultural entities including plant diseases, lesions, pests, crops, weeds, botanical species, plant organs, and structural features. It supports both text-only responses and responses grounded in masks or normalized bounding boxes.
+
+AgriScope is trained with **AgriGround**, a large-scale pixel-grounded agricultural instruction-tuning dataset containing **503,919 images** and **11,421,148 samples** across **14 tasks**.
+
+### Model Details
+
+| Property | Description |
+|---|---|
+| Model type | Pixel-grounded multimodal language model |
+| Domain | Agriculture and plant imagery |
+| Languages | English |
+| Parameter count | 2B total parameters, as reported in the manuscript |
+| Visual representation | Biological semantic and dense spatial encoders |
+| Grounding | Language-conditioned `[SEG]` states with a SAM2-driven decoder |
+| Adaptation | Projection alignment followed by LoRA instruction tuning |
+| License | Apache-2.0 for repository materials and future code release |
+| Model weights | **Coming soon** |
+
+## Architecture
+
+<p align="center">
+  <img src="images/architecture.png" alt="AgriScope architecture" width="100%">
+</p>
+
+AgriScope combines complementary semantic and spatial pathways:
+
+1. A biological contextual encoder extracts agricultural and biological semantics.
+2. A global contextual encoder preserves texture, morphology, boundaries, and dense spatial information.
+3. Projection layers map visual representations into the multimodal language space.
+4. The language model generates text and special `[SEG]` tokens for grounded concepts.
+5. Each `[SEG]` hidden state conditions the SAM2-driven mask decoder to produce a corresponding pixel-level mask.
+
+The manuscript implementation initializes its visual and grounding components from BioCLIP, DINOv3, and SAM2. The large pretrained encoders remain frozen while alignment modules and LoRA parameters are optimized.
+
+### Inputs
+
+- An agricultural RGB image
+- A natural-language instruction or question
+- An optional region for region-conditioned tasks
+- Optional conversation history for multi-turn interaction
+
+### Outputs
+
+- Natural-language captions and answers
+- Interleaved grounded captions with phrase-mask correspondence
+- Referring-expression, semantic, and part segmentation masks
+- Region-conditioned descriptions and conversations
+- Counts and normalized bounding boxes
+- Multi-turn grounded responses
+
+## Intended Uses
+
+### Primary Research Uses
+
+- Agricultural image captioning and visual question answering
+- Plant disease, pest, weed, crop, and species understanding
+- Referring expression and semantic segmentation
+- Grounded caption generation
+- Region-level and multi-turn grounded interaction
+- Agricultural object counting, detection, and localization
+- Research on interpretable and evidence-grounded agricultural AI
+
+### Out-of-Scope Uses
+
+- Autonomous pesticide, treatment, or crop-management decisions without expert review
+- Safety-critical agricultural robotics without independent perception safeguards
+- Regulatory, insurance, or legal determinations
+- Identification of entities outside the supported visual and agricultural domains
+
+## AgriGround Training Data
+
+<p align="center">
+  <img src="images/annotation_pipeline.png" alt="AgriGround annotation pipeline" width="100%">
+</p>
+
+AgriGround is produced using a four-stage annotation and task-generation pipeline:
+
+1. Generate detailed image captions, class descriptions, counts, and bounding-box metadata.
+2. Correct captions, identify grounded object phrases, and prepare segmentation prompts.
+3. Generate and align segmentation masks with grounded phrases.
+4. Synthesize instruction-following records for the 14 supported tasks.
+
+### Dataset Statistics
+
+| Split | Images | Samples | Average samples/image |
+|---|---:|---:|---:|
+| Train | 401,234 | 9,095,320 | 22.66 |
+| Test | 102,685 | 2,325,828 | 22.66 |
+| **Total** | **503,919** | **11,421,148** | **22.66** |
+
+| Source group | Images | Share |
+|---|---:|---:|
+| Classification datasets | 232,923 | 46.22% |
+| Detection datasets | 27,938 | 5.54% |
+| iNatAg subset | 169,324 | 33.60% |
+| Insects (IP102) | 73,734 | 14.63% |
+
+### Supported Tasks
+
+| Family | Tasks |
+|---|---|
+| Captioning | Image-level captioning, region-level captioning, grounded caption generation |
+| Segmentation | Referring expression segmentation, semantic segmentation, part segmentation |
+| Detection and localization | Phrase grounding, grounded counting, grounded detection, reasoning detection |
+| Conversation and QA | Region-level conversation, multi-turn grounded conversation, classification QA, negative absence QA |
+
+<p align="center">
+  <img src="images/task_examples_v3.png" alt="Task-specific AgriGround examples" width="62%">
+</p>
+
+See [docs/TASKS.md](docs/TASKS.md) for definitions and per-task sample counts.
+
+The source agricultural images are not distributed in this repository. Their original licenses and terms remain applicable. Annotation download instructions and dataset-specific licensing details will accompany the public dataset release.
+
+## Training Procedure
+
+Training follows two stages described in the manuscript:
+
+1. **Visual-language and grounding alignment:** optimize projection layers, grounding modules, and the segmentation decoder while keeping the pretrained backbones frozen.
+2. **Instruction tuning:** apply parameter-efficient LoRA adaptation using the 14 AgriGround tasks while retaining frozen visual encoders.
+
+The joint objective combines autoregressive language modeling with binary cross-entropy and Dice losses for segmentation supervision. Full hyperparameters, preprocessing, and reproducibility scripts will be released with the code.
+
+## Evaluation
+
+The following results are reported in the current manuscript draft.
+
+| Task | Metrics | AgriScope |
+|---|---|---:|
+| Image-level captioning | CIDEr / ASF | **146.4 / 86.7** |
+| Region-level captioning | CIDEr / ASF | **132.5 / 84.8** |
+| Classification QA | Accuracy / F1 | **82.4 / 80.7** |
+| Grounded counting | Accuracy | **74.8** |
+| Semantic segmentation | mIoU / Dice | **66.1 / 78.4** |
+| Referring expression segmentation | J&F / cIoU | **67.30 / 72.65** |
+| Grounded caption generation | METEOR / CIDEr | **27.9 / 118.6** |
+| Grounded caption generation | AP50 / mIoU / Recall | **63.9 / 59.4 / 74.2** |
+
+### Efficiency
+
+| Parameters | GFLOPs | GPU memory | Inference time |
+|---:|---:|---:|---:|
+| **2B** | **177** | **6 GB** | **480 ms/image** |
+
+Complete baseline comparisons, cross-dataset evaluation, ablations, and experimental settings will accompany the paper release.
+
+## Qualitative Results
+
+### Grounded Caption Generation
+
+<p align="center">
+  <img src="images/qualitative_gcg.png" alt="AgriScope grounded caption generation examples" width="100%">
+</p>
+
+### Representative Tasks
+
+<p align="center">
+  <img src="images/qualitative_tasks.png" alt="AgriScope representative task results" width="100%">
+</p>
+
+### Referring Expression Segmentation
+
+<p align="center">
+  <img src="images/referring_segmentation_comparison.png" alt="Referring expression segmentation comparison" width="100%">
+</p>
+
+## Limitations and Risks
+
+- AgriScope can produce plausible but incorrect descriptions, classifications, counts, or masks.
+- Performance may degrade under poor illumination, blur, occlusion, unusual viewpoints, severe domain shift, or very small targets.
+- Fine-grained diseases, species, and pests with similar visual characteristics may be confused.
+- Segmentation quality depends on the visual coverage and annotation quality of the training data.
+- Dataset composition may encode geographic, crop, acquisition, and class-frequency biases from its source datasets.
+- Outputs require review by qualified agricultural experts before being used for diagnosis or management decisions.
+
+## Repository Contents
+
+```text
+.
+|-- README.md
+|-- CITATION.cff
+|-- LICENSE
+|-- docs/
+|   `-- TASKS.md
+`-- images/
+    |-- overview.png
+    |-- architecture.png
+    |-- annotation_pipeline.png
+    |-- task_examples_v3.png
+    |-- qualitative_gcg.png
+    |-- qualitative_tasks.png
+    `-- referring_segmentation_comparison.png
+```
+
+## Release Roadmap
+
+| Resource | Status |
+|---|---|
+| Model weights and configuration | **Coming soon** |
+| Processor and inference example | **Coming soon** |
+| Training and evaluation code | **Coming soon** |
+| AgriGround train/test annotations | **Coming soon** |
+| Paper and final citation | **Coming soon** |
+
+Development updates and future code releases are tracked in the [AgriScope GitHub repository](https://github.com/boudiafA/AgriScope).
+
+## Citation
+
+The final paper link and citation will be added upon release. Until then, please use:
+
+```bibtex
+@misc{boudiaf2026agriscope,
+  title  = {AgriScope: Pixel-Grounded Multimodal Understanding for Agriculture Images},
+  author = {Boudiaf, Abderrahmene and Alanssari, Mohamad and Hussain, Irfan and Javed, Sajid},
+  year   = {2026},
+  note   = {Manuscript under review}
+}
+```
+
+## License
+
+Repository documentation and the future code release are provided under the [Apache License 2.0](LICENSE). AgriGround annotations and source images may be subject to separate terms, which will be documented with the dataset release.
+
+## Acknowledgments
+
+This work was conducted at Khalifa University of Science and Technology, Abu Dhabi, UAE. We acknowledge the creators and maintainers of the agricultural datasets and open-source foundation models that support this research.
+
+## Contact
+
+For questions and collaborations, use the [AgriScope GitHub issue tracker](https://github.com/boudiafA/AgriScope/issues) or the Hugging Face Community tab.

docs/TASKS.md

@@ -0,0 +1,65 @@
+# AgriGround Task Reference
+
+AgriGround contains 14 instruction-tuning tasks spanning captioning, segmentation, localization, counting, conversation, and agricultural question answering. Counts below follow the current manuscript statistics and include both train and test samples.
+
+## Task Distribution
+
+| Task | Output supervision | Train | Test | Total | Share |
+|---|---|---:|---:|---:|---:|
+| Semantic segmentation | Text + masks | 1,651,229 | 422,187 | 2,073,416 | 18.15% |
+| Phrase grounding | Bounding boxes | 1,638,327 | 418,948 | 2,057,275 | 18.01% |
+| Referring expression segmentation | Text + masks | 1,638,327 | 418,948 | 2,057,275 | 18.01% |
+| Part segmentation | Text + masks | 820,240 | 209,124 | 1,029,364 | 9.01% |
+| Region-level conversation | Region + multi-turn text | 749,141 | 191,757 | 940,898 | 8.24% |
+| Region-level captioning | Region + text | 722,833 | 185,023 | 907,856 | 7.95% |
+| Classification QA | Text | 639,962 | 163,786 | 803,748 | 7.04% |
+| Image-level captioning | Text | 400,872 | 102,587 | 503,459 | 4.41% |
+| Grounded caption generation | Interleaved text + masks | 379,023 | 97,007 | 476,030 | 4.17% |
+| Multi-turn grounded conversation | Multi-turn text + masks | 379,023 | 97,007 | 476,030 | 4.17% |
+| Grounded counting | Count | 21,865 | 5,596 | 27,461 | 0.24% |
+| Grounded detection | Count + bounding boxes | 21,637 | 5,535 | 27,172 | 0.24% |
+| Reasoning detection | Count + bounding boxes | 21,637 | 5,535 | 27,172 | 0.24% |
+| Negative absence QA | Text | 11,204 | 2,788 | 13,992 | 0.12% |
+| **Total** |  | **9,095,320** | **2,325,828** | **11,421,148** | **100%** |
+
+## Task Definitions
+
+### Captioning
+
+**Image-level captioning** generates a holistic agricultural description without requiring spatial output.
+
+**Region-level captioning** receives a specified image region and describes the localized agricultural content.
+
+**Grounded caption generation** produces a detailed caption with grounded phrases interleaved with `[SEG]` tokens. Every grounded phrase is paired with a corresponding mask.
+
+### Segmentation
+
+**Referring expression segmentation** segments the object or region described by a natural-language expression.
+
+**Semantic segmentation** segments one or more instances of a requested semantic category.
+
+**Part segmentation** targets a specific plant or object component, such as a stem, branch, leaf, flower structure, or lesion.
+
+### Detection and Localization
+
+**Phrase grounding** maps a phrase to a normalized bounding box.
+
+**Grounded counting** reports the number of requested agricultural objects in an image.
+
+**Grounded detection** reports the count and normalized bounding boxes for all requested instances.
+
+**Reasoning detection** uses an agricultural-context prompt to infer and localize the target instances.
+
+### Conversation and QA
+
+**Region-level conversation** supports follow-up questions about a selected image region.
+
+**Multi-turn grounded conversation** combines image-level discussion with later phrase localization or segmentation.
+
+**Classification QA** answers questions about species, diseases, pests, weeds, conditions, and other agricultural classes.
+
+**Negative absence QA** teaches the model to state when a requested entity is not visible instead of producing an unsupported mask.
+
+## Annotation Representation
+
+The public schema will be documented with the annotation release. Records are expected to include an image reference, task identifier, user instruction, target response, and task-dependent spatial supervision such as masks, regions, or normalized bounding boxes.