jquenum commited on
Commit
4c3c38f
·
verified ·
1 Parent(s): 1e10baa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -42
README.md CHANGED
@@ -27,7 +27,7 @@ With its advanced reasoning capabilities and superior performance on geospatial
27
  - **Training data**: we introduce the Geospatial Reasoning Segmentation Dataset (GRES), a collection of vision and language data designed around
28
  remote-sensing applications. GRES consists of two core components: PreGRES, a dataset consisting of over 1M remote-sensing specific visual instruction-tuning Q/A pairs for pre-training geospatial models, and GRES, a semi-synthetic dataset specialized for reasoning segmentation of remote-sensing data and consisting of 9,205 images and 27,615 natural language queries/answers within those images. From this LISAt dataset, we generate train, test, and validation splits consisting of 7,205, 1,500, and 500 images respectively.
29
 
30
- - **mplementation Details**: LISAT and LISATPRE are trained on eight DGX A100 80GB GPUs. In the first stage, we pretrain LISATPRE (context length = 2048) using LoRA for 1 epoch on PreGRES with next-token prediction cross-entropy loss. We employ the AdamW optimizer with a learning rate of 3e−4 and a cosine-decay learning rate scheduler, setting the batch size to 2 and gradient accumulation
31
  steps to 6.
32
 
33
  In the second stage, we train LISAT using GRES, as well as two traditional natural image referring segmentation datasets, FP-Ref-COCO and ReasonSeg. LoRA is applied to LISATPRE , while the SAM decoder undergoes full fine-tuning. The learning rate is set to 3e−4, with all other configurations remaining the
@@ -92,47 +92,6 @@ outputs = model.generate(**inputs)
92
  ```
93
 
94
 
95
-
96
- ## Intended Use
97
-
98
- ### Intended Use Cases
99
- LISAT-7B is intended for both **commercial** and **research** use, specifically in **geospatial** and **remote-sensing image analysis** tasks. The model is designed to generate segmentation masks, provide visual descriptions, and answer complex queries about remote-sensing images. It excels in reasoning over implicit queries referring to multiple objects, enabling more advanced analysis compared to traditional segmentation models. LISAT-7B is suitable for tasks in fields such as environmental monitoring, urban planning, and disaster response, where high-quality, detailed analysis of remote-sensing imagery is required.
100
-
101
- Additionally, LISAT-7B can be adapted for various **natural language processing** tasks, such as generating captions, answering questions, or improving other models through synthetic data generation and fine-tuning. The LISAT-7B model is licensed under the **LISAT Community License**, allowing for these use cases in a responsible manner.
102
-
103
- ### Out-of-scope Use
104
- LISAT-7B is **not** intended for use in any applications that violate applicable laws or regulations, including trade compliance and data protection laws. Use in any way that violates the **Acceptable Use Policy** or the **LISAT Community License** is prohibited. Additionally, any use of LISAT-7B beyond the supported **remote-sensing imagery** tasks outlined in this model card, including use in unsupported domains or languages, is not permitted.
105
-
106
- ## Responsibility & Safety
107
-
108
- As part of our responsible release approach, we followed a multi-pronged strategy to manage trust and safety risks in LISAT-7B:
109
-
110
- 1. **Enable developers** to deploy helpful, safe, and flexible experiences tailored to their target audience and the specific use cases supported by LISAT-7B.
111
- 2. **Protect developers** against adversarial users who may attempt to exploit the model's capabilities for harmful purposes.
112
- 3. **Provide safeguards** for the community to help prevent the misuse of LISAT-7B, ensuring responsible use in remote-sensing applications.
113
-
114
- ### Responsible Deployment
115
-
116
- LISAT-7B is a model designed to be used in various geospatial tasks, particularly remote-sensing applications. Our approach is focused on building helpful models that enable the world to benefit from the power of this technology, while aligning model safety with generic use cases to address a standard set of risks and harms.
117
-
118
- Developers are encouraged to tailor safety measures specific to their use cases, defining their own policies and deploying LISAT-7B with the necessary safeguards in their systems. LISAT-7B was developed following best practices outlined in our Responsible Use Guide. You can refer to the [Responsible Use Guide](#) to learn more about how to implement and enforce these safety practices in your deployment.
119
-
120
- ## Critical and Other Risks
121
-
122
- We specifically focused our efforts on mitigating the following critical risk areas:
123
-
124
- 1. **Geospatial Misuse**
125
- We conducted risk assessments to evaluate whether **LISAT-7B** could be exploited by malicious actors for harmful purposes, such as generating misleading or inaccurate geospatial interpretations that may contribute to dangerous decision-making in critical applications (e.g., military or surveillance purposes).
126
-
127
- 2. **Environmental Impact**
128
- Given the model's domain in remote-sensing imagery, we performed analyses to assess whether **LISAT-7B** could potentially misinterpret environmental data, leading to unsafe or harmful recommendations related to climate or natural resource management.
129
-
130
- 3. **Privacy and Security Risks**
131
- As **LISAT-7B** processes and analyzes images, we ensured that the model does not inadvertently expose sensitive or personal data, adhering to privacy standards and avoiding the generation of harmful insights from publicly or privately sourced imagery.
132
-
133
- 4. **Model Bias in Object Segmentation**
134
- Our evaluation includes assessing whether **LISAT-7B** exhibits bias in its segmentation or interpretation of geospatial data across various terrains, object types, or demographics. We worked with domain experts to ensure that the model's performance is accurate and fair, minimizing biases that could affect decision-making in areas like land use planning or environmental protection.
135
-
136
  ### Community
137
 
138
  Generative AI safety requires ongoing expertise and collaboration, and we believe in the strength of the open community to accelerate progress in this area. We encourage contributions and collaboration within open consortiums focused on AI safety and responsible model deployment. We also engage with communities through relevant standards, including the MLCommons safety evaluation framework.
 
27
  - **Training data**: we introduce the Geospatial Reasoning Segmentation Dataset (GRES), a collection of vision and language data designed around
28
  remote-sensing applications. GRES consists of two core components: PreGRES, a dataset consisting of over 1M remote-sensing specific visual instruction-tuning Q/A pairs for pre-training geospatial models, and GRES, a semi-synthetic dataset specialized for reasoning segmentation of remote-sensing data and consisting of 9,205 images and 27,615 natural language queries/answers within those images. From this LISAt dataset, we generate train, test, and validation splits consisting of 7,205, 1,500, and 500 images respectively.
29
 
30
+ - **Implementation Details**: LISAT and LISATPRE are trained on eight DGX A100 80GB GPUs. In the first stage, we pretrain LISATPRE (context length = 2048) using LoRA for 1 epoch on PreGRES with next-token prediction cross-entropy loss. We employ the AdamW optimizer with a learning rate of 3e−4 and a cosine-decay learning rate scheduler, setting the batch size to 2 and gradient accumulation
31
  steps to 6.
32
 
33
  In the second stage, we train LISAT using GRES, as well as two traditional natural image referring segmentation datasets, FP-Ref-COCO and ReasonSeg. LoRA is applied to LISATPRE , while the SAM decoder undergoes full fine-tuning. The learning rate is set to 3e−4, with all other configurations remaining the
 
92
  ```
93
 
94
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
95
  ### Community
96
 
97
  Generative AI safety requires ongoing expertise and collaboration, and we believe in the strength of the open community to accelerate progress in this area. We encourage contributions and collaboration within open consortiums focused on AI safety and responsible model deployment. We also engage with communities through relevant standards, including the MLCommons safety evaluation framework.