library_name: open_clip
base_model: apple/DFN2B-CLIP-ViT-L-14
tags:
- vision-language
- image-text-retrieval
- planetary-science
- remote-sensing
- mars
- geospatial
- open-clip
MarsScope-v1
MarsScope-v1 is a planetary vision-language model and aligns orbital imagery and scientific language in a shared embedding space, supporting text-to-image, image-to-image, and multimodal retrieval for label-free exploration of planetary surfaces. It is based on ViT-L-14-quickgelu, initialized from CLIP-DFN2B, and fine-tuned at 512 x 512 resolution on more than 200,000 curated pairs of planetary surface images and geomorphological descriptions.
Model Details
- Architecture:
ViT-L-14-quickgelu - Framework: OpenCLIP
- Initialization: CLIP-DFN2B
- Input resolution: 512 x 512
- Parameters: approximately 0.4B
- Training objective: contrastive image-text alignment
Applications
The MarScope framework was introduced in Natural Language-Driven Global Mapping of Martian Landforms for natural language-driven global mapping, process-oriented geomorphological retrieval, and visual search for rare or previously unmapped features.
It was subsequently evaluated in MarsRetrieval, which covers paired Martian image-text retrieval, fine-grained landform retrieval, and planetary-scale geo-localization.
Usage
For large-scale retrieval, encode and normalize the image and text queries separately, then rank images by cosine similarity. To illustrate the effect of planetary-domain fine-tuning, we compare the image-text cosine similarities produced by the CLIP-DFN2B base model and MarsScope-v1:
| Central peak crater | Yardangs |
|---|---|
 |
 |
pip install torch open_clip_torch pillow huggingface_hub
import open_clip
import torch
from huggingface_hub import hf_hub_download
from PIL import Image
repo_id = "claytonwang/MarsScope-v1"
model_id = f"hf-hub:{repo_id}"
device = "cuda" if torch.cuda.is_available() else "cpu"
model, _, preprocess = open_clip.create_model_and_transforms(model_id)
tokenizer = open_clip.get_tokenizer(model_id)
model = model.to(device).eval()
image_names = [
"central_peak_crater.png",
"yardangs.png",
]
text_prompts = [
"a satellite image of a central peak crater on Mars",
"a satellite image of yardangs on Mars",
]
image_paths = [
hf_hub_download(repo_id=repo_id, filename=name)
for name in image_names
]
images = torch.stack([
preprocess(Image.open(path).convert("RGB"))
for path in image_paths
]).to(device)
text = tokenizer(text_prompts).to(device)
with torch.inference_mode():
image_features = model.encode_image(images, normalize=True)
text_features = model.encode_text(text, normalize=True)
similarity = image_features @ text_features.T
# Rows: central_peak_crater.png, yardangs.png
# Columns: crater prompt, yardangs prompt
print(similarity)
Examples
CLIP-DFN2B
tensor([[0.3569, 0.2822],
[0.2729, 0.2723]])
MarsScope-v1
tensor([[0.4516, 0.0585],
[0.0963, 0.3537]])
Compared with the base CLIP-DFN2B, MarsScope-v1 shows stronger alignment between Martian landform images and their matching descriptions while more clearly separating mismatched pairs. For comprehensive evaluation, refer to MarsRetrieval.
Limitations
MarsScope-v1 is intended for planetary-science research. Retrieval results may be affected by image resolution, illumination, spatial scale, geographic coverage, and the terminology used in a query. Model outputs should be validated by domain experts before scientific interpretation.
Citation
@article{wang2026natural,
title = {Natural Language-Driven Global Mapping of Martian Landforms},
author = {Wang, Yiran and Wang, Shuoyuan and Wei, Zhaoran and Zhao, Jiannan
and Yao, Zhonghua and Xie, Zejian and Zhang, Songxin and Huang, Jun
and Jing, Bingyi and Wei, Hongxin},
journal = {arXiv preprint arXiv:2601.15949},
year = {2026}
}
@article{wang2026marsretrieval,
title = {MarsRetrieval: Benchmarking Vision-Language Models for
Planetary-Scale Geospatial Retrieval on Mars},
author = {Wang, Shuoyuan and Wang, Yiran and Wei, Hongxin},
journal = {arXiv preprint arXiv:2602.13961},
year = {2026}
}