EarthJev: typed answers about multispectral Sentinel-2 patches, without generating text

Domenico Benfenati, Antonio Maria Rinaldi, Cristian Tommasino — Department of Electrical Engineering and Information Technology (DIETI), University of Naples Federico II. Code: https://github.com/CosmoIknosLab/EarthJev · Paper: arXiv:2611.02096.

EarthJev answers typed questions about a Sentinel-2 L2A patch from all 12 bands. Each question declares its options (yes/no, one of K options, an ordered level), and the model returns a probability for every option, read from the logits of the option letters at a single position: no token is generated, so every answer is a valid option, and questions about the same patch share one encoding of the image.

• Backbone: Qwen3.5-0.8B, fully fine-tuned with the open OneJev recipe (cross-entropy + Brier on the option letters, options permuted, one epoch, learning rate 5e-6).
• 12-band input: the input layer of the vision encoder is widened from 3 to 12 channels (R, G, B kernels copied onto B04, B03, B02; the other bands start at zero) and is the only trained part of the encoder.
• This checkpoint: seed 0 of three, chosen on validation accuracy (85.63% on 36,698 validation questions). Temperatures per question type and number of options, fitted on the validation questions (calibration.json): choice:15 1.027, choice:2 0.991, choice:4 1.136, choice:5 1.025, noul:2 1.024.

Results

All models trained on the same 70,890 CORINE questions from 5% of BigEarthNet v2.0 (three seeds) and timed in one job on one H100 PCIe GPU (batch 1, bfloat16; each timer runs from the decoded image in memory to the answer). CORINE: 36,028 test questions (mean ± std over seeds), ECE = mean of the per-seed ECEs (15 bins). RSVQAxBEN: 29,501 presence questions written by other authors, never used for training. ms / image: all the questions of an image (6.1 on average) in one request.

Model	Params	Input	CORINE acc. (%)	ECE	RSVQAxBEN acc. (%)	ms / question	Slower than EarthJev	ms / image	Slower than EarthJev
EarthJev (this model)	0.8B	12 bands	86.0 ± 0.1	0.006	81.0	19.9	—	42.8	—
Qwen3.5-0.8B, generative	0.8B	12 bands	86.8 ± 0.2	0.005	80.5	134.7	6.8×	191.9	4.5×
Qwen3-VL-2B, LoRA	2.1B	RGB	86.6 ± 0.1	0.005	80.6	111.6	5.6×	176.2	4.1×
InternVL3-2B, LoRA	2.1B	RGB	87.1 ± 0.0	0.006	81.2	84.5	4.3×	157.3	3.7×
EarthDial-4B, LoRA	4.2B	RGB	89.5 ± 0.1	0.006	82.4	171.8	8.7×	431.6	10.1×
ResNet-50 VQA*	0.14B	12 bands	87.5 ± 0.2	0.089	79.4	12.1	0.6×	14.9	0.3×
Swin-T VQA*	0.15B	12 bands	86.5 ± 0.1	0.047	79.6	16.2	0.8×	17.9	0.4×
MambaVision-T VQA*	0.15B	12 bands	85.9 ± 0.3	0.064	78.4	15.3	0.8×	18.6	0.4×

* Fixed-vocabulary VQA models: a reference for the cost of a model that answers only the questions and options it was trained on. Replacing the nine non-RGB bands with their training mean costs EarthJev 4.5 points on CORINE.

Usage

Requirements: transformers with Qwen3.5 support, torch, and OneJev's engine (pip install "qev @ git+https://github.com/OmniJev/OneJev"). From a clone of this repository:

import numpy as np
from earthjev_infer import EarthJev

model = EarthJev.load(".")                       # this folder (downloaded with huggingface_hub.snapshot_download)
s2 = np.load("examples/S2A_MSIL2A_20170701T093031_N9999_R136_T35VPK_37_55.npy")   # [12, 120, 120] uint16
answers = model.predict(s2, {
    "dominant": {"type": "choice", "instructions": "Which land-cover class covers most of this patch?",
                 "criteria": {"artificial": "Artificial surfaces.", "agricultural": "Agricultural areas.",
                              "forest": "Forest and seminatural areas.", "wetlands": "Wetlands.",
                              "water": "Water bodies."}},
    "urban": {"type": "noul", "instructions": "Is there any urban fabric in this patch?"},
})
print(answers["dominant"])   # a probability for every option

The input is an array [12, H, W] with the Sentinel-2 L2A bands B01 B02 B03 B04 B05 B06 B07 B08 B8A B09 B11 B12 at 10 m (20 m and 60 m bands resampled by pixel repetition), digital numbers as distributed (reflectance × 10,000), as in BigEarthNet v2.0. examples/ holds two test patches (7 questions, 7 questions) with their CORINE test questions and an RGB preview.

Intended use and limitations

• Research on question answering and decision models for remote sensing. Not validated for operational use.
• Trained on 11,893 patches of 120 × 120 pixels from ten European countries (2017–2018) and on questions generated from CORINE Land Cover (minimum mapping unit 25 ha); accuracy outside this distribution, on other sizes, products (e.g. L1C) or question types is not established. On BigEarthNet.txt questions, whose types differ from the training questions, accuracy is about 60% on binary and 40% on four-option questions.
• Only questions with declared options: open requests ("describe this image") are outside the model's interface.

Training data and licenses

Weights: Apache-2.0 (base model Qwen3.5-0.8B, Apache-2.0). Training and test data: BigEarthNet v2.0 (reBEN) and BigEarthNet.txt (CDLA-Permissive-1.0; labels derived from CORINE Land Cover, © European Union, Copernicus Land Monitoring Service); RSVQAxBEN (CC-BY-4.0) for testing. The example patches come from the reBEN test split.

Citation

@article{benfenati2026earthjev,
  title   = {{EarthJev}: Calibrated, Non-Generative Answers to Natural-Language Questions about {Sentinel-2} Land Cover},
  author  = {Benfenati, Domenico and Rinaldi, Antonio Maria and Tommasino, Cristian},
  journal = {arXiv preprint arXiv:2611.02096},
  year    = {2026},
  url     = {https://arxiv.org/abs/2611.02096}
}

This study was partially supported by the PNRR MUR project PE0000013-FAIR.