AstroPRISM: Joint AION–Qwen3-4B alignment checkpoint

This private research checkpoint is a single AstroPRISM model jointly aligned on two caption streams:

• calibrated four-band Legacy Survey images;
• observed-frame DESI spectra.

Both streams trained the same checkpoint with shared AION and Qwen components. Each example contained one non-text modality; simultaneous image-plus-spectrum prompting was not trained or evaluated in this run.

Architecture

• Language model: Qwen3-4B, frozen during alignment.
• Astronomy encoder: AION Base, frozen during alignment and shared by both routes.
• Image input: float32[4,160,160] in DES-G, DES-R, DES-I, DES-Z band order.
• Spectrum input: flux, ivar, and wavelength as float32[7781], plus mask as bool[7781].
• AION output: 576 image tokens or 273 spectrum tokens, width 768.
• Separate two-layer projectors map each modality from width 768 to Qwen width 2560.
• Trainable parameters: 17,059,840 (image projector 8,527,360; spectrum projector 8,532,480).

The checkpoint contains 1,974 tensors and 4,452,498,917 tensor elements, including the frozen Qwen and shared AION tensors plus both projectors. The checkpoint SHA-256 is recorded in SHA256SUMS and provenance/model_manifest.json.

Pinned bases

• Qwen3-4B: Qwen/Qwen3-4B, revision 1cfa9a7208912126459214e8b04321603b3df60c.
• AION Base and codecs: polymathic-ai/aion-base, revision 40541618104bab0fa85c8af68daeb867a720bb8c.

See THIRD_PARTY_NOTICES.md for upstream licensing information. No license for AstroPRISM's original contributions or training data is asserted by this private release.

Training data

Training used the private Sand33p/AstroPRISM dataset. Each modality contains 646 candidate rows with a source-group-safe split:

Split	Image-caption	Spectrum-caption
Train	516	516
Validation	65	65
Test	65	65

The roster contains 537 source groups, so the 646 candidate rows are not guaranteed to represent 646 independent physical sources.

Alignment run

• Steps: 200
• Batch size: 2 per rank
• Hardware: 12 Aurora XPU ranks
• Precision: BF16 mixed precision
• Optimizer schedule: cosine, learning rate 1e-3, 10 warmup steps, no weight decay
• Seed: 42
• Frozen components: Qwen3-4B and AION Base
• Optimized components: image and spectrum projectors

The sanitized run configuration is in config/training_config.yaml; training/training_state.json records final checkpoint state. Optimizer and per-rank RNG files are intentionally excluded because this release targets inference rather than exact optimizer-state resumption.

Evaluation

Token-weighted, teacher-forced causal-LM losses on the held-out test split:

Stream	Test loss
Combined	1.4708027309
Image-caption	1.5210461083
Spectrum-caption	1.3005556689

These losses measure target-token prediction, not scientific correctness. Machine-readable values are in metrics/loss_metrics.json.

Held-out generations

Greedy generations for one held-out image and one held-out spectrum are in:

• examples/README.md — human-readable prompts and outputs;
• examples/test_image43_spectrum48.json — complete prompts, token IDs, references, provenance, timing, and stopping reasons.

The reference captions were not supplied during generation. Both uploaded examples reached EOS.

Loading and inference

This is a custom PRISM checkpoint and is not directly loadable with AutoModelForCausalLM.from_pretrained. Instantiate PRISM's UnifiedTransformer, construct the pinned Qwen and AION components, and then load model.safetensors using the PRISM checkpoint loader. Raw-input preprocessing and modality routing require the corresponding BaseMM_PRISM implementation and AION codecs.

inference/generate_astroprism_joint_examples.py is the exact runner used for the uploaded qualitative examples. The uploaded configuration uses Hub IDs and sanitized paths; download the private dataset locally and map its manifests into the PRISM data configuration before running it.

Source provenance is based on branch AstroPrism at Git revision 993d7f3a8217e93202a604f198de502b2ca234e0. The training implementation also contained uncommitted AstroPRISM worktree changes, so that revision alone is not a complete source snapshot.

Limitations

• This is a 646-candidate pilot trained on synthetic or weak-supervision captions.
• The held-out captions are not an independently expert-adjudicated benchmark.
• Qualitative outputs can invent morphology, emission lines, redshifts, or unsupported physical explanations.
• The uploaded image example calls the object lenticular while its reference describes an elliptical morphology.
• The uploaded spectrum example asserts [O III] while its reference emphasizes a possible 4000 Å break.
• Simultaneous image-plus-spectrum input was not evaluated.
• Do not use this model as a substitute for calibrated measurement or expert astronomical analysis.