File size: 3,509 Bytes
464d98e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | ---
base_model: Qwen/Qwen3.5-9B
tags: [astrobridge, captioner, lora, multimodal]
---
# AstroBridge Captioner
n-modality (image + spectra) astronomy captioner. LoRA adapter + fusion stack trained on top of
a frozen `Qwen/Qwen3.5-9B`. The base model itself is NOT included here — load it fresh from
`Qwen/Qwen3.5-9B` and apply this adapter on top.
## How to load
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.5-9B", dtype=torch.bfloat16, trust_remote_code=True
)
llm = PeftModel.from_pretrained(base, "UniverseTBD/astrobridge-model-v8")
tokenizer = AutoTokenizer.from_pretrained("UniverseTBD/astrobridge-model-v8")
# middle.pt (fusion stack: projectors/modality_identity/qformer/adapter) needs the captioner
# package's FusionStack class to reload — see captioner/model/captioner.py and
# captioner/train/stage1.py's run_stage1 for how it's constructed and wired to the LLM.
```
## Training info
- config_hash: 33ed49969a8fce7d
- quantization: None
- git_sha: 392fcf007fd11d12d04ffd13868d0c19dc25d190
- tier_histogram: {"single": 5564}
## Eval (groundedness gate)
```json
{
"per_modality": {
"image": {
"shuffle_test": {
"modality": "image",
"n": 39,
"mean_edit_distance": 246.84615384615384,
"null_result": false
},
"ablation_test": {
"modality": "image",
"n": 84,
"fraction_caption_changed": 1.0,
"null_result": false
},
"diversity_test": {
"modality": "image",
"n": 120,
"distinct_fraction": 1.0,
"top1_share": 0.008333333333333333,
"most_common_caption": "This object appears as a compact, smooth, and slightly elongated source oriented diagonally from the lower left toward the upper right. It features a bright, concentrated yellowish-white center that f",
"null_result": false
}
},
"spectra": {
"shuffle_test": {
"modality": "spectra",
"n": 26,
"mean_edit_distance": 181.15384615384616,
"null_result": false
},
"ablation_test": {
"modality": "spectra",
"n": 72,
"fraction_caption_changed": 1.0,
"null_result": false
},
"diversity_test": {
"modality": "spectra",
"n": 120,
"distinct_fraction": 1.0,
"top1_share": 0.008333333333333333,
"most_common_caption": "This optical spectrum of a galaxy in the local universe displays a red continuum characteristic of an early-type host with a prominent 4000 \u00c5 break. Superimposed on the continuum are narrow nebular em",
"null_result": false
}
},
"lightcurve": {
"shuffle_test": {
"modality": "lightcurve",
"n": 9,
"mean_edit_distance": 46.22222222222222,
"null_result": false
},
"ablation_test": {
"modality": "lightcurve",
"n": 44,
"fraction_caption_changed": 1.0,
"null_result": false
},
"diversity_test": {
"modality": "lightcurve",
"n": 91,
"distinct_fraction": 1.0,
"top1_share": 0.01098901098901099,
"most_common_caption": "The optical light curve spans approximately 170 days in the $g$ and $r$ bands, with the brightest sampled detections occurring near day 10.0 ($g = 17.19 \\pm 0.07$ mag, $r = 17.12 \\",
"null_result": false
}
}
}
}
```
|