| --- |
| base_model: Qwen/Qwen3.5-9B |
| tags: [astrobridge, captioner, lora, multimodal] |
| --- |
| |
| # AstroBridge Captioner |
|
|
| n-modality (image + spectra) astronomy captioner. LoRA adapter + fusion stack trained on top of |
| a frozen `Qwen/Qwen3.5-9B`. The base model itself is NOT included here — load it fresh from |
| `Qwen/Qwen3.5-9B` and apply this adapter on top. |
|
|
| ## How to load |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| import torch |
| |
| base = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen3.5-9B", dtype=torch.bfloat16, trust_remote_code=True |
| ) |
| llm = PeftModel.from_pretrained(base, "UniverseTBD/astrobridge-model-v8") |
| tokenizer = AutoTokenizer.from_pretrained("UniverseTBD/astrobridge-model-v8") |
| |
| # middle.pt (fusion stack: projectors/modality_identity/qformer/adapter) needs the captioner |
| # package's FusionStack class to reload — see captioner/model/captioner.py and |
| # captioner/train/stage1.py's run_stage1 for how it's constructed and wired to the LLM. |
| ``` |
|
|
| ## Training info |
| - config_hash: 33ed49969a8fce7d |
| - quantization: None |
| - git_sha: 392fcf007fd11d12d04ffd13868d0c19dc25d190 |
| - tier_histogram: {"single": 5564} |
| |
| ## Eval (groundedness gate) |
| ```json |
| { |
| "per_modality": { |
| "image": { |
| "shuffle_test": { |
| "modality": "image", |
| "n": 39, |
| "mean_edit_distance": 246.84615384615384, |
| "null_result": false |
| }, |
| "ablation_test": { |
| "modality": "image", |
| "n": 84, |
| "fraction_caption_changed": 1.0, |
| "null_result": false |
| }, |
| "diversity_test": { |
| "modality": "image", |
| "n": 120, |
| "distinct_fraction": 1.0, |
| "top1_share": 0.008333333333333333, |
| "most_common_caption": "This object appears as a compact, smooth, and slightly elongated source oriented diagonally from the lower left toward the upper right. It features a bright, concentrated yellowish-white center that f", |
| "null_result": false |
| } |
| }, |
| "spectra": { |
| "shuffle_test": { |
| "modality": "spectra", |
| "n": 26, |
| "mean_edit_distance": 181.15384615384616, |
| "null_result": false |
| }, |
| "ablation_test": { |
| "modality": "spectra", |
| "n": 72, |
| "fraction_caption_changed": 1.0, |
| "null_result": false |
| }, |
| "diversity_test": { |
| "modality": "spectra", |
| "n": 120, |
| "distinct_fraction": 1.0, |
| "top1_share": 0.008333333333333333, |
| "most_common_caption": "This optical spectrum of a galaxy in the local universe displays a red continuum characteristic of an early-type host with a prominent 4000 \u00c5 break. Superimposed on the continuum are narrow nebular em", |
| "null_result": false |
| } |
| }, |
| "lightcurve": { |
| "shuffle_test": { |
| "modality": "lightcurve", |
| "n": 9, |
| "mean_edit_distance": 46.22222222222222, |
| "null_result": false |
| }, |
| "ablation_test": { |
| "modality": "lightcurve", |
| "n": 44, |
| "fraction_caption_changed": 1.0, |
| "null_result": false |
| }, |
| "diversity_test": { |
| "modality": "lightcurve", |
| "n": 91, |
| "distinct_fraction": 1.0, |
| "top1_share": 0.01098901098901099, |
| "most_common_caption": "The optical light curve spans approximately 170 days in the $g$ and $r$ bands, with the brightest sampled detections occurring near day 10.0 ($g = 17.19 \\pm 0.07$ mag, $r = 17.12 \\", |
| "null_result": false |
| } |
| } |
| } |
| } |
| ``` |
| |
