feat(model): match frozen encoder dtype to LLM compute precision
convitom+1: Claude Opus 4.7
2cd8b5a
Image encoder weights were loaded at FP32 (~340MB) while compute ran in
BF16 under autocast, forcing a cast every forward and wasting VRAM. Now
BioViLTEncoder accepts a `dtype` arg and CXRVisionLanguageModel passes
`llm_dtype` (resolved once, shared with the LLM) so encoder weights live
at BF16/FP16 directly. Also adds 3 GPU profile presets (L4/3090 default,
T4, A100) in the Colab notebook so switching hardware is a comment swap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
chore: untrack __pycache__ bytecode (already in .gitignore)
convitom+1: Claude Opus 4.7
8b5bef1
The 5 *.pyc files were tracked before .gitignore got the __pycache__/
rule, so git kept following them. Untrack (keep local copies); the
existing ignore rule prevents them coming back.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
feat(scripts): offline dataset resize + tar-shard tooling for cloud training
convitom+1: Claude Opus 4.7
47a30f7
Add build_resized_dataset.py and resize_and_shard.ipynb to pre-shrink the
MIMIC-CXR tree to RAD-DINO's 518px working resolution and pack it into tar
shards. Avoids re-decoding ~100GB of full-res JPGs every epoch (encoder is
only ~1-2% of compute; the win is removing decode + transfer cost, not a
feature cache which would be larger and inflexible). Shorter-edge>=518
keying matches the processor exactly so features are unchanged; --square
optionally bakes the center-crop. Notebook reads/writes the HF dataset
repo (parallel + resumable), no change to dataset.py/cxr_vlm.py.
Also fixes leftover rad_dino.py docstring header from the earlier rename.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The file rename landed in 215ecd6 but the importer updates in
model/__init__.py, model/cxr_vlm.py, evaluation/{evaluate,inference}.py
and training/train.py were left uncommitted, so a clean checkout of HEAD
raised ImportError (from .image_encoder import …, file no longer exists).
Commit only those import-line fixes (6 lines) to make HEAD importable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- chexpert_classifier.py: 14 binary heads → 14×3 softmax (negative/
positive/uncertain per pathology, META-CXR / CheXpert U-MultiClass).
Add format_pnu/buckets_to_pnu as the single source of truth for the
PNU 3-section prompt string (shared with the oracle builder so GT and
predicted prompts are byte-identical).
- mimic_cxr_builder.py: GT chexpert.csv → PNU string (1→pos, 0→neg,
-1→uncertain, blank/NaN→neg). VQA now carries the SAME PNU context
(abnormality-guided VQA, RaDialog-style). O(1) image lookup (was
O(N²)); index every image so report-less studies still serve VQA.
- Drop the obsolete uncertain_policy knob (U-MultiClass is the only
behaviour now) from builder, dataset.py, dataset_resolver, config.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
