Visual Question Answering
Transformers
Safetensors
baby_medgemma
feature-extraction
medical
chest-x-ray
vision-language-model
interpretability
mechanistic-interpretability
research
custom_code
Instructions to use saillab/FlipLens with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use saillab/FlipLens with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "visual-question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("visual-question-answering", model="saillab/FlipLens", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("saillab/FlipLens", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Card: explicit scope note + link to what the experiments prove
Browse files
README.md
CHANGED
|
@@ -21,12 +21,20 @@ pretty_name: baby-MedGemma
|
|
| 21 |
A small, **MedGemma-faithful probe** for causally isolating *where paraphrase
|
| 22 |
sensitivity in medical vision-language models originates*: a frozen
|
| 23 |
`google/medsiglip-448` encoder with a small **Gemma-3 decoder** (rotary position
|
| 24 |
-
embeddings, RMSNorm, grouped-query attention: 6 query / 2 key-value heads),
|
| 25 |
-
|
| 26 |
|
| 27 |
- **Code, docs, and results:** https://github.com/thedatasense/babymedgemma
|
|
|
|
| 28 |
- **Interactive write-up:** https://bineshkumar.me/phd-thesis/causality/
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
## Load and run
|
| 31 |
|
| 32 |
```python
|
|
@@ -67,13 +75,8 @@ narrow regime: Mann-Whitney U p = 1.4e-6, Cliff's delta = 1.00). A rank-1
|
|
| 67 |
difference-direction patch restores flipped answers at the early layers with
|
| 68 |
clean controls, so the flip is a low-rank, language-side, readout-stage
|
| 69 |
direction. The origin is the training data; the fix is paraphrase augmentation.
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
Research artifact for interpretability of paraphrase sensitivity in medical VLMs.
|
| 74 |
-
It is a controlled probe, not the deployed MedGemma-4B, trained on ~1,800
|
| 75 |
-
questions over a small word-level vocabulary; it grounds only weakly and
|
| 76 |
-
manipulates adaptation-stage data. **Not a medical device; not for clinical use.**
|
| 77 |
|
| 78 |
## Training data and license
|
| 79 |
|
|
|
|
| 21 |
A small, **MedGemma-faithful probe** for causally isolating *where paraphrase
|
| 22 |
sensitivity in medical vision-language models originates*: a frozen
|
| 23 |
`google/medsiglip-448` encoder with a small **Gemma-3 decoder** (rotary position
|
| 24 |
+
embeddings, RMSNorm, grouped-query attention: 6 query / 2 key-value heads), image
|
| 25 |
+
tokens prepended inline. Loadable directly with the `transformers` API.
|
| 26 |
|
| 27 |
- **Code, docs, and results:** https://github.com/thedatasense/babymedgemma
|
| 28 |
+
- **What the experiments prove (a plain read):** https://github.com/thedatasense/babymedgemma/blob/main/docs/EXPERIMENTS.md
|
| 29 |
- **Interactive write-up:** https://bineshkumar.me/phd-thesis/causality/
|
| 30 |
|
| 31 |
+
> **Scope.** This is a controlled research probe, **not a general medical VQA
|
| 32 |
+
> model**. It has a small domain-specific vocabulary (733 words from the
|
| 33 |
+
> chest-X-ray presence questions) and grounds weakly, so it is loadable and
|
| 34 |
+
> runnable for inspection and reproduction only. It supports a *sufficiency*
|
| 35 |
+
> claim about the cause of paraphrase sensitivity, not proof that the deployed
|
| 36 |
+
> MedGemma-4B has this exact origin. Not a medical device; not for clinical use.
|
| 37 |
+
|
| 38 |
## Load and run
|
| 39 |
|
| 40 |
```python
|
|
|
|
| 75 |
difference-direction patch restores flipped answers at the early layers with
|
| 76 |
clean controls, so the flip is a low-rank, language-side, readout-stage
|
| 77 |
direction. The origin is the training data; the fix is paraphrase augmentation.
|
| 78 |
+
See [`docs/EXPERIMENTS.md`](https://github.com/thedatasense/babymedgemma/blob/main/docs/EXPERIMENTS.md)
|
| 79 |
+
for what each experiment does and does not prove.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
## Training data and license
|
| 82 |
|