Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,47 @@
 *.7z filter=lfs diff=lfs merge=lfs -text
 *.arrow filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
 *.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
 *.ftz filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text
 *.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
 *.model filter=lfs diff=lfs merge=lfs -text
 *.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
 *.onnx filter=lfs diff=lfs merge=lfs -text
 *.ot filter=lfs diff=lfs merge=lfs -text
 *.parquet filter=lfs diff=lfs merge=lfs -text
 *.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
 *.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
 saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
 *.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,270 @@
+---
+frameworks:
+- ""
+language:
+- en
+license: apache-2.0
+tags:
+- OneScience
+- medical multimodal large language model
+- interactive inference
+---
+<p align="center">
+  <strong>
+    <span style="font-size: 30px;">MedGemma</span>
+  </strong>
+</p>
+
+# Model Overview
+
+MedGemma is an open-source medical multimodal large language model from Google. It is based on the [Gemma 3](https://ai.google.dev/gemma/docs/core) architecture and is trained for medical text and medical image understanding. MedGemma provides two variants:
+
+- **MedGemma 4B**: a multimodal model that supports joint input of medical text and medical images.
+- **MedGemma 27B**: a text-only model focused on medical text understanding and question answering.
+
+# Model Description
+
+MedGemma 4B uses an image encoder and has been pretrained on multiple types of de-identified medical data, including chest X-rays (CXR), dermatology images, ophthalmology images, and histopathology slides. Its language model component has been trained on radiology images, pathology images, ophthalmology images, dermatology images, and medical text.
+
+Weights and datasets are not available at the moment. They will be uploaded to Hugging Face soon, and command-line downloads will be supported later.
+
+# Use Cases
+
+| Use case | Description |
+| :---: | :---: |
+| Medical question answering | Evaluates the model's question-answering capability on medical knowledge benchmarks such as MedQA. |
+| Medical image analysis | Supports tasks such as anatomical localization on chest X-rays (CXR) and longitudinal comparison of multi-timepoint images. |
+| Domain fine-tuning | Performs parameter-efficient fine-tuning with LoRA on datasets such as NCT colon histopathology images. |
+| Unified inference | Provides interactive and batch-file inference through `MedicalInferenceRunner`. |
+
+# Use and Limitations
+
+MedGemma is a foundation model for developing healthcare AI applications and is intended to serve as a starting point for downstream development, adaptation, and validation. Developers should fully validate, adapt, and meaningfully modify the model for their specific use case.
+
+Outputs generated by MedGemma are not intended to directly inform clinical diagnosis, patient management decisions, treatment recommendations, or any other direct clinical practice. All model outputs should be considered preliminary and require independent verification, clinical correlation, and further investigation.
+
+# Usage
+
+## 1. Using OneCode
+
+You can try intelligent one-click AI4S programming through the OneCode online environment:
+
+[Try intelligent one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
+
+## 2. Manual Installation and Usage
+
+**Hardware Requirements**
+
+- Running on a GPU or DCU is recommended.
+- CPU can be used for connectivity checks, but it is relatively slow.
+- DCU users need to install DTK in advance. DTK 25.04.2 or later is recommended, or the OneScience-recommended version that matches the current cluster.
+
+**Software Requirements**
+
+DCU users who want to learn more about adaptation details can contact liubiao@sugon.com.
+
+**Environment Checks**
+
+- NVIDIA GPU:
+
+```bash
+nvidia-smi
+```
+
+- Hygon DCU:
+
+```bash
+hy-smi
+```
+
+### Environment Preparation
+
+1. Check the `botocore` version. If the version is too old, upgrade it:
+
+    ```bash
+    pip install --upgrade boto3==1.43.36 botocore==1.43.36
+    ```
+
+2. Check the `transformers` version. If the version is too old, upgrade it:
+
+    ```bash
+    pip install --upgrade transformers==5.12.1
+    ```
+
+## Quick Start
+
+### 1. Install the Runtime Environment
+
+```bash
+conda create -n onescience311 python=3.11 -y
+conda activate onescience311
+pip install onescience[bio] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
+```
+
+If the following code cannot find required libraries at runtime, activate CUDA as shown below.
+
+```bash
+source ${ROCM_PATH}/cuda/env.sh
+export LD_LIBRARY_PATH="$CONDA_PREFIX/lib:$LD_LIBRARY_PATH"
+export LD_LIBRARY_PATH="$CONDA_PREFIX/lib/python3.11/site-packages/fastpt/torch/lib:$LD_LIBRARY_PATH"
+```
+
+### 2. Download the Model Package
+
+```bash
+# By default, the package is downloaded to the model folder under the current path. To change this, adjust the path after local_dir.
+hf download --model OneScience-Sugon/Medgemma --local-dir ./model
+cd model
+```
+
+### Training Weights and Datasets
+
+Weights and datasets are not provided at the moment. They will be uploaded to Hugging Face soon, and command-line downloads will be supported later.
+
+### 3. Usage Modes
+
+**Data and Model Weights**
+
+```bash
+hf download --dataset OneScience-Sugon/medgemma --local-dir ./dataset # Weights and datasets are not available at the moment. They will be uploaded to Hugging Face soon, and command-line downloads will be supported later.
+```
+
+By default, the scripts load the model from:
+
+```text
+${ONESCIENCE_DATASETS_DIR}/medgemma/Hugging Face/google/medgemma-1.5-4b-it
+```
+
+By default, the scripts load datasets from the following paths:
+
+| Task | Data | Default path |
+|------|------|----------|
+| MedQA evaluation | MedQA parquet data | `${ONESCIENCE_DATASETS_DIR}/medgemma/medqa` |
+| Chest X-ray anatomical localization | Chest X-ray images | `${ONESCIENCE_DATASETS_DIR}/medgemma/Chest_Xray/...` |
+| Chest X-ray longitudinal comparison | Two chest X-rays from before and after | `${ONESCIENCE_DATASETS_DIR}/medgemma/test_compare/...` |
+| Pathology image fine-tuning | NCT-CRC-HE-100K / CRC-VAL-HE-7K | `${ONESCIENCE_DATASETS_DIR}/medgemma/nct/...` |
+
+Download the model in advance and place it in this directory, or override it through the `model_path` environment variable.
+
+### Detailed Usage
+
+#### 1. Integration Test
+
+Verify whether MedGemma modules, configurations, data adapters, and image processing components can be imported correctly in OneScience:
+
+```bash
+python tests/test_integration.py
+```
+
+---
+
+#### 2. Medical Question Answering Evaluation (`run_evaluate_on_medqa.sh`)
+
+Evaluate the model's medical question-answering capability on the MedQA dataset. By default, 10 samples are processed for quick validation.
+
+```bash
+bash scripts/run_evaluate_on_medqa.sh
+```
+
+Output:
+
+- `scripts/medqa_results/medqa_results.json`: detailed results for each sample
+- `scripts/medqa_results/summary.txt`: summary metrics such as accuracy
+
+---
+
+#### 3. Chest X-Ray Anatomical Localization (`run_cxr_anatomy.sh`)
+
+Perform anatomical localization on one or more chest X-rays. The script runs both single-image mode and batch mode internally:
+
+```bash
+bash scripts/run_cxr_anatomy.sh
+```
+
+Output:
+
+- `scripts/outputs/result_*.json`: localization coordinates and labels
+- `scripts/outputs/result_*.png`: visualization images with bounding-box annotations
+- `scripts/outputs/batch_summary.json`: summary results for batch mode
+
+---
+
+#### 4. Chest X-Ray Longitudinal Comparison (`run_cxr_longitudinal_comparison.sh`)
+
+Compare two chest X-rays from the same patient taken at different timepoints:
+
+```bash
+bash scripts/run_cxr_longitudinal_comparison.sh
+```
+
+Output:
+
+- `scripts/compare_outputs/compare_<image1>_vs_<image2>.txt`: text comparison report
+- `scripts/compare_outputs/compare_<image1>_vs_<image2>.json`: structured JSON results
+
+---
+
+#### 5. Pathology Image LoRA Fine-Tuning (`run_fine_tune.sh`)
+
+Perform LoRA fine-tuning on the NCT colon histopathology image dataset:
+
+```bash
+bash scripts/run_fine_tune.sh
+```
+
+Output:
+
+- `scripts/medgemma-nct-lora/`: LoRA weights, training logs, and evaluation results
+
+> Note: The script automatically checks and fixes the `boto3==1.43.36` and `botocore==1.43.36` versions to avoid dependency conflicts.
+
+---
+
+#### 6. Using the Inference Runner
+
+`runner/medical_inference_runner.py` provides a unified inference entry point and supports both interactive and batch-file inference.
+
+##### Interactive Inference
+
+```bash
+export PYTHONPATH=../../../src:$PYTHONPATH
+python runner/medical_inference_runner.py \
+    --config configs/inference_config.yaml \
+    --interactive
+```
+
+##### Batch-File Inference
+
+```bash
+export PYTHONPATH=../../../src:$PYTHONPATH
+python runner/medical_inference_runner.py \
+    --config configs/inference_config.yaml \
+    --input data/example_input.json
+```
+
+### Notes
+
+- Make sure the `ONESCIENCE_DATASETS_DIR` environment variable is correctly set before running the scripts.
+- The scripts use `HIP_VISIBLE_DEVICES=0` by default and can run directly on Hygon DCU platforms. On CUDA platforms, replace it with `CUDA_VISIBLE_DEVICES=0` or adjust it according to the available devices.
+- To use vLLM for accelerated inference, make sure the corresponding version of vLLM is installed and configure `use_vllm: true`.
+- The chest X-ray anatomical localization and pathology fine-tuning scripts automatically fix `boto3` / `botocore` versions to avoid dependency conflicts.
+- The 4B multimodal model requires substantial GPU memory for inference. At least 24 GB of memory on a single device is recommended. Multi-device execution can be controlled through `num_gpus` or an external `CUDA_VISIBLE_DEVICES` setting.
+
+# Official OneScience Information
+
+| Platform | OneScience main repository | Skills repository |
+| --- | --- | --- |
+| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
+| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
+
+# Citation and License
+
+The MedGemma model is licensed under the [Health AI Developer Foundations License](https://developers.google.com/health-ai-developer-foundations/terms), and the example code in this repository is licensed under Apache 2.0.
+
+For more information, see:
+
+- [Developer documentation](https://developers.google.com/health-ai-developer-foundations/medgemma/get-started)
+- [Model card](https://developers.google.com/health-ai-developer-foundations/medgemma/model-card)
+- [Community guidelines](https://developers.google.com/health-ai-developer-foundations/community-guidelines)
+- [Hugging Face](https://huggingface.co/models?other=medgemma)
+- [Google Model Garden](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/medgemma)

configs/__init__.py

@@ -0,0 +1,3 @@
+from .configs_base import medgemma_base_configs
+
+__all__ = ["medgemma_base_configs"]

configs/configs_base.py

@@ -0,0 +1,73 @@
+# MedGemma 基础配置
+# 遵循 Protenix 的配置模式
+
+from onescience.models.protenix.config.extend_types import (
+    DefaultNoneWithType,
+    GlobalConfigValue,
+    ListValue,
+    RequiredValue,
+    ValueMaybeNone,
+)
+
+basic_configs = {
+    "project": "MedGemma",
+    "run_name": RequiredValue(str),
+    "base_dir": RequiredValue(str),
+    "seed": 42,
+    "deterministic": False,
+    "use_wandb": False,
+    "load_checkpoint_path": "",
+    "eval_only": True,  # MedGemma 主要用于推理
+}
+
+model_configs = {
+    # Model settings
+    "variant": RequiredValue(str),  # "4b" or "27b"
+    "model_path": RequiredValue(str),  # 模型权重路径
+    "tokenizer_path": DefaultNoneWithType(str),  # Tokenizer 路径，默认与 model_path 相同
+    "prompt_format": "chat",  # "chat" or "instruct"
+    "is_multimodal": True,  # 4B 支持多模态，27B 仅文本
+}
+
+inference_configs = {
+    # Inference settings
+    "gpu_memory_utilization": 0.9,
+    "max_model_len": DefaultNoneWithType(int),  # 最大序列长度，None 表示使用模型默认值
+    "tensor_parallel_size": 1,  # Tensor 并行大小
+    "default_max_tokens": 500,  # 默认生成最大 token 数
+    "temperature": 0.7,
+    "top_p": 0.9,
+    "top_k": DefaultNoneWithType(int),
+    "min_p": DefaultNoneWithType(float),
+    "batch_size": 1,
+    "num_workers": 0,
+    "use_vllm": True,  # 是否使用 vLLM（如果为 False，则使用 transformers）
+}
+
+data_configs = {
+    # Data settings
+    "input_json_path": DefaultNoneWithType(str),  # 输入 JSON 文件路径
+    "input_dir": DefaultNoneWithType(str),  # 输入目录
+    "image_input_width": 224,
+    "image_input_height": 224,
+    "max_parallel_download_workers": 4,
+    "worker_download_parallelism": "THREAD",  # "THREAD" or "PROCESS"
+    "use_msa": False,  # MedGemma 不使用 MSA（这是 Protenix 特有的）
+}
+
+output_configs = {
+    # Output settings
+    "dump_dir": RequiredValue(str),
+    "save_predictions": True,
+    "output_format": "json",  # "json" or "jsonl"
+    "save_intermediate": False,
+}
+
+# 合并所有配置
+medgemma_base_configs = {
+    **basic_configs,
+    "model": model_configs,
+    "inference": inference_configs,
+    "data": data_configs,
+    "output": output_configs,
+}

configs/inference_config.yaml

@@ -0,0 +1,52 @@
+# MedGemma 推理配置示例
+# 此文件展示如何配置 MedGemma 进行推理
+
+# 项目设置
+project: "MedGemma"
+run_name: "medgemma_inference_demo"
+base_dir: "./medgemma_output"
+
+# 模型设置
+model:
+  variant: "4b"  # "4b" (多模态) 或 "27b" (仅文本)
+  model_path: "/public/share/sugonhpcapp01/onestore/onedatasets/medgemma/modelscope/google/medgemma-1.5-4b-it"  # 替换为实际模型路径
+  tokenizer_path: null  # null 表示使用 model_path
+  prompt_format: "chat"  # "chat" 或 "instruct"
+  is_multimodal: true  # 4B 支持多模态
+
+# 推理设置
+inference:
+  gpu_memory_utilization: 0.9
+  max_model_len: null  # null 表示使用模型默认值
+  tensor_parallel_size: 1  # GPU 并行数
+  default_max_tokens: 500
+  temperature: 0.7
+  top_p: 0.9
+  top_k: null
+  min_p: null
+  batch_size: 1
+  num_workers: 0
+  use_vllm: false  # 使用 vLLM 加速
+
+# 数据设置
+data:
+  input_json_path: "./input_data.json"  # 输入数据路径
+  input_dir: null
+  image_input_width: 224
+  image_input_height: 224
+  max_parallel_download_workers: 4
+  worker_download_parallelism: "THREAD"
+  use_msa: false
+
+# 输出设置
+output:
+  dump_dir: "./medgemma_predictions"
+  save_predictions: true
+  output_format: "json"  # "json" 或 "jsonl"
+  save_intermediate: false
+
+# 基础设置
+seed: 42
+deterministic: false
+use_wandb: false
+eval_only: true

models/__init__.py

@@ -0,0 +1,22 @@
+from .medgemma import MedGemma
+from .model_runner import VLLMModelRunner, TransformersModelRunner
+from .predictor_wrapper import MedGemmaPredictor
+from .config import (
+    ConfigManager,
+    parse_configs,
+    load_config,
+    save_config,
+)
+
+__all__ = [
+    "MedGemma",
+    "VLLMModelRunner",
+    "TransformersModelRunner",
+    "MedGemmaPredictor",
+    "ConfigManager",
+    "parse_configs",
+    "load_config",
+    "save_config",
+]
+
+__version__ = "0.1.0"

models/config/__init__.py

@@ -0,0 +1,21 @@
+from .config import (
+    ConfigManager,
+    parse_configs,
+    parse_sys_args,
+    load_config,
+    save_config,
+)
+from .config_base import (
+    MEDGEMMA_BASE_CONFIG,
+    MEDGEMMA_INFERENCE_CONFIG,
+)
+
+__all__ = [
+    "ConfigManager",
+    "parse_configs",
+    "parse_sys_args",
+    "load_config",
+    "save_config",
+    "MEDGEMMA_BASE_CONFIG",
+    "MEDGEMMA_INFERENCE_CONFIG",
+]

models/config/config.py

@@ -0,0 +1,33 @@
+# MedGemma 配置管理器
+# 重用 Protenix 的 ConfigManager
+
+from onescience.models.protenix.config.config import (
+    ConfigManager,
+    parse_configs as protenix_parse_configs,
+    parse_sys_args,
+    load_config,
+    save_config,
+)
+
+__all__ = [
+    "ConfigManager",
+    "parse_configs",
+    "parse_sys_args",
+    "load_config",
+    "save_config",
+]
+
+
+def parse_configs(base_configs: dict, sys_args=None, fill_required_with_null: bool = False):
+    """
+    解析 MedGemma 配置
+
+    Args:
+        base_configs: 基础配置字典
+        sys_args: 命令行参数（可选）
+        fill_required_with_null: 是否用 None 填充必需值
+
+    Returns:
+        ConfigDict: 解析后的配置
+    """
+    return protenix_parse_configs(base_configs, sys_args, fill_required_with_null)

models/config/config_base.py

@@ -0,0 +1,27 @@
+# MedGemma 模型配置
+# 此文件定义了 MedGemma 模型的基础配置结构
+
+from onescience.models.protenix.config.extend_types import (
+    RequiredValue,
+    ValueMaybeNone,
+)
+
+# MedGemma 基础配置
+MEDGEMMA_BASE_CONFIG = {
+    "variant": RequiredValue(str),  # "4b" or "27b"
+    "model_path": RequiredValue(str),
+    "tokenizer_path": ValueMaybeNone(str),
+    "prompt_format": "chat",
+    "is_multimodal": True,
+}
+
+# 推理配置
+MEDGEMMA_INFERENCE_CONFIG = {
+    "gpu_memory_utilization": 0.9,
+    "max_model_len": ValueMaybeNone(int),
+    "tensor_parallel_size": 1,
+    "default_max_tokens": 500,
+    "temperature": 0.7,
+    "top_p": 0.9,
+    "batch_size": 1,
+}

models/medgemma.py

@@ -0,0 +1,245 @@
+# MedGemma 主模型类
+# 继承 OneScience Module 基类，集成到 OneScience 框架
+
+import logging
+from typing import Any, Dict, List, Optional
+
+import torch
+import torch.nn as nn
+
+from onescience.modules.module import Module
+from models.model_runner import VLLMModelRunner, TransformersModelRunner
+from models.predictor_wrapper import MedGemmaPredictor
+
+logger = logging.getLogger(__name__)
+
+
+class MedGemma(Module):
+    """
+    MedGemma: 医学大语言模型
+
+    支持:
+    - 4B 多模态模型（文本 + 医学图像）
+    - 27B 文本模型
+    - DICOM/CT/CXR/WSI 图像输入
+    - OpenAI Chat Completion API 格式
+
+    继承 OneScience Module 基类，提供统一接口
+    """
+
+    def __init__(self, configs: Any) -> None:
+        """
+        初始化 MedGemma 模型
+
+        Args:
+            configs: 配置对象（ConfigDict）
+        """
+        super(MedGemma, self).__init__()
+        self.configs = configs
+
+        # 模型变体（4B 或 27B）
+        self.model_variant = configs.model.variant
+        self.is_multimodal = configs.model.is_multimodal
+
+        logger.info(f"Initializing MedGemma {self.model_variant} model")
+        logger.info(f"Multimodal: {self.is_multimodal}")
+
+        # 初始化模型运行器
+        self._init_model_runner()
+
+        # 初始化推理包装器
+        self.predictor = MedGemmaPredictor(
+            model_runner=self.model_runner,
+            configs=configs,
+        )
+
+        logger.info("MedGemma model initialized successfully")
+
+    def _init_model_runner(self):
+        """初始化模型运行器（vLLM 或 Transformers）"""
+        if self.configs.inference.use_vllm:
+            try:
+                self.model_runner = VLLMModelRunner(
+                    model_path=self.configs.model.model_path,
+                    tokenizer_path=self.configs.model.tokenizer_path,
+                    gpu_memory_utilization=self.configs.inference.gpu_memory_utilization,
+                    max_model_len=self.configs.inference.max_model_len,
+                    tensor_parallel_size=self.configs.inference.tensor_parallel_size,
+                )
+                logger.info("Using vLLM model runner")
+            except Exception as e:
+                logger.warning(f"vLLM initialization failed: {e}")
+                logger.info("Falling back to Transformers model runner")
+                self._init_transformers_runner()
+        else:
+            self._init_transformers_runner()
+
+    def _init_transformers_runner(self):
+        """初始化 Transformers 运行器"""
+        device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model_runner = TransformersModelRunner(
+            model_path=self.configs.model.model_path,
+            tokenizer_path=self.configs.model.tokenizer_path,
+            device=device,
+        )
+        logger.info("Using Transformers model runner")
+
+    def forward(
+        self,
+        messages: List[Dict[str, Any]],
+        max_tokens: Optional[int] = None,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+        n: int = 1,
+    ) -> Dict[str, Any]:
+        """
+        前向传播（使用 OpenAI Chat Completion 格式）
+
+        Args:
+            messages: 消息列表，格式为 [{"role": "user", "content": "..."}]
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度（0-2）
+            top_p: Nucleus 采样参数（0-1）
+            n: 生成数量
+
+        Returns:
+            OpenAI 格式的响应字典
+        """
+        if max_tokens is None:
+            max_tokens = self.configs.inference.default_max_tokens
+
+        return self.predictor.predict(
+            messages=messages,
+            max_tokens=max_tokens,
+            temperature=temperature,
+            top_p=top_p,
+            n=n,
+        )
+
+    @torch.no_grad()
+    def inference(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        推理方法（兼容 BiologyInferenceRunner）
+
+        Args:
+            data: 输入数据字典，可包含:
+                - messages: 消息列表
+                - instances: 实例列表（用于批处理）
+                - parameters: 推理参数
+
+        Returns:
+            预测结果字典
+        """
+        # 转换数据格式
+        if "instances" in data:
+            messages = self._convert_instances_to_messages(data["instances"])
+        elif "messages" in data:
+            messages = data["messages"]
+        else:
+            raise ValueError("Input data must contain 'messages' or 'instances'")
+
+        # 提取推理参数
+        parameters = data.get("parameters", {})
+        max_tokens = parameters.get("max_tokens", self.configs.inference.default_max_tokens)
+        temperature = parameters.get("temperature", self.configs.inference.temperature)
+        top_p = parameters.get("top_p", self.configs.inference.top_p)
+        n = parameters.get("n", 1)
+
+        # 运行推理
+        return self.forward(
+            messages=messages,
+            max_tokens=max_tokens,
+            temperature=temperature,
+            top_p=top_p,
+            n=n,
+        )
+
+    def _convert_instances_to_messages(
+        self,
+        instances: List[Dict[str, Any]]
+    ) -> List[Dict[str, Any]]:
+        """
+        将实例列表转换为消息格式
+
+        Args:
+            instances: 实例列表
+
+        Returns:
+            消息列表
+        """
+        messages = []
+        for instance in instances:
+            if "role" in instance and "content" in instance:
+                messages.append(instance)
+            elif "text" in instance:
+                messages.append({"role": "user", "content": instance["text"]})
+            elif "question" in instance:
+                messages.append({"role": "user", "content": instance["question"]})
+            else:
+                logger.warning(f"Unknown instance format: {instance}")
+
+        return messages
+
+    def predict_text(
+        self,
+        text: str,
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+    ) -> str:
+        """
+        简化的文本预测接口
+
+        Args:
+            text: 输入文本
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+
+        Returns:
+            生成的文本
+        """
+        messages = [{"role": "user", "content": text}]
+        response = self.forward(messages, max_tokens, temperature)
+
+        if response["choices"]:
+            return response["choices"][0]["message"]["content"]
+        return ""
+
+    def predict_multimodal(
+        self,
+        text: str,
+        images: List[Any],
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+    ) -> str:
+        """
+        多模态预测接口（文本 + 图像）
+
+        Args:
+            text: 输入文本
+            images: 图像列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+
+        Returns:
+            生成的文本
+        """
+        if not self.is_multimodal:
+            logger.warning("Model is not multimodal, ignoring images")
+            return self.predict_text(text, max_tokens, temperature)
+
+        # TODO: 实现多模态推理
+        # 需要集成图像编码器和多模态提示格式
+        messages = [{"role": "user", "content": text}]
+        response = self.predictor.predict_with_images(
+            messages=messages,
+            images=images,
+            max_tokens=max_tokens,
+            temperature=temperature,
+        )
+
+        if response["choices"]:
+            return response["choices"][0]["message"]["content"]
+        return ""
+
+    def __repr__(self) -> str:
+        return f"MedGemma(variant={self.model_variant}, multimodal={self.is_multimodal})"

models/model_runner.py

@@ -0,0 +1,280 @@
+# MedGemma vLLM 模型运行器
+# 实现基于 vLLM 的推理引擎
+
+import logging
+from typing import Any, Dict, List, Optional, Set, Mapping
+import numpy as np
+
+logger = logging.getLogger(__name__)
+
+
+class VLLMModelRunner:
+    """
+    基于 vLLM 的 MedGemma 模型运行器
+    兼容 MedGemma serving_framework 的 ModelRunner 接口
+    """
+
+    def __init__(
+        self,
+        model_path: str,
+        tokenizer_path: Optional[str] = None,
+        gpu_memory_utilization: float = 0.9,
+        max_model_len: Optional[int] = None,
+        tensor_parallel_size: int = 1,
+        trust_remote_code: bool = True,
+    ):
+        """
+        初始化 vLLM 模型运行器
+
+        Args:
+            model_path: 模型权重路径
+            tokenizer_path: Tokenizer 路径（默认与 model_path 相同）
+            gpu_memory_utilization: GPU 内存使用率
+            max_model_len: 最大序列长度
+            tensor_parallel_size: Tensor 并行大小
+            trust_remote_code: 是否信任远程代码
+        """
+        self.model_path = model_path
+        self.tokenizer_path = tokenizer_path or model_path
+
+        try:
+            from vllm import LLM
+            self.vllm_available = True
+        except ImportError:
+            logger.warning("vLLM not available. Install with: pip install vllm")
+            self.vllm_available = False
+            self.llm = None
+            return
+
+        logger.info(f"Initializing vLLM with model: {model_path}")
+        self.llm = LLM(
+            model=self.model_path,
+            tokenizer=self.tokenizer_path,
+            gpu_memory_utilization=gpu_memory_utilization,
+            max_model_len=max_model_len,
+            tensor_parallel_size=tensor_parallel_size,
+            trust_remote_code=trust_remote_code,
+        )
+        logger.info("vLLM model loaded successfully")
+
+    def run_model_multiple_output(
+        self,
+        model_input: Mapping[str, np.ndarray] | np.ndarray,
+        model_name: str = "default",
+        model_version: Optional[int] = None,
+        model_output_keys: Optional[Set[str]] = None,
+        parameters: Optional[Mapping[str, Any]] = None,
+    ) -> Mapping[str, np.ndarray]:
+        """
+        运行推理（兼容 MedGemma ModelRunner 接口）
+
+        Args:
+            model_input: 输入数据（字典或 numpy 数组）
+            model_name: 模型名称
+            model_version: 模型版本
+            model_output_keys: 输出键集合
+            parameters: 推理参数
+
+        Returns:
+            输出字典
+        """
+        if not self.vllm_available or self.llm is None:
+            raise RuntimeError("vLLM is not available")
+
+        from vllm import SamplingParams
+
+        parameters = parameters or {}
+        model_output_keys = model_output_keys or {"text_output"}
+
+        # 提取 prompt
+        if isinstance(model_input, dict):
+            prompt = model_input.get("prompt", "")
+            if isinstance(prompt, np.ndarray):
+                prompt = prompt.tobytes().decode('utf-8')
+        else:
+            # 如果是 numpy 数组，解码为字符串
+            prompt = model_input.tobytes().decode('utf-8')
+
+        # 创建采样参数
+        sampling_params = SamplingParams(
+            max_tokens=parameters.get("max_tokens", 500),
+            temperature=parameters.get("temperature", 0.7),
+            top_p=parameters.get("top_p", 0.9),
+            top_k=parameters.get("top_k", -1),
+            n=parameters.get("n", 1),
+        )
+
+        # 运行推理
+        outputs = self.llm.generate([prompt], sampling_params)
+
+        # 格式化输出
+        result = {}
+        if "text_output" in model_output_keys:
+            result["text_output"] = np.array([
+                output.text.encode('utf-8') for output in outputs[0].outputs
+            ])
+        if "num_input_tokens" in model_output_keys:
+            result["num_input_tokens"] = np.array([len(outputs[0].prompt_token_ids)])
+        if "num_output_tokens" in model_output_keys:
+            result["num_output_tokens"] = np.array([
+                len(output.token_ids) for output in outputs[0].outputs
+            ])
+
+        return result
+
+    def generate(
+        self,
+        prompts: List[str],
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+        top_k: int = -1,
+        n: int = 1,
+    ) -> List[Dict[str, Any]]:
+        """
+        简化的生成接口
+
+        Args:
+            prompts: 输入提示列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+            top_p: Nucleus 采样参数
+            top_k: Top-K 采样参数
+            n: 生成数量
+
+        Returns:
+            生成结果列表
+        """
+        if not self.vllm_available or self.llm is None:
+            raise RuntimeError("vLLM is not available")
+
+        from vllm import SamplingParams
+
+        sampling_params = SamplingParams(
+            max_tokens=max_tokens,
+            temperature=temperature,
+            top_p=top_p,
+            top_k=top_k,
+            n=n,
+        )
+
+        outputs = self.llm.generate(prompts, sampling_params)
+
+        results = []
+        for output in outputs:
+            result = {
+                "prompt": output.prompt,
+                "outputs": [
+                    {
+                        "text": o.text,
+                        "token_ids": o.token_ids,
+                        "cumulative_logprob": o.cumulative_logprob,
+                        "finish_reason": o.finish_reason,
+                    }
+                    for o in output.outputs
+                ],
+                "num_input_tokens": len(output.prompt_token_ids),
+            }
+            results.append(result)
+
+        return results
+
+
+class TransformersModelRunner:
+    """
+    基于 Transformers 的备用模型运行器
+    当 vLLM 不可用时使用
+    """
+
+    def __init__(
+        self,
+        model_path: str,
+        tokenizer_path: Optional[str] = None,
+        device: str = "cuda",
+        torch_dtype: str = "auto",
+    ):
+        """
+        初始化 Transformers 模型运行器
+
+        Args:
+            model_path: 模型路径
+            tokenizer_path: Tokenizer 路径
+            device: 设备（cuda 或 cpu）
+            torch_dtype: 数据类型
+        """
+        import torch
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+
+        self.device = device
+        self.tokenizer_path = tokenizer_path or model_path
+
+        logger.info(f"Loading model with transformers: {model_path}")
+        self.tokenizer = AutoTokenizer.from_pretrained(self.tokenizer_path)
+        self.model = AutoModelForCausalLM.from_pretrained(
+            model_path,
+            torch_dtype=torch_dtype if torch_dtype != "auto" else "auto",
+            device_map=device,
+        )
+        logger.info("Model loaded successfully")
+
+    def generate(
+        self,
+        prompts: List[str],
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+        top_k: int = 50,
+        n: int = 1,
+    ) -> List[Dict[str, Any]]:
+        """
+        生成文本
+
+        Args:
+            prompts: 输入提示列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+            top_p: Nucleus 采样参数
+            top_k: Top-K 采样参数
+            n: 生成数量
+
+        Returns:
+            生成结果列表
+        """
+        import torch
+
+        results = []
+        for prompt in prompts:
+            inputs = self.tokenizer(prompt, return_tensors="pt").to(self.device)
+
+            with torch.no_grad():
+                outputs = self.model.generate(
+                    **inputs,
+                    max_new_tokens=max_tokens,
+                    temperature=temperature,
+                    top_p=top_p,
+                    top_k=top_k,
+                    num_return_sequences=n,
+                    do_sample=temperature > 0,
+                )
+
+            generated_texts = [
+                self.tokenizer.decode(output, skip_special_tokens=True)
+                for output in outputs
+            ]
+
+            result = {
+                "prompt": prompt,
+                "outputs": [
+                    {
+                        "text": text,
+                        "token_ids": None,
+                        "cumulative_logprob": None,
+                        "finish_reason": "stop",
+                    }
+                    for text in generated_texts
+                ],
+                "num_input_tokens": len(inputs["input_ids"][0]),
+            }
+            results.append(result)
+
+        return results

models/predictor_wrapper.py

@@ -0,0 +1,201 @@
+# MedGemma 推理包装器
+# 包装 MedGemma 原始 predictor.py 的推理逻辑
+
+import logging
+import os
+import sys
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class MedGemmaPredictor:
+    """
+    MedGemma 推理包装器
+    包装原始 MedGemma predictor 逻辑，提供 OneScience 兼容接口
+    """
+
+    def __init__(self, model_runner: Any, configs: Any):
+        """
+        初始化推理包装器
+
+        Args:
+            model_runner: 模型运行器（VLLMModelRunner 或 TransformersModelRunner）
+            configs: 配置对象
+        """
+        self.model_runner = model_runner
+        self.configs = configs
+
+        # 尝试导入原始 MedGemma 组件（如果可用）
+        self._init_medgemma_components()
+
+    def _init_medgemma_components(self):
+        """初始化 MedGemma 原始组件"""
+        try:
+            # 添加 MedGemma 原始代码路径到 sys.path
+            medgemma_base = os.path.abspath(
+                os.path.join(os.path.dirname(__file__), "..", "..", "..", "..", "..", "..", "medgemma", "python")
+            )
+
+            if os.path.exists(medgemma_base) and medgemma_base not in sys.path:
+                sys.path.insert(0, medgemma_base)
+                logger.info(f"Added MedGemma path: {medgemma_base}")
+
+            # 尝试导入 MedGemma predictor 组件
+            try:
+                from serving import predictor
+                self.has_original_predictor = True
+                logger.info("Successfully imported original MedGemma predictor")
+            except ImportError as e:
+                logger.warning(f"Could not import original MedGemma predictor: {e}")
+                self.has_original_predictor = False
+
+        except Exception as e:
+            logger.warning(f"Error initializing MedGemma components: {e}")
+            self.has_original_predictor = False
+
+    def predict(
+        self,
+        messages: List[Dict[str, Any]],
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+        n: int = 1,
+    ) -> Dict[str, Any]:
+        """
+        运行推理
+
+        Args:
+            messages: OpenAI Chat Completion 格式的消息列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+            top_p: Nucleus 采样参数
+            n: 生成数量
+
+        Returns:
+            OpenAI 兼容格式的响应
+        """
+        # 转换消息为 prompt
+        prompt = self._messages_to_prompt(messages)
+
+        # 运行模型推理
+        results = self.model_runner.generate(
+            prompts=[prompt],
+            max_tokens=max_tokens,
+            temperature=temperature,
+            top_p=top_p,
+            n=n,
+        )
+
+        # 格式化响应为 OpenAI 格式
+        return self._format_openai_response(results[0], messages)
+
+    def _messages_to_prompt(self, messages: List[Dict[str, Any]]) -> str:
+        """
+        将 OpenAI 消息格式转换为 prompt
+
+        Args:
+            messages: 消息列表
+
+        Returns:
+            格式化的 prompt 字符串
+        """
+        prompt_parts = []
+
+        for message in messages:
+            role = message.get("role", "user")
+            content = message.get("content", "")
+
+            # 处理不同角色的消息
+            if role == "system":
+                prompt_parts.append(f"System: {content}")
+            elif role == "user":
+                prompt_parts.append(f"User: {content}")
+            elif role == "assistant":
+                prompt_parts.append(f"Assistant: {content}")
+            else:
+                prompt_parts.append(f"{role}: {content}")
+
+        # 添加 Assistant 前缀以开始生成
+        prompt_parts.append("Assistant:")
+
+        return "\n".join(prompt_parts)
+
+    def _format_openai_response(
+        self,
+        result: Dict[str, Any],
+        messages: List[Dict[str, Any]]
+    ) -> Dict[str, Any]:
+        """
+        将模型输出格式化为 OpenAI Chat Completion 格式
+
+        Args:
+            result: 模型生成结果
+            messages: 原始消息
+
+        Returns:
+            OpenAI 格式的响应
+        """
+        import time
+        import uuid
+
+        choices = []
+        for idx, output in enumerate(result["outputs"]):
+            choice = {
+                "index": idx,
+                "message": {
+                    "role": "assistant",
+                    "content": output["text"].replace(result["prompt"], "").strip(),
+                },
+                "finish_reason": output.get("finish_reason", "stop"),
+            }
+            choices.append(choice)
+
+        response = {
+            "id": f"chatcmpl-{uuid.uuid4().hex[:8]}",
+            "object": "chat.completion",
+            "created": int(time.time()),
+            "model": self.configs.model.variant,
+            "choices": choices,
+            "usage": {
+                "prompt_tokens": result.get("num_input_tokens", 0),
+                "completion_tokens": sum(
+                    len(output.get("token_ids", [])) if output.get("token_ids") else 0
+                    for output in result["outputs"]
+                ),
+                "total_tokens": result.get("num_input_tokens", 0) + sum(
+                    len(output.get("token_ids", [])) if output.get("token_ids") else 0
+                    for output in result["outputs"]
+                ),
+            },
+        }
+
+        return response
+
+    def predict_with_images(
+        self,
+        messages: List[Dict[str, Any]],
+        images: List[Any],
+        max_tokens: int = 500,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+    ) -> Dict[str, Any]:
+        """
+        多模态推理（文本 + 图像）
+
+        Args:
+            messages: 消息列表
+            images: 图像列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+            top_p: Nucleus 采样参数
+
+        Returns:
+            响应字典
+        """
+        # TODO: 实现多模态推理
+        # 这需要集成 MedGemma 的图像处理逻辑
+        logger.warning("Multimodal inference not yet fully implemented")
+
+        # 暂时只处理文本
+        return self.predict(messages, max_tokens, temperature, top_p)

runner/__init__.py

@@ -0,0 +1,3 @@
+from .medical_inference_runner import MedicalInferenceRunner
+
+__all__ = ["MedicalInferenceRunner"]

runner/medical_inference_runner.py

@@ -0,0 +1,319 @@
+# MedGemma 医学推理运行器
+# 类似 Protenix 的 BiologyInferenceRunner
+import sys
+from pathlib import Path
+
+DIR = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(DIR))
+
+import logging
+import os
+import json
+from typing import Any, Dict, List, Optional
+import torch
+
+from models.medgemma import MedGemma
+from models.config import parse_configs, load_config
+
+logger = logging.getLogger(__name__)
+
+class AttrDict(dict):
+    """同时支持 config.key 和 config['key']。"""
+
+    def __getattr__(self, key):
+        try:
+            return self[key]
+        except KeyError as exc:
+            raise AttributeError(key) from exc
+
+    def __setattr__(self, key, value):
+        self[key] = value
+
+def to_attr_dict(value):
+    if isinstance(value, dict):
+        return AttrDict({
+            key: to_attr_dict(item)
+            for key, item in value.items()
+        })
+
+    if isinstance(value, list):
+        return [to_attr_dict(item) for item in value]
+
+    if isinstance(value, tuple):
+        return tuple(to_attr_dict(item) for item in value)
+
+    return value
+
+
+class MedicalInferenceRunner:
+    """
+    MedGemma 医学推理运行器
+    提供统一的推理接口
+    """
+
+    def __init__(self, configs: Any) -> None:
+        """
+        初始化推理运行器
+
+        Args:
+            configs: 配置对象
+        """
+        #self.configs = configs
+        self.configs = to_attr_dict(configs)
+        self.init_env()
+        self.init_basics()
+        self.init_model()
+        self.init_dumper()
+
+        logger.info("MedicalInferenceRunner initialized")
+
+    def init_env(self) -> None:
+        """初始化环境"""
+        self.use_cuda = torch.cuda.is_available()
+        if self.use_cuda:
+            self.device = torch.device("cuda:0")
+            torch.cuda.set_device(self.device)
+            logger.info(f"Using GPU: {torch.cuda.get_device_name(0)}")
+        else:
+            self.device = torch.device("cpu")
+            logger.info("Using CPU")
+
+    def init_basics(self) -> None:
+        """初始化基础设置"""
+        self.dump_dir = self.configs.output.dump_dir
+        self.error_dir = os.path.join(self.dump_dir, "errors")
+        os.makedirs(self.dump_dir, exist_ok=True)
+        os.makedirs(self.error_dir, exist_ok=True)
+        logger.info(f"Output directory: {self.dump_dir}")
+
+    def init_model(self) -> None:
+        """初始化 MedGemma 模型"""
+        logger.info("Loading MedGemma model...")
+        try:
+            self.model = MedGemma(self.configs)
+            logger.info("Model loaded successfully")
+        except Exception as e:
+            logger.error(f"Failed to load model: {e}")
+            raise
+
+    def init_dumper(self) -> None:
+        """初始化结果保存器"""
+        self.output_format = self.configs.output.output_format
+        self.save_predictions = self.configs.output.save_predictions
+        logger.info(f"Output format: {self.output_format}")
+
+    @torch.no_grad()
+    def predict(
+        self,
+        messages: List[Dict[str, Any]],
+        max_tokens: Optional[int] = None,
+        temperature: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        """
+        运行推理
+
+        Args:
+            messages: 消息列表
+            max_tokens: 最大生成 token 数
+            temperature: 采样温度
+
+        Returns:
+            预测结果
+        """
+        if max_tokens is None:
+            max_tokens = self.configs.inference.default_max_tokens
+        if temperature is None:
+            temperature = self.configs.inference.temperature
+
+        try:
+            result = self.model.forward(
+                messages=messages,
+                max_tokens=max_tokens,
+                temperature=temperature,
+            )
+            return result
+        except Exception as e:
+            logger.error(f"Prediction failed: {e}")
+            return {"error": str(e)}
+
+    def run_from_file(self, input_path: str) -> None:
+        """
+        从文件运行推理
+
+        Args:
+            input_path: 输入文件路径（JSON 或 JSONL）
+        """
+        logger.info(f"Loading input from: {input_path}")
+
+        # 读取输入数据
+        if input_path.endswith('.jsonl'):
+            samples = self._load_jsonl(input_path)
+        elif input_path.endswith('.json'):
+            samples = self._load_json(input_path)
+        else:
+            raise ValueError(f"Unsupported file format: {input_path}")
+
+        logger.info(f"Loaded {len(samples)} samples")
+
+        # 处理每个样本
+        results = []
+        for idx, sample in enumerate(samples):
+            logger.info(f"Processing sample {idx + 1}/{len(samples)}")
+
+            try:
+                # 提取消息
+                if "messages" in sample:
+                    messages = sample["messages"]
+                elif "text" in sample:
+                    messages = [{"role": "user", "content": sample["text"]}]
+                elif "question" in sample:
+                    messages = [{"role": "user", "content": sample["question"]}]
+                else:
+                    logger.warning(f"Sample {idx} has no valid input")
+                    continue
+
+                # 运行推理
+                result = self.predict(messages)
+
+                # 添加样本 ID
+                result["sample_id"] = sample.get("id", idx)
+
+                # 保存结果
+                if self.save_predictions:
+                    self._save_result(result, idx)
+
+                results.append(result)
+
+            except Exception as e:
+                logger.error(f"Error processing sample {idx}: {e}")
+                self._save_error(sample, idx, str(e))
+
+        logger.info(f"Completed processing {len(results)} samples")
+
+        # 保存汇总结果
+        self._save_summary(results)
+
+    def _load_json(self, filepath: str) -> List[Dict[str, Any]]:
+        """加载 JSON 文件"""
+        with open(filepath, 'r', encoding='utf-8') as f:
+            data = json.load(f)
+
+        if isinstance(data, list):
+            return data
+        else:
+            return [data]
+
+    def _load_jsonl(self, filepath: str) -> List[Dict[str, Any]]:
+        """加载 JSONL 文件"""
+        samples = []
+        with open(filepath, 'r', encoding='utf-8') as f:
+            for line in f:
+                if line.strip():
+                    samples.append(json.loads(line))
+        return samples
+
+    def _save_result(self, result: Dict[str, Any], idx: int) -> None:
+        """保存单个结果"""
+        output_path = os.path.join(
+            self.dump_dir,
+            f"prediction_{idx}.{self.output_format}"
+        )
+
+        with open(output_path, 'w', encoding='utf-8') as f:
+            json.dump(result, f, indent=2, ensure_ascii=False)
+
+    def _save_error(self, sample: Dict[str, Any], idx: int, error: str) -> None:
+        """保存错误信息"""
+        error_path = os.path.join(
+            self.error_dir,
+            f"error_{idx}.json"
+        )
+
+        error_data = {
+            "sample": sample,
+            "error": error,
+        }
+
+        with open(error_path, 'w', encoding='utf-8') as f:
+            json.dump(error_data, f, indent=2, ensure_ascii=False)
+
+    def _save_summary(self, results: List[Dict[str, Any]]) -> None:
+        """保存汇总结果"""
+        summary_path = os.path.join(self.dump_dir, "summary.json")
+
+        summary = {
+            "total_samples": len(results),
+            "successful": sum(1 for r in results if "error" not in r),
+            "failed": sum(1 for r in results if "error" in r),
+            "results": results,
+        }
+
+        with open(summary_path, 'w', encoding='utf-8') as f:
+            json.dump(summary, f, indent=2, ensure_ascii=False)
+
+        logger.info(f"Summary saved to: {summary_path}")
+
+    def run_interactive(self) -> None:
+        """交互式推理"""
+        logger.info("Starting interactive mode. Type 'quit' to exit.")
+
+        while True:
+            try:
+                user_input = input("\nUser: ")
+                if user_input.lower() in ['quit', 'exit', 'q']:
+                    break
+
+                messages = [{"role": "user", "content": user_input}]
+                result = self.predict(messages)
+
+                # 提取响应
+                if "choices" in result and result["choices"]:
+                    response = result["choices"][0]["message"]["content"]
+                    print(f"\nAssistant: {response}")
+                else:
+                    print(f"\nError: {result}")
+
+            except KeyboardInterrupt:
+                break
+            except Exception as e:
+                logger.error(f"Error: {e}")
+
+        logger.info("Exiting interactive mode")
+
+
+def main():
+    """主函数"""
+    import argparse
+
+    parser = argparse.ArgumentParser(description="MedGemma Medical Inference Runner")
+    parser.add_argument("--config", type=str, required=True, help="Config file path")
+    parser.add_argument("--input", type=str, help="Input file path (JSON/JSONL)")
+    parser.add_argument("--interactive", action="store_true", help="Interactive mode")
+    parser.add_argument("--model_path", type=str, help="Override model path")
+    parser.add_argument("--dump_dir", type=str, help="Override output directory")
+
+    args = parser.parse_args()
+
+    # 加载配置
+    configs = load_config(args.config)
+
+    # 覆盖配置
+    if args.model_path:
+        configs.model.model_path = args.model_path
+    if args.dump_dir:
+        configs.output.dump_dir = args.dump_dir
+
+    # 创建运行器
+    runner = MedicalInferenceRunner(configs)
+
+    # 运行推理
+    if args.interactive:
+        runner.run_interactive()
+    elif args.input:
+        runner.run_from_file(args.input)
+    else:
+        logger.error("Either --input or --interactive must be specified")
+
+
+if __name__ == "__main__":
+    main()

scripts/notebook_conver/cxr_anatomy_localization_with_hugging_face.py

@@ -0,0 +1,270 @@
+#!/usr/bin/env python3
+# c6.py - MedGemma 胸部X光解剖结构定位（增强版）
+# 功能：
+#   - 单张/批量图片推理
+#   - 指定输出目录
+#   - 指定使用的GPU数量
+#   - 坐标自动适配（0-1 或 0-1000）
+#   - 本地模型加载（local_files_only）
+
+import os
+import sys
+import argparse
+import json
+import re
+import torch
+import numpy as np
+import skimage
+from PIL import Image, ImageDraw
+from pathlib import Path
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+# ---------- Monkey-patch for torch<2.6 ----------
+import transformers.masking_utils as masking_utils
+if torch.__version__ < "2.6":
+    _orig_causal = masking_utils.create_causal_mask
+    _orig_sliding = masking_utils.create_sliding_window_causal_mask
+
+    def _safe_pop(kw):
+        kw.pop("or_mask_function", None)
+        kw.pop("and_mask_function", None)
+
+    def _new_causal(*a, **kw):
+        _safe_pop(kw)
+        return _orig_causal(*a, **kw)
+
+    def _new_sliding(*a, **kw):
+        _safe_pop(kw)
+        return _orig_sliding(*a, **kw)
+
+    masking_utils.create_causal_mask = _new_causal
+    masking_utils.create_sliding_window_causal_mask = _new_sliding
+
+
+def pad_image_to_square(image_array):
+    """将图像填充为正方形（与训练预处理一致）"""
+    image_array = skimage.util.img_as_ubyte(image_array)
+    if image_array.ndim < 3:
+        image_array = skimage.color.gray2rgb(image_array)
+    if image_array.shape[2] == 4:
+        image_array = skimage.color.rgba2rgb(image_array)
+
+    h, w = image_array.shape[:2]
+    if h < w:
+        dh = w - h
+        image_array = np.pad(image_array, ((dh // 2, dh - dh // 2), (0, 0), (0, 0)))
+    elif w < h:
+        dw = h - w
+        image_array = np.pad(image_array, ((0, 0), (dw // 2, dw - dw // 2), (0, 0)))
+    return image_array
+
+
+def draw_bounding_box(image, bbox_data, object_name):
+    """绘制边界框（自动识别 0-1 或 0-1000 坐标）"""
+    w_orig, h_orig = image.size
+    new_w = 512
+    new_h = int(h_orig * (new_w / w_orig))
+    scaled = image.resize((new_w, new_h))
+    draw = ImageDraw.Draw(scaled)
+
+    for item in bbox_data:
+        box = item.get("box_2d")
+        label = item.get("label", object_name)
+        if box and len(box) == 4:
+            y0, x0, y1, x1 = box
+
+            # 若坐标值均在 [-1,1] 区间（实际上≤1），视为 0-1 归一化，乘以 1000
+            if max(abs(y0), abs(x0), abs(y1), abs(x1)) <= 1.0:
+                y0, x0, y1, x1 = y0 * 1000, x0 * 1000, y1 * 1000, x1 * 1000
+
+            # 转换到像素坐标（以 512 宽为基准）
+            x0_px = x0 / 1000 * new_w
+            y0_px = y0 / 1000 * new_h
+            x1_px = x1 / 1000 * new_w
+            y1_px = y1 / 1000 * new_h
+
+            draw.rectangle([(x0_px, y0_px), (x1_px, y1_px)], outline="red", width=2)
+            if label:
+                draw.text((x0_px, y0_px - 15), label, fill="red")
+    return scaled
+
+
+def build_prompt(object_name):
+    """构造单结构定位提示词"""
+    return f"""Instructions:
+The following user query will require outputting bounding boxes. The format of bounding boxes coordinates is [y0, x0, y1, x1] where (y0, x0) must be top-left corner and (y1, x1) the bottom-right corner. This implies that x0 < x1 and y0 < y1. Always normalize the x and y coordinates the range [0, 1000], meaning that a bounding box starting at 15% of the image width would be associated with an x coordinate of 150. You MUST output a single parseable json list of objects enclosed into ```json...``` brackets, for instance ```json[{{"box_2d": [800, 3, 840, 471], "label": "car"}}, {{"box_2d": [400, 22, 600, 73], "label": "dog"}}]``` is a valid output. Now answer to the user query.
+
+Remember "left" refers to the patient's left side where the heart is and sometimes underneath an L in the upper right corner of the image.
+
+Query:
+Where is the {object_name}? Don't give a final answer without reasoning. Output the final answer in the format "Final Answer: X" where X is a JSON list of objects. The object needs a "box_2d" and "label" key. Answer:"""
+
+
+def extract_bbox_data(response):
+    """从模型回复中提取 JSON 边界框列表（增强鲁棒性）"""
+    json_str = ""
+    # 策略1：优先提取 Final Answer: 后面的 ```json...```
+    final_match = re.search(r'Final Answer:\s*```json\s*(\[.*?\])\s*```', response, re.DOTALL)
+    if final_match:
+        json_str = final_match.group(1).strip()
+    else:
+        # 策略2：查找 ```json...```
+        if "```json" in response:
+            start = response.find("```json") + len("```json")
+            end = response.find("```", start)
+            if end != -1:
+                json_str = response[start:end].strip()
+    if not json_str:
+        # 回退：提取最后一个 JSON 数组
+        matches = re.findall(r'\[.*\]', response, re.DOTALL)
+        if matches:
+            json_str = matches[-1]
+
+    if json_str:
+        try:
+            return json.loads(json_str)
+        except json.JSONDecodeError as e:
+            print(f"    JSON 解析错误: {e}\n    提取的字符串: {json_str}")
+    return []
+
+
+def process_single_image(img_path, model, processor, object_name, output_dir, preprocess=True):
+    """处理单张图像，返回结果字典"""
+    print(f"\n{'='*60}\nProcessing: {img_path}")
+    image = Image.open(img_path).convert("RGB")
+    orig_size = image.size
+    if preprocess:
+        image = Image.fromarray(pad_image_to_square(np.array(image)))
+        print(f"  Preprocessed: {image.size} (original: {orig_size})")
+
+    prompt = build_prompt(object_name)
+    messages = [
+        {"role": "user", "content": [
+            {"type": "image", "image": image},
+            {"type": "text", "text": prompt}
+        ]}
+    ]
+
+    # 处理输入
+    inputs = processor.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_tensors="pt",
+        return_dict=True,
+    )
+    device = model.device
+    inputs = {k: v.to(device) for k, v in inputs.items()}
+
+    # 推理
+    with torch.inference_mode():
+        gen = model.generate(**inputs, max_new_tokens=1000, do_sample=False)
+
+    input_len = inputs["input_ids"].shape[1]
+    response = processor.batch_decode(gen[:, input_len:], skip_special_tokens=True)[0]
+
+    # 提取坐标
+    bbox_data = extract_bbox_data(response)
+    print(f"  Detected {len(bbox_data)} bounding box(es) for '{object_name}'.")
+
+    # 保存标注图片
+    base = Path(img_path).stem
+    out_img_path = os.path.join(output_dir, f"result_{base}_{object_name.replace(' ', '_')}.png")
+    if bbox_data:
+        draw_bounding_box(image, bbox_data, object_name).save(out_img_path)
+        print(f"  Annotated image saved: {out_img_path}")
+
+    result = {
+        "image_path": str(img_path),
+        "original_size": list(orig_size),
+        "object_name": object_name,
+        "bounding_boxes": bbox_data,
+        "raw_response": response
+    }
+    # 保存单张 JSON（可选）
+    json_path = os.path.join(output_dir, f"result_{base}_{object_name.replace(' ', '_')}.json")
+    with open(json_path, "w") as f:
+        json.dump(result, f, indent=2)
+    return result
+
+
+def main():
+    parser = argparse.ArgumentParser(description="MedGemma 胸部X光单结构定位（支持批量）")
+    parser.add_argument("--image_path", help="单张图像路径")
+    parser.add_argument("--input_dir", help="批量处理图像文件夹")
+    parser.add_argument("--model_path", required=True, help="本地模型目录路径")
+    parser.add_argument("--object_name", default="right clavicle", help="要定位的解剖结构名称")
+    parser.add_argument("--output_dir", default="./outputs", help="结果保存目录（默认 ./outputs）")
+    parser.add_argument("--num_gpus", type=int, default=None, help="使用的GPU数量（如 2）")
+    parser.add_argument("--preprocess", action="store_true", default=True, help="是否将图像填充为正方形")
+    args = parser.parse_args()
+
+    if not args.image_path and not args.input_dir:
+        parser.error("必须指定 --image_path 或 --input_dir 之一")
+    if args.image_path and args.input_dir:
+        parser.error("不能同时指定 --image_path 和 --input_dir")
+
+    os.makedirs(args.output_dir, exist_ok=True)
+
+    # GPU 设置
+    if args.num_gpus and args.num_gpus > 0:
+        available = torch.cuda.device_count()
+        num = min(args.num_gpus, available)
+        os.environ["CUDA_VISIBLE_DEVICES"] = ",".join(str(i) for i in range(num))
+        print(f"Using GPUs: {os.environ['CUDA_VISIBLE_DEVICES']} (available: {available})")
+    else:
+        print(f"All GPUs ({torch.cuda.device_count()}) available.")
+
+    # 加载模型
+    print(f"Loading model from {args.model_path}")
+    processor = AutoProcessor.from_pretrained(
+        args.model_path,
+        trust_remote_code=True
+        #local_files_only=True
+    )
+    model = AutoModelForImageTextToText.from_pretrained(
+        args.model_path,
+        dtype=torch.bfloat16,
+        device_map="auto",
+        trust_remote_code=True,
+        attn_implementation="eager"
+        #local_files_only=True
+    )
+    print("Model loaded.\n")
+
+    # 构建图像列表
+    if args.image_path:
+        image_paths = [args.image_path]
+    else:
+        # 扫描文件夹中所有常见图像文件
+        exts = ('.png', '.jpg', '.jpeg', '.bmp', '.tif', '.tiff')
+        image_paths = sorted([
+            os.path.join(args.input_dir, f)
+            for f in os.listdir(args.input_dir)
+            if f.lower().endswith(exts)
+        ])
+        if not image_paths:
+            print(f"No images found in {args.input_dir}")
+            return
+    print(f"Images to process: {len(image_paths)}, target: '{args.object_name}'")
+
+    # 逐张处理
+    all_results = []
+    for p in image_paths:
+        try:
+            res = process_single_image(p, model, processor, args.object_name, args.output_dir, args.preprocess)
+            all_results.append(res)
+        except Exception as e:
+            print(f"  Error processing {p}: {e}")
+
+    # 批量汇总
+    if len(image_paths) > 1 or args.input_dir:
+        summary_path = os.path.join(args.output_dir, "batch_summary.json")
+        with open(summary_path, "w") as f:
+            json.dump(all_results, f, indent=2)
+        print(f"\nBatch summary saved to: {summary_path}")
+    print("Done.")
+
+
+if __name__ == "__main__":
+    main()

scripts/notebook_conver/cxr_longitudinal_comparison.py

@@ -0,0 +1,197 @@
+#!/usr/bin/env python3
+# compare_cxr.py - MedGemma 纵向CXR对比（仿照 c6.py）
+# 功能：
+#   - 比较两张胸部X光片（如治疗前后），生成描述报告
+#   - 支持本地模型、指定GPU数量、自定义prompt、图像预处理
+#   - 结果保存为文本文件
+
+import os
+import sys
+import argparse
+import json
+import torch
+import numpy as np
+import skimage
+from PIL import Image
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+# ---------- Monkey-patch for torch<2.6 ----------
+import transformers.masking_utils as masking_utils
+if torch.__version__ < "2.6":
+    _orig_causal = masking_utils.create_causal_mask
+    _orig_sliding = masking_utils.create_sliding_window_causal_mask
+
+    def _safe_pop(kw):
+        kw.pop("or_mask_function", None)
+        kw.pop("and_mask_function", None)
+
+    def _new_causal(*a, **kw):
+        _safe_pop(kw)
+        return _orig_causal(*a, **kw)
+
+    def _new_sliding(*a, **kw):
+        _safe_pop(kw)
+        return _orig_sliding(*a, **kw)
+
+    masking_utils.create_causal_mask = _new_causal
+    masking_utils.create_sliding_window_causal_mask = _new_sliding
+
+
+def pad_image_to_square(image_array):
+    """将图像填充为正方形（与训练预处理一致）"""
+    image_array = skimage.util.img_as_ubyte(image_array)
+    if image_array.ndim < 3:
+        image_array = skimage.color.gray2rgb(image_array)
+    if image_array.shape[2] == 4:
+        image_array = skimage.color.rgba2rgb(image_array)
+
+    h, w = image_array.shape[:2]
+    if h < w:
+        dh = w - h
+        image_array = np.pad(image_array, ((dh // 2, dh - dh // 2), (0, 0), (0, 0)))
+    elif w < h:
+        dw = h - w
+        image_array = np.pad(image_array, ((0, 0), (dw // 2, dw - dw // 2), (0, 0)))
+    return image_array
+
+
+def build_default_prompt():
+    """默认的纵向CXR对比提示词"""
+    return (
+        "Provide a comparison of these two images and include details from "
+        "the image which students should take note of when reading longitudinal CXR."
+    )
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="MedGemma 纵向胸部X光片对比"
+    )
+    parser.add_argument("--model_path", required=True, help="本地模型目录路径")
+    parser.add_argument("--image1", required=True, help="第一张图像路径（如治疗前）")
+    parser.add_argument("--image2", required=True, help="第二张图像路径（如治疗后）")
+    parser.add_argument("--prompt", default=None,
+                        help="自定义对比提示词（不指定则使用默认）")
+    parser.add_argument("--output_dir", default="./outputs",
+                        help="结果保存目录（默认 ./outputs）")
+    parser.add_argument("--num_gpus", type=int, default=None,
+                        help="使用的GPU数量（建议在命令行用 CUDA_VISIBLE_DEVICES 控制）")
+    parser.add_argument("--preprocess", action="store_true", default=False,
+                        help="是否将图像填充为正方形（默认不填充）")
+    args = parser.parse_args()
+
+    os.makedirs(args.output_dir, exist_ok=True)
+
+    # GPU 设置（若在脚本内设，需在 import torch 之前，此处仅作提示）
+    if args.num_gpus and args.num_gpus > 0:
+        # 实际限制应在运行脚本前通过环境变量设置，这里只打印建议
+        print(f"提示：建议在命令行中设置 CUDA_VISIBLE_DEVICES=0,1,... 来限制GPU。"
+              f"当前脚本不修改可见设备。")
+    else:
+        print(f"可用GPU数量: {torch.cuda.device_count()}")
+
+    # 加载模型（全本地）
+    print(f"Loading model from {args.model_path}")
+    processor = AutoProcessor.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        local_files_only=True
+    )
+    model = AutoModelForImageTextToText.from_pretrained(
+        args.model_path,
+        dtype=torch.bfloat16,
+        device_map="auto",
+        trust_remote_code=True,
+        attn_implementation="eager",
+        local_files_only=True
+    )
+    print("Model loaded.\n")
+
+    # 加载图像
+    print(f"Loading images:\n  {args.image1}\n  {args.image2}")
+    image1 = Image.open(args.image1).convert("RGB")
+    image2 = Image.open(args.image2).convert("RGB")
+    print(f"Original sizes: {image1.size}, {image2.size}")
+
+    if args.preprocess:
+        img1_arr = pad_image_to_square(np.array(image1))
+        img2_arr = pad_image_to_square(np.array(image2))
+        image1 = Image.fromarray(img1_arr)
+        image2 = Image.fromarray(img2_arr)
+        print(f"Preprocessed sizes: {image1.size}, {image2.size}")
+
+    # 构造提示词
+    prompt = args.prompt if args.prompt else build_default_prompt()
+    print(f"Prompt: {prompt[:100]}...")
+
+    # 构造 messages（包含两张图像）
+    messages = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "image", "image": image1},
+                {"type": "image", "image": image2},
+                {"type": "text", "text": prompt}
+            ]
+        }
+    ]
+
+    # 处理输入
+    print("Processing inputs...")
+    inputs = processor.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_tensors="pt",
+        return_dict=True,
+    )
+    device = model.device
+    inputs = {k: v.to(device) for k, v in inputs.items()}
+    if "pixel_values" in inputs:
+        print(f"Pixel values shape: {inputs['pixel_values'].shape}")
+
+    # 推理
+    print("Running inference...")
+    with torch.inference_mode():
+        generate_ids = model.generate(
+            **inputs,
+            max_new_tokens=600,
+            do_sample=False,
+        )
+
+    input_len = inputs["input_ids"].shape[1]
+    response = processor.batch_decode(
+        generate_ids[:, input_len:], skip_special_tokens=True
+    )[0]
+
+    # 输出结果
+    print("\n" + "=" * 60)
+    print("MODEL RESPONSE:")
+    print(response)
+    print("=" * 60)
+
+    # 保存为文本文件
+    base1 = os.path.splitext(os.path.basename(args.image1))[0]
+    base2 = os.path.splitext(os.path.basename(args.image2))[0]
+    out_txt = os.path.join(args.output_dir, f"compare_{base1}_vs_{base2}.txt")
+    with open(out_txt, "w", encoding="utf-8") as f:
+        f.write(f"Image1: {args.image1}\nImage2: {args.image2}\n")
+        f.write(f"Prompt: {prompt}\n\n")
+        f.write(f"Response:\n{response}\n")
+    print(f"Comparison result saved to: {out_txt}")
+
+    # 可选保存 JSON
+    out_json = os.path.join(args.output_dir, f"compare_{base1}_vs_{base2}.json")
+    result = {
+        "image1": args.image1,
+        "image2": args.image2,
+        "prompt": prompt,
+        "response": response
+    }
+    with open(out_json, "w") as f:
+        json.dump(result, f, indent=2)
+    print(f"JSON saved to: {out_json}")
+
+
+if __name__ == "__main__":
+    main()

scripts/notebook_conver/detect_image_token.py

@@ -0,0 +1,90 @@
+# detect_image_token.py
+import os
+import sys
+import argparse
+from transformers import AutoProcessor
+from PIL import Image
+
+def detect_image_token():
+    parser = argparse.ArgumentParser(description='Detect correct image token for MedGemma model')
+    parser.add_argument('--model_path', type=str, required=True,
+                        help='Path to local MedGemma model directory')
+    args = parser.parse_args()
+
+    model_path = args.model_path
+    
+    print(f"Loading processor from: {model_path}")
+    
+    try:
+        processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
+        tokenizer = processor.tokenizer
+        
+        print("Inspecting tokenizer for image-related tokens...")
+        
+        # Check various attributes that might contain image token info
+        attrs_to_check = [
+            'image_token', 'image_token_id',
+            'pad_token', 'bos_token', 'eos_token', 'unk_token',
+            'additional_special_tokens', 'additional_special_tokens_ids'
+        ]
+        
+        for attr in attrs_to_check:
+            try:
+                value = getattr(tokenizer, attr, None)
+                if value is not None:
+                    print(f"{attr}: {value}")
+            except:
+                print(f"{attr}: Could not access")
+        
+        # Look for common image tokens in vocab
+        common_image_tokens = [
+            "<image>", "<img>", "<IMG>", "<IMAGE>", 
+            "<|image|>", "[IMG]", "[IMAGE]",
+            "<start_of_image>", "<end_of_image>",
+            "<image_soft_token>", "<image_token>",
+            "<img>", "</img>", "<IMG>", "</IMG>",
+            "image", "IMAGE", "img", "IMG",
+            "<vision>", "</vision>", "<visual>", "</visual>"
+        ]
+        
+        print("\nChecking common image tokens:")
+        for token in common_image_tokens:
+            try:
+                token_id = tokenizer.convert_tokens_to_ids(token)
+                # If the token exists in vocab, convert_tokens_to_ids returns its ID
+                # Otherwise it returns the unk_token_id
+                unk_id = tokenizer.unk_token_id
+                if token_id != unk_id:
+                    print(f"Found token '{token}' with ID: {token_id}")
+                else:
+                    print(f"Token '{token}' not in vocab (would use UNK token ID: {unk_id})")
+            except Exception as e:
+                print(f"Error checking token '{token}': {e}")
+        
+        # Also test some sample prompts with different image tokens
+        print("\nTesting sample prompts with different image tokens:")
+        test_prompts = [
+            "Describe this image: <image>",
+            "What do you see? <image>",
+            "Analyze: <image>",
+            "Look at this: <image>",
+            "Explain: <|image|>",
+            "Look: <start_of_image><image_soft_token><end_of_image>"
+        ]
+        
+        for i, prompt in enumerate(test_prompts[:3]):  # Only test first few to avoid errors
+            try:
+                print(f"\nTest {i+1}: '{prompt}'")
+                inputs = processor(text=prompt, return_tensors="pt")
+                print(f"  Input IDs shape: {inputs['input_ids'].shape}")
+                # Decode to see actual tokens
+                decoded = tokenizer.decode(inputs['input_ids'][0])
+                print(f"  Decoded: {decoded[:200]}...")
+            except Exception as e:
+                print(f"  Error: {e}")
+        
+    except Exception as e:
+        print(f"Error loading processor: {e}")
+
+if __name__ == "__main__":
+    detect_image_token()

scripts/notebook_conver/evaluate_on_medqa.py

@@ -0,0 +1,240 @@
+#!/usr/bin/env python3
+# medqa_eval.py – MedGemma on MedQA (仿照 c6.py 风格)
+# 功能：
+#   1. 加载本地 MedGemma 模型（纯文本推理）
+#   2. 加载 MedQA 数据集（支持在线或本地 Parquet）
+#   3. 逐条生成答案并提取选项
+#   4. 计算准确率，保存详细结果和汇总
+
+import os
+import sys
+import argparse
+import json
+import re
+import torch
+import numpy as np
+from tqdm import tqdm
+from transformers import AutoProcessor, AutoModelForImageTextToText
+import datasets
+
+# ---------- Monkey-patch for torch<2.6 ----------
+import transformers.masking_utils as masking_utils
+if torch.__version__ < "2.6":
+    _orig_causal = masking_utils.create_causal_mask
+    _orig_sliding = masking_utils.create_sliding_window_causal_mask
+
+    def _safe_pop(kw):
+        kw.pop("or_mask_function", None)
+        kw.pop("and_mask_function", None)
+
+    def _new_causal(*a, **kw):
+        _safe_pop(kw)
+        return _orig_causal(*a, **kw)
+
+    def _new_sliding(*a, **kw):
+        _safe_pop(kw)
+        return _orig_sliding(*a, **kw)
+
+    masking_utils.create_causal_mask = _new_causal
+    masking_utils.create_sliding_window_causal_mask = _new_sliding
+
+
+def format_prompt(question: str, options: dict) -> str:
+    """构造选择题提示词"""
+    options_str = f"(A) {options['A']} (B) {options['B']} (C) {options['C']} (D) {options['D']}"
+    prompt = f"""Answer the given question. Think step by step.
+You can directly provide the answer (A single letter), without further additions. E.g. "Final Answer: (A)".
+Question: {question}
+{options_str}
+"""
+    return prompt
+
+
+# 答案提取正则（与原脚本一致）
+ANSWER_PATTERNS = [
+    r'The final answer is\s\(([A-J])\)',
+    r'The final answer is\s\**\(([A-J])\)\**',
+    r'The final answer is\s\$\\boxed{([A-J])}\$',
+    r'Final Answer:\(([A-J])\)',
+    r'Final Answer:\s\(([A-J])\)',
+    r'Final Answer:\s\(?([A-J])',
+    r'Final Answer:\s*\**\(([A-J])\)\**',
+    r'\**Final Answer:\**\s\(([A-J])\)',
+]
+
+
+#def extract_answer(text: str) -> str:
+#    """从模型回复中提取选项字母"""
+#    if not isinstance(text, str) or not text:
+#        return None
+#    for pat in ANSWER_PATTERNS:
+#        m = re.search(pat, text)
+#        if m:
+#            return m.group(1)
+#    return None
+
+def extract_answer(text: str) -> str:
+    """从模型回复中提取选项字母，支持多种回退策略"""
+    if not isinstance(text, str) or not text:
+        return None
+
+    # 1. 优先用原始正则匹配标准格式
+    for pat in ANSWER_PATTERNS:
+        m = re.search(pat, text)
+        if m:
+            return m.group(1)
+
+    # 2. 回退：在 <unused95> 之后（正式回答部分）寻找最后一个单独的大写字母
+    if "<unused95>" in text:
+        post_think = text.split("<unused95>")[-1]
+        # 寻找形如 " (A)" 或 "(A)" 的选项
+        matches = re.findall(r'\(([A-J])\)', post_think)
+        if matches:
+            return matches[-1]   # 通常最后一个选项是最终答案
+        # 如果仍未找到，尝试寻找单独的大写字母（可能模型说 "Answer: A"）
+        m = re.search(r'\b([A-J])\b\s*$', post_think)
+        if m:
+            return m.group(1)
+
+    # 3. 全局回退：在整个回复中找最后出现的 (X) 格式
+    matches = re.findall(r'\(([A-J])\)', text)
+    if matches:
+        return matches[-1]
+
+    return None
+
+
+def main():
+    parser = argparse.ArgumentParser(description="MedGemma MedQA 评测")
+    parser.add_argument("--model_path", required=True, help="本地 MedGemma 模型路径")
+    parser.add_argument("--output_dir", default="./medqa_outputs", help="输出目录（保存结果）")
+    parser.add_argument("--max_samples", type=int, default=None,
+                        help="限制评测样本数（默认全部）")
+    parser.add_argument("--num_gpus", type=int, default=None,
+                        help="使用的 GPU 数量（建议在命令行用 CUDA_VISIBLE_DEVICES 控制）")
+    parser.add_argument("--max_new_tokens", type=int, default=2048,
+                        help="生成的最大 token 数")
+    parser.add_argument("--temperature", type=float, default=0.0,
+                        help="生成温度（0 为确定性）")
+    parser.add_argument("--parquet_dir", type=str, default=None,
+                        help="本地 MedQA Parquet 文件夹路径（包含 train/test/dev 等文件）")
+    args = parser.parse_args()
+
+    os.makedirs(args.output_dir, exist_ok=True)
+
+    # GPU 设置提示
+    if args.num_gpus:
+        print("提示：请通过环境变量限制 GPU，例如 CUDA_VISIBLE_DEVICES=0,1")
+    print(f"可用 GPU 数量: {torch.cuda.device_count()}")
+
+    # 加载本地模型（纯文本推理，仍使用 AutoModelForImageTextToText）
+    print(f"Loading model from {args.model_path} ...")
+    processor = AutoProcessor.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        local_files_only=True
+    )
+    model = AutoModelForImageTextToText.from_pretrained(
+        args.model_path,
+        dtype=torch.bfloat16,
+        device_map="auto",
+        trust_remote_code=True,
+        attn_implementation="eager",
+        local_files_only=True
+    )
+    print("Model loaded.\n")
+
+    # 加载 MedQA 数据集
+    if args.parquet_dir:
+        print(f"Loading MedQA dataset from local Parquet directory: {args.parquet_dir}")
+        data_files = {
+            "train": os.path.join(args.parquet_dir, "train-*.parquet"),
+            "test": os.path.join(args.parquet_dir, "test-*.parquet"),
+            "validation": os.path.join(args.parquet_dir, "dev-*.parquet"),  # 验证集文件前缀为 dev
+        }
+        dataset = datasets.load_dataset("parquet", data_files=data_files)
+    else:
+        print("Loading MedQA dataset from openlifescienceai/medqa...")
+        dataset = datasets.load_dataset("openlifescienceai/medqa")
+
+    test_data = dataset["test"]
+    if args.max_samples and args.max_samples < len(test_data):
+        test_data = test_data.select(range(args.max_samples))
+    print(f"Total test samples: {len(test_data)}")
+
+    # 准备结果容器
+    results = []
+    correct = 0
+
+    # 逐条推理
+    for idx, item in enumerate(tqdm(test_data, desc="Inference")):
+        data = item["data"]  # 注意数据集结构：item["data"] 包含 Question, Options, Correct Option
+        question = data["Question"]
+        options = data["Options"]
+        gold_answer = data["Correct Option"]
+
+        prompt = format_prompt(question, options)
+        messages = [
+            {"role": "system", "content": [{"type": "text", "text": "SYSTEM INSTRUCTION: think silently if needed."}]},
+            {"role": "user", "content": [{"type": "text", "text": prompt}]}
+        ]
+
+        # 应用聊天模板
+        inputs = processor.apply_chat_template(
+            messages,
+            add_generation_prompt=True,
+            tokenize=True,
+            return_tensors="pt",
+            return_dict=True
+        )
+        device = model.device
+        inputs = {k: v.to(device) for k, v in inputs.items()}
+
+        with torch.inference_mode():
+            gen = model.generate(
+                **inputs,
+                max_new_tokens=args.max_new_tokens,
+                do_sample=(args.temperature > 0),
+                temperature=args.temperature if args.temperature > 0 else None
+            )
+
+        input_len = inputs["input_ids"].shape[1]
+        response = processor.batch_decode(gen[:, input_len:], skip_special_tokens=True)[0]
+
+        # 提取答案
+        extracted = extract_answer(response)
+        is_correct = (extracted == gold_answer)
+        if is_correct:
+            correct += 1
+
+        # 保存单条结果
+        results.append({
+            "index": idx,
+            "question": question,
+            "options": options,
+            "gold_answer": gold_answer,
+            "model_response": response,
+            "extracted_answer": extracted,
+            "correct": is_correct
+        })
+
+    # 计算准确率
+    total = len(test_data)
+    accuracy = correct / total if total > 0 else 0.0
+    print(f"\nAccuracy: {accuracy:.4f} ({correct}/{total})")
+
+    # 保存结果
+    output_file = os.path.join(args.output_dir, "medqa_results.json")
+    with open(output_file, "w", encoding="utf-8") as f:
+        json.dump({"accuracy": accuracy, "total": total, "correct": correct, "details": results}, f, indent=2)
+    print(f"Results saved to {output_file}")
+
+    # 同时输出摘要文本
+    summary_file = os.path.join(args.output_dir, "summary.txt")
+    with open(summary_file, "w") as f:
+        f.write(f"Accuracy: {accuracy:.4f} ({correct}/{total})\n")
+    print(f"Summary saved to {summary_file}")
+
+
+if __name__ == "__main__":
+    main()

scripts/notebook_conver/fine_tune_with_hugging_face.py

@@ -0,0 +1,333 @@
+#!/usr/bin/env python3
+# train_nct.py – MedGemma 组织分类微调 & 评估 (仿照 c6.py)
+# 功能：
+#   1. 加载本地 NCT-CRC-HE-100K 训练集 和 CRC-VAL-HE-7K 测试集（自动解压 zip）
+#   2. 使用 QLoRA 微调 MedGemma（4-bit 量化）
+#   3. 在测试集上评估准确率和 F1
+
+import os
+import sys
+import argparse
+import json
+import zipfile
+import torch
+import numpy as np
+from typing import Any
+from datasets import load_dataset, ClassLabel
+from transformers import (
+    AutoProcessor,
+    AutoModelForImageTextToText,
+#    BitsAndBytesConfig,
+    pipeline,
+)
+from peft import LoraConfig
+from trl import SFTConfig, SFTTrainer
+import evaluate
+
+# ---------- Monkey-patch for torch<2.6 ----------
+import transformers.masking_utils as masking_utils
+if torch.__version__ < "2.6":
+    _orig_causal = masking_utils.create_causal_mask
+    _orig_sliding = masking_utils.create_sliding_window_causal_mask
+
+    def _safe_pop(kw):
+        kw.pop("or_mask_function", None)
+        kw.pop("and_mask_function", None)
+
+    def _new_causal(*a, **kw):
+        _safe_pop(kw)
+        return _orig_causal(*a, **kw)
+
+    def _new_sliding(*a, **kw):
+        _safe_pop(kw)
+        return _orig_sliding(*a, **kw)
+
+    masking_utils.create_causal_mask = _new_causal
+    masking_utils.create_sliding_window_causal_mask = _new_sliding
+
+
+TISSUE_CLASSES = [
+    "A: adipose",
+    "B: background",
+    "C: debris",
+    "D: lymphocytes",
+    "E: mucus",
+    "F: smooth muscle",
+    "G: normal colon mucosa",
+    "H: cancer-associated stroma",
+    "I: colorectal adenocarcinoma epithelium",
+]
+
+PROMPT = f"What is the most likely tissue type shown in the histopathology image?\n" + "\n".join(TISSUE_CLASSES)
+
+
+def extract_zip(zip_path: str, extract_to: str) -> str:
+    """解压 zip 文件，返回解压后的目录路径"""
+    if not os.path.exists(extract_to):
+        os.makedirs(extract_to, exist_ok=True)
+        with zipfile.ZipFile(zip_path, "r") as zf:
+            zf.extractall(extract_to)
+    return extract_to
+
+
+def format_data(example: dict[str, Any]) -> dict[str, Any]:
+    """构造训练 messages 格式"""
+    example["messages"] = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "image"},
+                {"type": "text", "text": PROMPT},
+            ],
+        },
+        {
+            "role": "assistant",
+            "content": [
+                {"type": "text", "text": TISSUE_CLASSES[example["label"]]},
+            ],
+        },
+    ]
+    return {
+        "image": example["image"],   # 必须包含这一行
+        "text": "...处理好的文本..."  # 必须有文本字段
+    }
+    #return example
+
+
+def format_test_data(example: dict[str, Any]) -> dict[str, Any]:
+    """构造测试 messages 格式（无 assistant）"""
+    example["messages"] = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "image"},
+                {"type": "text", "text": PROMPT},
+            ],
+        },
+    ]
+    return example
+
+
+def collate_fn(examples: list[dict[str, Any]], processor):
+    texts = []
+    images = []
+    for example in examples:
+        images.append([example["image"].convert("RGB")])
+        texts.append(
+            processor.apply_chat_template(
+                example["messages"], add_generation_prompt=False, tokenize=False
+            ).strip()
+        )
+
+    batch = processor(text=texts, images=images, return_tensors="pt", padding=True)
+
+    labels = batch["input_ids"].clone()
+    # 掩码图像 token 和填充 token
+    image_token_id = [
+        processor.tokenizer.convert_tokens_to_ids(
+            processor.tokenizer.special_tokens_map["boi_token"]
+        )
+    ]
+    labels[labels == processor.tokenizer.pad_token_id] = -100
+    for tok_id in image_token_id:
+        labels[labels == tok_id] = -100
+    labels[labels == 262144] = -100
+
+    batch["labels"] = labels
+    return batch
+
+
+def postprocess(prediction: list[dict[str, str]], do_full_match: bool = False) -> int:
+    """将模型预测转换为类别索引"""
+    response_text = prediction[0]["generated_text"]
+    if do_full_match:
+        try:
+            return LABEL_FEATURE.str2int(response_text)
+        except:
+            return -1
+    for label in TISSUE_CLASSES:
+        if label in response_text or f"({label.replace(': ', ') ')}" in response_text:
+            return LABEL_FEATURE.str2int(label)
+    return -1
+
+
+def main():
+    parser = argparse.ArgumentParser(description="MedGemma NCT 组织分类微调与评估")
+    parser.add_argument("--model_path", required=True, help="本地 MedGemma 模型路径")
+    parser.add_argument("--train_zip", default="./NCT-CRC-HE-100K.zip", help="训练集 zip 路径")
+    parser.add_argument("--test_zip", default="./CRC-VAL-HE-7K.zip", help="测试集 zip 路径")
+    parser.add_argument("--output_dir", default="./medgemma-nct-lora", help="微调模型输出目录")
+    parser.add_argument("--num_train_epochs", type=int, default=1)
+    parser.add_argument("--learning_rate", type=float, default=2e-4)
+    parser.add_argument("--per_device_train_batch_size", type=int, default=4)
+    parser.add_argument("--max_train_samples", type=int, default=9000, help="训练样本数")
+    parser.add_argument("--max_val_samples", type=int, default=1000, help="验证样本数")
+    parser.add_argument("--max_test_samples", type=int, default=1000, help="测试样本数（评估用）")
+    parser.add_argument("--num_gpus", type=int, default=None, help="使用的 GPU 数量（建议在命令行用 CUDA_VISIBLE_DEVICES 控制）")
+    parser.add_argument("--eval_only", action="store_true", help="仅评估，跳过训练")
+    parser.add_argument("--skip_train", action="store_true", help="跳过训练（与 eval_only 相同）")
+    args = parser.parse_args()
+
+    # GPU 设置提示
+    if args.num_gpus:
+        print("提示：请通过环境变量限制 GPU，例如 CUDA_VISIBLE_DEVICES=0,1")
+    print(f"可用 GPU 数量: {torch.cuda.device_count()}")
+
+    # 解压数据集
+    train_dir = os.path.splitext(args.train_zip)[0]
+    test_dir = os.path.splitext(args.test_zip)[0]
+    if not os.path.exists(train_dir):
+        print(f"解压训练集 {args.train_zip} -> {train_dir}")
+        extract_zip(args.train_zip, train_dir)
+    if not os.path.exists(test_dir):
+        print(f"解压测试集 {args.test_zip} -> {test_dir}")
+        extract_zip(args.test_zip, test_dir)
+
+    # 加载数据集
+    print("加载训练集...")
+    data = load_dataset(train_dir, split="train")
+    data = data.train_test_split(
+        train_size=args.max_train_samples,
+        test_size=args.max_val_samples,
+        shuffle=True,
+        seed=42,
+    )
+    data["validation"] = data.pop("test")
+
+    # 格式化数据
+    #data = data.map(format_data, remove_columns=data["train"].column_names)
+    data = data.map(format_data)
+
+    # 加载模型 (QLoRA 量化)
+    print(f"加载模型 {args.model_path}...")
+    model_kwargs = dict(
+        attn_implementation="eager",
+        torch_dtype=torch.bfloat16,
+        device_map="auto",
+        local_files_only=True,
+    )
+   # model_kwargs["quantization_config"] = BitsAndBytesConfig(
+   #     load_in_4bit=True,
+   #     bnb_4bit_use_double_quant=True,
+   #     bnb_4bit_quant_type="nf4",
+   #     bnb_4bit_compute_dtype=torch.bfloat16,
+   #     bnb_4bit_quant_storage=torch.bfloat16,
+   # )
+
+    model = AutoModelForImageTextToText.from_pretrained(args.model_path, **model_kwargs)
+    processor = AutoProcessor.from_pretrained(args.model_path, local_files_only=True)
+    processor.tokenizer.padding_side = "right"  # 训练用右填充
+
+    # LoRA 配置
+    peft_config = LoraConfig(
+        lora_alpha=16,
+        lora_dropout=0.05,
+        r=16,
+        bias="none",
+        target_modules="all-linear",
+        task_type="CAUSAL_LM",
+        modules_to_save=["lm_head", "embed_tokens"],
+    )
+
+    # 训练参数
+    sft_args = SFTConfig(
+        output_dir=args.output_dir,
+        num_train_epochs=args.num_train_epochs,
+        per_device_train_batch_size=args.per_device_train_batch_size,
+        per_device_eval_batch_size=args.per_device_train_batch_size,
+        gradient_accumulation_steps=4,
+        gradient_checkpointing=True,
+        optim="adamw_torch_fused",
+        logging_steps=50,
+        save_strategy="epoch",
+        eval_strategy="steps",
+        eval_steps=50,
+        learning_rate=args.learning_rate,
+        bf16=True,
+        max_grad_norm=0.3,
+        warmup_ratio=0.03,
+        lr_scheduler_type="linear",
+        push_to_hub=False,
+        report_to="none",
+        gradient_checkpointing_kwargs={"use_reentrant": False},
+        dataset_kwargs={"skip_prepare_dataset": True},
+        remove_unused_columns=False,
+        label_names=["labels"],
+    )
+
+    if not args.eval_only and not args.skip_train:
+        print("开始训练...")
+        trainer = SFTTrainer(
+            model=model,
+            args=sft_args,
+            train_dataset=data["train"],
+            eval_dataset=data["validation"].select(range(min(200, len(data["validation"])))),
+            peft_config=peft_config,
+            processing_class=processor,
+            data_collator=lambda examples: collate_fn(examples, processor),
+        )
+        trainer.train()
+        trainer.save_model()
+        print(f"模型已保存到 {args.output_dir}")
+        del trainer
+        torch.cuda.empty_cache()
+
+    # ---------- 评估 ----------
+    print("加载测试集...")
+    test_data = load_dataset(test_dir, split="train")
+    test_data = test_data.shuffle(seed=42).select(range(args.max_test_samples))
+    test_data = test_data.map(format_test_data, remove_columns=test_data.column_names)
+
+    # 设置标签特征（全局变量，供 postprocess 使用）
+    global LABEL_FEATURE
+    test_data = test_data.cast_column("label", ClassLabel(names=TISSUE_CLASSES))
+    LABEL_FEATURE = test_data.features["label"]
+
+    # 构建评估管���
+    model_id = args.output_dir if (args.eval_only or args.skip_train) else args.model_path
+
+    print(f"创建评估管道，模型: {model_id}")
+    eval_pipe = pipeline(
+        "image-text-to-text",
+        model=model_id,
+        processor=processor,
+        torch_dtype=torch.bfloat16,
+        device_map="auto",
+    )
+    eval_pipe.model.generation_config.do_sample = False
+    eval_pipe.model.generation_config.pad_token_id = processor.tokenizer.eos_token_id
+    processor.tokenizer.padding_side = "left"  # 推理用左填充
+
+    # 执行推理
+    print("运行推理...")
+    outputs = eval_pipe(
+        text=test_data["messages"],
+        images=test_data["image"],
+        max_new_tokens=40,
+        batch_size=64,
+        return_full_text=False,
+    )
+
+    # 后处理
+    do_full_match = args.eval_only or args.skip_train  # 微调后模型输出可能更精确
+    predictions = [postprocess(out, do_full_match) for out in outputs]
+    references = test_data["label"]
+
+    # 计算指标
+    accuracy_metric = evaluate.load("accuracy")
+    f1_metric = evaluate.load("f1")
+    acc = accuracy_metric.compute(predictions=predictions, references=references)
+    f1 = f1_metric.compute(predictions=predictions, references=references, average="weighted")
+    metrics = {**acc, **f1}
+    print(f"评估结果: {metrics}")
+
+    # 保存结果
+    os.makedirs(args.output_dir, exist_ok=True)
+    result_file = os.path.join(args.output_dir, "eval_metrics.json")
+    with open(result_file, "w") as f:
+        json.dump(metrics, f, indent=2)
+    print(f"结果已保存到 {result_file}")
+
+
+if __name__ == "__main__":
+    main()

scripts/run_cxr_anatomy.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+set -euo pipefail
+
+SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+EXAMPLE_DIR=$(cd "${SCRIPT_DIR}/.." && pwd)
+REPO_ROOT=$(cd "${SCRIPT_DIR}/../../../.." && pwd)
+#source "${REPO_ROOT}/env.sh"
+cd $SCRIPT_DIR
+pwd
+
+
+##  single image
+HIP_VISIBLE_DEVICES=0 \
+	python ./notebook_conver/cxr_anatomy_localization_with_hugging_face.py    \
+	--model_path ${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it  \
+     	--image_path "${ONESCIENCE_DATASETS_DIR}/medgemma/Chest_Xray/COVID19_Pneumonia_Normal_Chest_Xray_PA_Dataset/covid/COVID-19 (89).jpg" \
+	--object_name "right clavicle" \
+	--num_gpus 2
+
+## multiple images
+HIP_VISIBLE_DEVICES=0 \
+	python ./notebook_conver/cxr_anatomy_localization_with_hugging_face.py \
+    	--model_path ${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it \
+    	--input_dir "${ONESCIENCE_DATASETS_DIR}/medgemma/test_images"   \
+   	--object_name "right clavicle" \
+	--num_gpus 2

scripts/run_cxr_longitudinal_comparison.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+set -euo pipefail
+
+SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+EXAMPLE_DIR=$(cd "${SCRIPT_DIR}/.." && pwd)
+REPO_ROOT=$(cd "${SCRIPT_DIR}/../../../.." && pwd)
+#source "${REPO_ROOT}/env.sh"
+cd $SCRIPT_DIR
+pwd
+
+HIP_VISIBLE_DEVICES=0 \
+	python ./notebook_conver/cxr_longitudinal_comparison.py    \
+	--model_path ${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it  \
+	--image1 ${ONESCIENCE_DATASETS_DIR}/medgemma/test_compare/longitudinal_cxr_before.png \
+	--image2 ${ONESCIENCE_DATASETS_DIR}/medgemma/test_compare/longitudinal_cxr_after.png \
+	--output_dir ./compare_outputs \
+	#--preprocess   # 可选，如果图像不是正方形且需要填充
+

scripts/run_evaluate_on_medqa.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+set -euo pipefail
+
+SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+EXAMPLE_DIR=$(cd "${SCRIPT_DIR}/.." && pwd)
+REPO_ROOT=$(cd "${SCRIPT_DIR}/../../../.." && pwd)
+#source "${REPO_ROOT}/env.sh"
+cd $SCRIPT_DIR
+pwd
+
+
+HIP_VISIBLE_DEVICES=0 \
+	python ./notebook_conver/evaluate_on_medqa.py    \
+	--model_path ${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it  \
+	--parquet_dir ${ONESCIENCE_DATASETS_DIR}/medgemma/medqa \
+	--output_dir ./medqa_results \
+	--max_samples 10   # 可选：先测试 100 条
+

scripts/run_fine_tune.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+set -euo pipefail
+
+SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+EXAMPLE_DIR=$(cd "${SCRIPT_DIR}/.." && pwd)
+REPO_ROOT=$(cd "${SCRIPT_DIR}/../../../.." && pwd)
+#source "${REPO_ROOT}/env.sh"
+cd $SCRIPT_DIR
+pwd
+
+
+# ==========================================
+# 自动检查并升级关键依赖
+# ==========================================
+echo "🔍  自动检查并修复关键依赖版本..."
+python -c "
+import pkg_resources
+pkg_resources.require('transformers==5.12.1')
+" || pip install --upgrade transformers==5.12.1
+
+echo "🔍  自动检查 accelerate 版本..."
+python -c "
+import pkg_resources
+pkg_resources.require('accelerate>=0.29.0')
+" || pip install --upgrade accelerate
+# 如果升级到最新版（如 1.15.x+）出现循环导入报错，可将上面那行换成：
+# " || pip install accelerate==1.0.0
+
+# ==========================================
+# 执行微调训练
+# ==========================================
+HIP_VISIBLE_DEVICES=0 \
+	python ./notebook_conver/fine_tune_with_hugging_face.py    \
+	--model_path ${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it  \
+	--train_zip ${ONESCIENCE_DATASETS_DIR}/medgemma/nct/NCT-CRC-HE-100K.zip \
+	--test_zip ${ONESCIENCE_DATASETS_DIR}/medgemma/nct/CRC-VAL-HE-7K.zip \
+	--output_dir ./medgemma-nct-lora \
+	--max_train_samples 9000 --max_val_samples 1000 --max_test_samples 1000
+

tests/test_integration.py

@@ -0,0 +1,180 @@
+#!/usr/bin/env python3
+"""
+MedGemma 集成测试脚本
+验证 MedGemma 是否正确集成到 OneScience 中
+"""
+
+import os
+import sys
+
+# 添加项目路径
+project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", ".."))
+sys.path.insert(0, os.path.join(project_root, "src"))
+
+print("=" * 60)
+print("MedGemma Integration Test")
+print("=" * 60)
+
+# 测试 1: 导入模块
+print("\n[Test 1] Importing modules...")
+try:
+    from models import MedGemma, VLLMModelRunner, TransformersModelRunner
+    from models.config import parse_configs, load_config
+    from onescience.datapipes.biology.adapters.medgemma_infer_adapter import MedGemmaInferAdapter
+    from onescience.datapipes.medical import ChatFormatter, DICOMLoader, MedicalImageProcessor
+    print("✓ All modules imported successfully")
+except ImportError as e:
+    print(f"✗ Import failed: {e}")
+    sys.exit(1)
+
+# 测试 2: 配置解析
+print("\n[Test 2] Testing configuration parsing...")
+try:
+    from configs.configs_base import medgemma_base_configs
+    from ml_collections import ConfigDict
+
+    # 创建测试配置
+    test_config = medgemma_base_configs.copy()
+    test_config.update({
+        "run_name": "integration_test",
+        "base_dir": "/tmp/medgemma_test",
+        "model": {
+            "variant": "4b",
+            "model_path": "/tmp/fake_model",
+        },
+        "output": {
+            "dump_dir": "/tmp/medgemma_output"
+        }
+    })
+
+    configs = parse_configs(test_config, fill_required_with_null=True)
+    print(f"✓ Configuration parsed successfully")
+    print(f"  - Model variant: {configs.model.variant}")
+    print(f"  - GPU memory utilization: {configs.inference.gpu_memory_utilization}")
+except Exception as e:
+    print(f"✗ Configuration parsing failed: {e}")
+    sys.exit(1)
+
+# 测试 3: 数据适配器
+print("\n[Test 3] Testing data adapter...")
+try:
+    adapter = MedGemmaInferAdapter()
+
+    # 测试文本适配
+    sample = {
+        "text": "What are the symptoms of diabetes?",
+        "max_tokens": 500,
+    }
+
+    features = adapter.adapt_features(sample)
+    print("✓ Data adapter working")
+    print(f"  - Messages: {len(features['messages'])} message(s)")
+    print(f"  - Parameters: {features['parameters']}")
+except Exception as e:
+    print(f"✗ Data adapter failed: {e}")
+    sys.exit(1)
+
+# 测试 4: Chat 格式化器
+print("\n[Test 4] Testing chat formatter...")
+try:
+    formatter = ChatFormatter()
+
+    messages = formatter.format_medical_query(
+        question="What causes hypertension?",
+        patient_info={"age": 65, "gender": "male"},
+    )
+
+    print("✓ Chat formatter working")
+    print(f"  - Formatted {len(messages)} message(s)")
+except Exception as e:
+    print(f"✗ Chat formatter failed: {e}")
+    sys.exit(1)
+
+# 测试 5: 医学图像处理器
+print("\n[Test 5] Testing medical image processor...")
+try:
+    import numpy as np
+
+    processor = MedicalImageProcessor(target_size=(224, 224))
+
+    # 创建假 CT 图像
+    fake_ct = np.random.randint(-1000, 400, (512, 512), dtype=np.int16)
+
+    processed = processor.process_ct_image(fake_ct)
+
+    print("✓ Medical image processor working")
+    print(f"  - Output shape: {processed.shape}")
+    print(f"  - Output dtype: {processed.dtype}")
+except Exception as e:
+    print(f"✗ Medical image processor failed: {e}")
+    import traceback
+    traceback.print_exc()
+
+# 测试 6: DICOM 加载器
+print("\n[Test 6] Testing DICOM loader...")
+try:
+    loader = DICOMLoader()
+
+    if loader.pydicom_available:
+        print("✓ DICOM loader initialized (pydicom available)")
+    else:
+        print("⚠ DICOM loader initialized (pydicom not available)")
+        print("  Install pydicom for DICOM support: pip install pydicom")
+except Exception as e:
+    print(f"✗ DICOM loader failed: {e}")
+
+# 测试 7: 模型运行器检查
+print("\n[Test 7] Checking model runners...")
+try:
+    # 检查 vLLM
+    try:
+        import vllm
+        print("✓ vLLM is available")
+        vllm_available = True
+    except ImportError:
+        print("⚠ vLLM not available (install with: pip install vllm)")
+        vllm_available = False
+
+    # 检查 transformers
+    try:
+        import transformers
+        print("✓ Transformers is available")
+    except ImportError:
+        print("✗ Transformers not available (required)")
+        sys.exit(1)
+except Exception as e:
+    print(f"✗ Model runner check failed: {e}")
+
+# 测试 8: 推理运行器
+print("\n[Test 8] Testing inference runner initialization...")
+try:
+    from runner import MedicalInferenceRunner
+
+    # 注意：不实际初始化模型（因为需要真实模型文件）
+    print("✓ MedicalInferenceRunner imported successfully")
+    print("  (Skipping actual model initialization - requires real model files)")
+except Exception as e:
+    print(f"✗ Inference runner import failed: {e}")
+    sys.exit(1)
+
+# 总结
+print("\n" + "=" * 60)
+print("Integration Test Summary")
+print("=" * 60)
+print("✓ Core functionality: PASSED")
+print("✓ Configuration system: PASSED")
+print("✓ Data adapters: PASSED")
+print("✓ Medical data processing: PASSED")
+
+if vllm_available:
+    print("✓ Performance optimization (vLLM): AVAILABLE")
+else:
+    print("⚠ Performance optimization (vLLM): NOT AVAILABLE")
+    print("  Consider installing: pip install vllm")
+
+print("\n✅ MedGemma integration test completed successfully!")
+print("\nNext steps:")
+print("  1. Download a MedGemma model")
+print("  2. Configure configs/inference_config.yaml")
+print("  3. Run: python -m runner.medical_inference_runner --config configs/inference_config.yaml --interactive")
+print("=" * 60)