EEG-Captioner

This repository contains the portable inference weights for EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties.

It includes a Dual-Domain EEG Encoder, an EEG-to-language projector, and a LoRA adapter for the Qwen2.5-Omni-7B Thinker. It does not include the Qwen base model, TUEG recordings, patient data, optimizer states, or annotation-service credentials.

Files

encoder.pt                         Dual-Domain EEG Encoder
eeg_projector.pt                   EEG token-to-Qwen projector
qwen_lora/adapter_model.safetensors
qwen_lora/adapter_config.json      Qwen Thinker LoRA
qwen_lora/tokenizer.json
qwen_lora/tokenizer_config.json
qwen_lora/chat_template.jinja
bundle.json                        Architecture metadata and SHA-256 hashes

The bundle is approximately 66 MiB. The base model must be obtained separately from Qwen/Qwen2.5-Omni-7B.

Code and installation

Source code, preprocessing, training configurations, evaluation, and validation notes are available at:

https://github.com/lyJinAoki/EEG-Captioner

git clone https://github.com/lyJinAoki/EEG-Captioner.git
cd EEG-Captioner
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[llm,preprocess]'

Download this model repository into checkpoints/final and point QWEN_PATH to a local Qwen2.5-Omni-7B snapshot:

hf download LyJinAoki/EEG-Captioner --local-dir checkpoints/final
export QWEN_PATH=/path/to/Qwen2.5-Omni-7B

eeg-captioner infer \
  --bundle checkpoints/final \
  --qwen-path "$QWEN_PATH" \
  --eeg-h5 data/processed/example.h5 \
  --trial 0 \
  --start-sample 0

For signal-level QA, add --question 'YOUR QUESTION'. See the GitHub README for EDF preprocessing and supported questions.

Input contract

• One 10-second segment sampled at 200 Hz.
• Preprocessed H5 signals are stored in microvolts.
• Fixed channel order: FP1 FP2 F3 F4 C3 C4 P3 P4 O1 O2 OZ F7 F8 T3 T4 T5 T6 FZ CZ PZ.
• A missing OZ may be zero-filled; all other channels are required.
• The packaged loader applies the historical /100 model scaling once.

Use the preprocessing implementation in the associated GitHub repository to avoid introducing a different channel, unit, filtering, or segmentation convention.

Training and evaluation provenance

The adapter is the final checkpoint of the mainline sequence:

1. Dual-Domain classifier pretraining.
2. Signal-level QA language adaptation.
3. Joint QA/caption stages with caption fractions 0.25, 0.75, and 0.90.

The historical research report associated with this checkpoint recorded QA Macro-BA 88.81%, ROUGE-L 0.4865, and BLEU-4 0.3302. These are provenance values, not scores recomputed under the cleaner independent split protocol in the public code. Historical QA model selection and reporting used a combined validation/test diagnostic set; consult the repository's reproduction notes before comparing results.

The original external GPT factual-consistency judgments are not included, and no replacement factual-consistency score is claimed.

Validation

The portable bundle was strictly loaded with the public code and run on an NVIDIA A100-PCIE-40GB using a real preprocessed EEG segment. It produced a non-empty caption and QA answer, with finite encoder output. Synthetic tests also cover preprocessing, feature extraction, the full short training chain, evaluation, adapter save/reload, and two-GPU distributed execution.

No EEG recordings, generated patient descriptions, or subject identifiers are included in this model repository.

Limitations and responsible use

This is a research model for descriptions of measurable signal properties. It is not a medical device and must not be used for diagnosis, treatment decisions, emergency triage, or unsupervised clinical reporting. Outputs can be incorrect, incomplete, or linguistically plausible without being signal-faithful. Dataset coverage, preprocessing assumptions, filename-level splitting, and the external caption annotation model can introduce bias and limit generalization.

Users are responsible for obtaining lawful access to EEG data, protecting patient privacy, validating performance for their population and acquisition setup, and retaining qualified human oversight.

License

The files in this model repository are released under Apache License 2.0. Qwen2.5-Omni is an external dependency under its own Apache 2.0 license. TUEG data and MNE template/atlas assets are not included and retain their own terms.

Citation

Please cite the accompanying paper once its final bibliographic record is available:

Leyuan Jin, Jiahao Mei, Yixuan Yuan, Shimeng Li, Mengyue Wu, and Yue Ding. “EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties.”