| --- |
| license: mit |
| language: |
| - en |
| - zh |
| tags: |
| - multimodal |
| - sentiment-analysis |
| - multimodal-sentiment-analysis |
| - missing-modality |
| datasets: |
| - Bluexxx/SemMSA-data |
| base_model: |
| - Qwen/Qwen3-1.7B |
| --- |
| |
| # SemMSA checkpoints |
|
|
| Checkpoints of **SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Data** |
| ([paper](https://arxiv.org/abs/2609.30238), [code](https://github.com/peacelwh/SemMSA), |
| [project page](https://peacelwh.github.io/SemMSA/), [data](https://huggingface.co/datasets/Bluexxx/SemMSA-data)). |
|
|
| SemMSA constructs sentiment-relevant latent semantics with a frozen LLM (Qwen3-1.7B) through Cross-modal Semantic |
| Refinement (CSR) and aligns them with language, visual and acoustic representations through Cross-modal Spectral |
| Alignment (CSA). |
|
|
| ## Released checkpoints |
|
|
| Checkpoints for **CH-SIMS** are released; checkpoints for MOSI and MOSEI will be added later. |
|
|
| ``` |
| <dataset>/<config>/best_<metric>_<seed>.pth |
| ``` |
|
|
| `<config>` is the training configuration `configs/<dataset>/<config>.yaml` of the code repository. Following the |
| evaluation protocol of LNLN and P-RMF, one checkpoint is kept for each metric; `configs/<dataset>/eval.yaml` lists the |
| checkpoint used for every reported metric. |
|
|
| | Config | Files (seeds 1111, 1112, 1113) | Used for | |
| |---|---|---| |
| | `sims/main` | `best_Mult_acc_2_*.pth`, `best_MAE_*.pth` | Acc-2, Corr | |
| | `sims/f1` | `best_Mult_acc_2_*.pth` | F1 | |
| | `sims/reg` | `best_Mult_acc_3_*.pth`, `best_MAE_*.pth` | Acc-3, MAE | |
| | `sims/acc5` | `best_Mult_acc_5_*.pth` | Acc-5 | |
|
|
| The files contain only the trainable weights: the frozen Qwen3-1.7B (and BERT, when it is frozen) are loaded from |
| `pretrained/`. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/peacelwh/SemMSA.git && cd SemMSA |
| pip install -r requirements.txt && pip install -e . |
| bash scripts/download_pretrained.sh |
| bash scripts/download_data.sh sims |
| bash scripts/download_checkpoints.sh sims # this repository -> checkpoints/ |
| bash scripts/evaluate.sh sims 0 # robust evaluation under missing rates 0.0-0.9 |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{li2026semmsa, |
| title = {SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Data}, |
| author = {Li, Wenhao and Wu, Zhibin and Xiao, Chong and Wang, Qiangchang}, |
| journal = {arXiv preprint arXiv:2609.30238}, |
| year = {2026} |
| } |
| ``` |
|
|
