README.md

---
license: apache-2.0
pipeline_tag: any-to-any
tags:
- UniCharacter
- customized-multimodal-role-play
- multimodal-role-play
- character-customization
- text-to-image
- image-generation
---

# UniCharacter

UniCharacter is a collection of character-specific checkpoints for **Customized Multimodal Role-Play (CMRP)**, introduced in the paper [Towards Customized Multimodal Role-Play](https://arxiv.org/abs/2605.08129).

[**Project Page**](https://tangc03.github.io/UniCharacter.github.io/) | [**GitHub**](https://github.com/Tangc03/UniCharacter) | [**Paper**](https://arxiv.org/abs/2605.08129)

The model is designed to customize a character's persona, dialogue style, and visual identity so that the character can respond consistently across text and image generation settings. Using a unified multimodal model, UniCharacter employs a two-stage training framework containing Unified Supervised Finetuning (Unified-SFT) and character-specific group relative policy optimization (Character-GRPO).

## Repository Contents

This repository contains separate checkpoint folders for multiple characters. Each character directory includes sharded `safetensors` weights and a `model.safetensors.index.json` file.

Available character folders include:

- `Adrien_Brody`, `Bo`, `Butin`, `Chandler`, `Coco`, `Furina`, `Gao_Qiqiang`, `Hermione`, `Ichihime`, `Joey`, `Leonardo`, `Mam`, `Miki_Nikaidou`, `Mydieu`, `Pikachu`, `Rin_Tohsaka`, `Saber`, `Will_In_Vietnam`, `Wukong`, `YuiYagi`

## Quick Usage Example

To use these checkpoints, please follow the installation instructions in the [official repository](https://github.com/Tangc03/UniCharacter). Below is an example of the unified inference interface:

```python
from inference import create_unicharacter_inference
from pathlib import Path

# Initialize the unified inference (modify paths according to your environment)
inference = create_unicharacter_inference(
    model_path="models/BAGEL-7B-MoT",
    checkpoint_path="<checkpoint_path>",
    vit_checkpoint_path="<vit_checkpoint_path>",
    max_mem_per_gpu="40GiB",
    seed=42,
)

out_dir = Path("test_images/outputs")
out_dir.mkdir(parents=True, exist_ok=True)

# 1) Text-to-image generation (Role T2I)
res = inference.generate_image("Ichihime chasing a butterfly")
res["image"].save(out_dir / "t2i_ichihime.png")

# 2) Visual understanding / VQA
res = inference.visual_understanding(
    "data/personalized_data/train/Mahjong Soul-Ichihime/1.png",
    "What's the color of Ichihime's hair?",
)
print("VQA:", res["text"])

# 3) Knowledge QA
res = inference.knowledge_qa("When do you born?")
print("Knowledge QA:", res["text"])

# 4) Multimodal role-play
res = inference.role_play(
    character_name="Ichihime",
    description="",
    opening="",
    user_text="Hi, Ichihime. How are you?",
)
print("Role-play:", res["response"])
```

## Download

Download the full repository:

```bash
huggingface-cli download Tangc03/UniCharacter --local-dir UniCharacter
```

Download a single character checkpoint folder:

```bash
huggingface-cli download Tangc03/UniCharacter --include "Hermione/*" --local-dir UniCharacter
```

## Citation

If you use UniCharacter, please cite:

```bibtex
@article{tang2026towards,
  title={Towards Customized Multimodal Role-Play},
  author={Tang, Chao and Wu, Jianzong and Shi, Qingyu and Tian, Ye and Zhang, Aixi and Jiang, Hao and Zhang, Jiangning and Tong, Yunhai},
  journal={arXiv preprint arXiv:2605.08129},
  year={2026}
}
```