README.md

metadata

title: Light-MER
emoji: 🎭
colorFrom: yellow
colorTo: green
sdk: gradio
sdk_version: 5.23.0
app_file: app.py
short_description: Sub-1B multimodal emotion recognition from video and audio
python_version: '3.10'
startup_duration_timeout: 10m

Light-MER: Lightweight Multimodal Emotion Recognition

A sub-1B parameter multimodal emotion recognition model that processes video, audio, and text to recognize emotions.

Model

• Paper: Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
• GitHub: GAIR-Lab/Light-MER
• Model: kevin233333/Light-MER

How it works

Light-MER uses a Qwen3-0.6B LLM backbone with CLIP-ViT-Base visual encoder and Chinese-HuBERT audio encoder. The model processes video frames, audio clips, and optional subtitles to perform emotion recognition and reasoning.

Usage

1. Upload a video file (with or without audio)
2. Optionally provide a separate audio file or subtitle text
3. Ask a question about the emotional state (e.g., "Please infer the person's emotional state and provide your reasoning process.")
4. Click "Recognize Emotion" to get the model's response