README.md

---
title: Light-MER
emoji: 🎭
colorFrom: yellow
colorTo: green
sdk: gradio
sdk_version: "5.23.0"
app_file: app.py
short_description: Sub-1B multimodal emotion recognition from video and audio
python_version: "3.10"
startup_duration_timeout: 10m
---

# Light-MER: Lightweight Multimodal Emotion Recognition

A sub-1B parameter multimodal emotion recognition model that processes video, audio, and text to recognize emotions.

## Model

- **Paper**: [Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?](https://arxiv.org/abs/2607.12787)
- **GitHub**: [GAIR-Lab/Light-MER](https://github.com/GAIR-Lab/Light-MER)
- **Model**: [kevin233333/Light-MER](https://huggingface.co/kevin233333/Light-MER)

## How it works

Light-MER uses a Qwen3-0.6B LLM backbone with CLIP-ViT-Base visual encoder and Chinese-HuBERT audio encoder.
The model processes video frames, audio clips, and optional subtitles to perform emotion recognition and reasoning.

## Usage

1. Upload a video file (with or without audio)
2. Optionally provide a separate audio file or subtitle text
3. Ask a question about the emotional state (e.g., "Please infer the person's emotional state and provide your reasoning process.")
4. Click "Recognize Emotion" to get the model's response