File size: 1,809 Bytes
12820a1 4aa090b 12820a1 bc8ad03 12820a1 5ca4f54 12820a1 bc8ad03 12820a1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | ---
license: apache-2.0
language:
- en
base_model:
- stable-diffusion-v1-5/stable-diffusion-v1-5
- liuhaotian/llava-llama-2-13b-chat-lightning-preview
tags:
- Image-to-Image
- Action-Generation
- HOI
- Egocentric-Vision
- Vision-Language-Model
---
# LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning
### ECCV 2024 (Oral, Best Paper Finalist)
[Project Page](https://bolinlai.github.io/Lego_EgoActGen/) | [Paper](https://arxiv.org/pdf/2312.03849) | [Dataset](https://huggingface.co/datasets/bolinlai/LEGO-Dataset) | [Code](https://github.com/BolinLai/LEGO)
[Bolin Lai](https://bolinlai.github.io/), [Xiaoliang Dai](https://sites.google.com/view/xiaoliangdai/), [Lawrence Chen](https://www.lawrencechen.me/), [Guan Pang](https://scholar.google.com/citations?user=7v1LZxUAAAAJ&hl=en), [James M. Rehg](https://rehg.org/), [Miao Liu](https://aptx4869lm.github.io/)
<img src='https://bolinlai.github.io/Lego_EgoActGen/figures/visualization_new_actions.png'/>
This repo is the model weights finetuned on Epic-Kitchens for our paper "LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning". More repos are available in this [collection](https://huggingface.co/collections/bolinlai/lego-67b386cf642909c56776f754).
Please refer to the code on [github](https://github.com/BolinLai/LEGO) for detailed instructions on how to use it.
If you find LEGO useful for your work, please cite using this BibTeX.
```BibTex
@inproceedings{lai2024lego,
title={Lego: Learning egocentric action frame generation via visual instruction tuning},
author={Lai, Bolin and Dai, Xiaoliang and Chen, Lawrence and Pang, Guan and Rehg, James M and Liu, Miao},
booktitle={European Conference on Computer Vision},
pages={135--155},
year={2024},
organization={Springer}
}
``` |