Enhance model card with metadata, links, and usage for robotics model

README.md

@@ -1,10 +1,88 @@
 ---
+license: apache-2.0
+pipeline_tag: robotics
+library_name: lerobot
 tags:
 - model_hub_mixin
 - pytorch_model_hub_mixin
+- gaze-tracking
+- foveated-vision
+- robot-learning
 ---
 
-This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
-- Code: [More Information Needed]
-- Paper: [More Information Needed]
-- Docs: [More Information Needed]
+# Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers
+
+This repository contains a model related to the paper **"Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers"**.
+
+Human vision is a highly active process driven by gaze, which directs attention and fixation to task-relevant regions and dramatically reduces visual processing. This work explores how incorporating human-like active gaze into robotic policies can enhance both efficiency and performance. Our approach significantly reduces computational overhead and improves performance and robustness in robotic tasks by leveraging foveated image processing and foveated Vision Transformers.
+
+*   **Paper:** [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://huggingface.co/papers/2507.15833)
+*   **Project Website:** [https://ian-chuang.github.io/gaze-av-aloha/](https://ian-chuang.github.io/gaze-av-aloha/)
+*   **Code Repository:** [https://github.com/ian-chuang/gaze-av-aloha](https://github.com/ian-chuang/gaze-av-aloha)
+
+![hero](https://github.com/ian-chuang/gaze-av-aloha/raw/main/media/hero.gif)
+
+This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration.
+
+## Installation
+
+To set up the environment and install all necessary dependencies for the `gaze-av-aloha` project, follow these steps:
+
+```bash
+# Clone the repository and initialize submodules
+git clone https://github.com/ian-chuang/gaze-av-aloha.git
+cd gaze-av-aloha
+git submodule init
+git submodule update
+
+# Create and activate a new Conda environment
+conda create -n gaze python=3.10
+conda activate gaze
+
+# Install LeRobot
+pip install git+https://github.com/huggingface/lerobot.git@483be9aac217c2d8ef16982490f22b2ad091ab46
+
+# Install FFmpeg for video logging
+conda install ffmpeg=7.1.1 -c conda-forge
+
+# Install AV-ALOHA packages
+pip install -e ./gym_av_aloha
+pip install -e ./gaze_av_aloha
+```
+
+Make sure you're logged in to Hugging Face: `huggingface-cli login`
+
+## Usage (Training a policy)
+
+You can train and evaluate policies using the provided `train.py` script from the GitHub repository. Pretrained ViT weights and Gaze models are available on Hugging Face. An example for training the `Fov-Act` policy (end-to-end gaze as action) is shown below:
+
+```bash
+python gaze_av_aloha/scripts/train.py \
+  policy=foveated_vit_policy \
+  task=<task_name_e.g._av_aloha_sim_thread_needle> \
+  policy.vision_encoder_kwargs.repo_id=iantc104/mae_vitb_foveated_vit \
+  policy.optimizer_lr_backbone=1e-5 \
+  wandb.enable=true \
+  wandb.project=<your_project_name> \
+  wandb.entity=<your_wandb_entity> \
+  wandb.job_name=fov-act \
+  device=cuda
+```
+
+Replace `<task_name_e.g._av_aloha_sim_thread_needle>`, `<your_project_name>`, and `<your_wandb_entity>` with your desired values. For detailed instructions on available tasks, other policy configurations (e.g., Fov-UNet, Fine, Coarse), and how to use pretrained models, please refer to the [official GitHub repository](https://github.com/ian-chuang/gaze-av-aloha).
+
+## Citation
+
+If you find this work helpful or inspiring, please consider citing it:
+
+```bibtex
+@misc{chuang2025lookfocusactefficient,
+      title={Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers},
+      author={Ian Chuang and Andrew Lee and Dechen Gao and Jinyu Zou and Iman Soltani},
+      year={2025},
+      eprint={2507.15833},
+      archivePrefix={arXiv},
+      primaryClass={cs.RO},
+      url={https://arxiv.org/abs/2507.15833},
+}
+```