Update README.md

README.md

@@ -1,3 +1,29 @@
----
-license: apache-2.0
----
+# <img src="assets/icon.png" width="35" /> ReFocus
+
+This repo contains the model for the paper "ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding"
+
+[**🌐 Homepage**](https://zeyofu.github.io/ReFocus/) |[**📑 Paper**](https://arxiv.org/abs/2501.05452) |  [**🔗 Code**](https://github.com/zeyofu/ReFocus_Code)
+
+
+# Introduction
+
+![Alt text](assets/teaser.png)
+
+# ReFocus Finetuning
+We follow the [Phi-3 Cookbook](https://github.com/microsoft/Phi-3CookBook/blob/main/md/04.Fine-tuning/FineTuning_Vision.md) for the supervised finetuning experiments. 
+
+## Inference with the Finetuned Model
+We release our best finetuned ReFocus model with full chain-of-thought data in this [HuggingFace Link](https://huggingface.co/Fiaa/ReFocus).
+
+This model is finetuned based on Phi-3.5-vision, and we used the following prompt during evaluation
+```
+<|image|>\n{question}\nThought:
+```
+To enforce the model to generate bounding box coordinates to refocus, you could try this prompt:
+```
+<|image_1|>\n{question}\nThought: The areas to focus on in the image have bounding box coordinates:
+```
+
+---
+license: apache-2.0
+---