Add model card and metadata for Heima

README.md

@@ -1,4 +1,33 @@
 ---
 base_model:
 - meta-llama/Llama-3.2-11B-Vision-Instruct
----
+library_name: transformers
+pipeline_tag: image-text-to-text
+license: llama3.2
+---
+
+# Heima: Efficient Reasoning with Hidden Thinking
+
+Heima (short for "hidden llama") is a Chain-of-Thought (CoT) compression framework designed for Multimodal Large Language Models (MLLMs). It condenses lengthy textual reasoning into a small set of abstract "thinking tokens," preserving essential reasoning capabilities while significantly improving inference efficiency.
+
+- **Paper:** [Efficient Reasoning with Hidden Thinking](https://huggingface.co/papers/2501.19201)
+- **Repository:** [https://github.com/shawnricecake/heima](https://github.com/shawnricecake/heima)
+
+## Model Description
+The Heima framework addresses the redundancy and verbosity of traditional textual CoT. By training the model to utilize latent thinking tokens, the Heima Encoder can maintain high problem-solving accuracy while reducing the number of generated tokens. 
+
+This repository contains the weights for the Heima Encoder, based on the Llama-3.2-11B-Vision-Instruct architecture. To reconstruct the reasoning process into human-readable text, an associated Heima Decoder (interpreter) can be used to map the thinking tokens back into textual sequences.
+
+## Performance
+Experiments across diverse reasoning benchmarks demonstrate that Heima improves reasoning efficiency while maintaining or even achieving better zero-shot accuracy compared to standard verbose CoT methods.
+
+## Citation
+If you find Heima useful for your research, please cite:
+```bibtex
+@article{shen2025efficient,
+  title={Efficient Reasoning with Hidden Thinking},
+  author={Shen, Xuan and Wang, Yizhou and Shi, Xiangxi and Wang, Yanzhi and Zhao, Pu and Gu, Jiuxiang},
+  journal={arXiv preprint arXiv:2501.19201},
+  year={2025}
+}
+```