Upload folder using huggingface_hub

README.md

@@ -5,6 +5,7 @@ library_name: transformers
 base_model:
   - OpenGVLab/InternViT-6B-448px-V1-2
   - NousResearch/Nous-Hermes-2-Yi-34B
+new_version: OpenGVLab/InternVL2_5-38B
 base_model_relation: merge
 language:
   - multilingual
@@ -19,16 +20,20 @@ tags:
 
 # InternVL-Chat-V1-2
 
-[\[📂 GitHub\]](https://github.com/OpenGVLab/InternVL)  [\[🆕 Blog\]](https://internvl.github.io/blog/)  [\[📜 InternVL 1.0 Paper\]](https://arxiv.org/abs/2312.14238)  [\[📜 InternVL 1.5 Report\]](https://arxiv.org/abs/2404.16821)
+[\[📂 GitHub\]](https://github.com/OpenGVLab/InternVL)  [\[🆕 Blog\]](https://internvl.github.io/blog/)  [\[📜 InternVL 1.0\]](https://arxiv.org/abs/2312.14238)  [\[📜 InternVL 1.5\]](https://arxiv.org/abs/2404.16821) [\[📜 Mini-InternVL\]](https://arxiv.org/abs/2410.16261)
 
 [\[🗨️ Chat Demo\]](https://internvl.opengvlab.com/)  [\[🤗 HF Demo\]](https://huggingface.co/spaces/OpenGVLab/InternVL)  [\[🚀 Quick Start\]](#quick-start)  [\[📖 中文解读\]](https://zhuanlan.zhihu.com/p/706547971)  [\[📖 Documents\]](https://internvl.readthedocs.io/en/latest/)
 
+<div align="center">
+  <img width="500" alt="image" src="https://cdn-uploads.huggingface.co/production/uploads/64006c09330a45b03605bba3/zJsd2hqd3EevgXo6fNgC-.png">
+</div>
+
 ## Introduction
 
 We are excited to introduce [🤗 InternVL-Chat-V1-2](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-2). Inspired by [LLaVA-NeXT-34B](https://llava-vl.github.io/blog/2024-01-30-llava-next/), we have also adopted [Nous-Hermes-2-Yi-34B](https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B) as the language model. Below is the pipeline.
 
 <p align="center">
-    <img src="https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/GIEKCvNc1Y5iMQqLv645p.png" style="width: 100%;">
+  <img width="600" alt="image" src="https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/GIEKCvNc1Y5iMQqLv645p.png">
 </p>
 
 From the experimental results, we've observed that **a stronger language model (34B) can better leverage the powerful capabilities of our vision foundation model.**
@@ -100,7 +105,7 @@ We provide an example code to run InternVL-Chat-V1-2 using `transformers`.
 
 We also welcome you to experience the InternVL2 series models in our [online demo](https://internvl.opengvlab.com/).
 
-> Please use transformers==4.37.2 to ensure the model works normally.
+> Please use transformers>=4.37.2 to ensure the model works normally.
 
 ### Model Loading
 
@@ -455,7 +460,7 @@ print(f'User: {question}')
 print(f'Assistant: {response}')
 ```
 
-#### Streaming output
+#### Streaming Output
 
 Besides this method, you can also use the following code to get streamed output.
 
@@ -493,6 +498,12 @@ This project is released under the MIT license. Parts of this project contain co
 If you find this project useful in your research, please consider citing:
 
 ```BibTeX
+@article{gao2024mini,
+  title={Mini-internvl: A flexible-transfer pocket multimodal model with 5\% parameters and 90\% performance},
+  author={Gao, Zhangwei and Chen, Zhe and Cui, Erfei and Ren, Yiming and Wang, Weiyun and Zhu, Jinguo and Tian, Hao and Ye, Shenglong and He, Junjun and Zhu, Xizhou and others},
+  journal={arXiv preprint arXiv:2410.16261},
+  year={2024}
+}
 @article{chen2023internvl,
   title={InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks},
   author={Chen, Zhe and Wu, Jiannan and Wang, Wenhai and Su, Weijie and Chen, Guo and Xing, Sen and Zhong, Muyan and Zhang, Qinglong and Zhu, Xizhou and Lu, Lewei and Li, Bin and Luo, Ping and Lu, Tong and Qiao, Yu and Dai, Jifeng},

configuration_internvl_chat.py

@@ -38,11 +38,11 @@ class InternVLChatConfig(PretrainedConfig):
         super().__init__(**kwargs)
 
         if vision_config is None:
-            vision_config = {}
+            vision_config = {'architectures': ['InternVisionModel']}
             logger.info('vision_config is None. Initializing the InternVisionConfig with default values.')
 
         if llm_config is None:
-            llm_config = {}
+            llm_config = {'architectures': ['LlamaForCausalLM']}
             logger.info('llm_config is None. Initializing the LlamaConfig config with default values (`LlamaConfig`).')
 
         self.vision_config = InternVisionConfig(**vision_config)

modeling_intern_vit.py

@@ -3,6 +3,7 @@
 # Copyright (c) 2024 OpenGVLab
 # Licensed under The MIT License [see LICENSE for details]
 # --------------------------------------------------------
+
 from typing import Optional, Tuple, Union
 
 import torch