Upload folder using huggingface_hub

README.md

@@ -1,9 +1,20 @@
 ---
 license: mit
 pipeline_tag: image-text-to-text
+library_name: transformers
 base_model: 
-  - OpenGVLab/InternViT-6B-448px
+  - OpenGVLab/InternViT-6B-448px-V1-0
   - meta-llama/Llama-2-13b-hf
+base_model_relation: merge
+language:
+  - multilingual
+tags:
+  - internvl
+  - vision
+  - ocr
+  - multi-image
+  - video
+  - custom_code
 ---
 
 # InternVL-Chat-V1-1
@@ -15,7 +26,7 @@ base_model:
 ## Introduction
 
 We released [🤗 InternVL-Chat-V1-1](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-1), featuring a structure similar to LLaVA, including a ViT, an MLP projector, and an LLM.
-As shown in the figure below, we connected our InternViT-6B to LLaMA2-13B through a simple MLP projector. Note that the LLaMA2-13B used here is not the original model but an internal chat version obtained by incrementally pre-training and fine-tuning the LLaMA2-13B base model for Chinese language tasks. Overall, our model has a total of 19 billion parameters.
+As shown in the figure below, we connected our [InternViT-6B](https://huggingface.co/OpenGVLab/InternViT-6B-448px) to LLaMA2-13B through a simple MLP projector. Note that the LLaMA2-13B used here is not the original model but an internal chat version obtained by incrementally pre-training and fine-tuning the LLaMA2-13B base model for Chinese language tasks. Overall, our model has a total of 19 billion parameters.
 
 <p align="center">
     <img src="https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/HD29tU-g0An9FpQn1yK8X.png" style="width: 100%;">
@@ -49,12 +60,12 @@ This model can also conduct an in-depth analysis of AAAI's official website and
 
 - **Training Strategy:**
 
-  - Pretraining Stage
-    - Learnable Component: InternViT-6B + LLaMA2-13B
+  - Pre-training Stage
+    - Learnable Component: ViT + MLP
     - Data: Trained on 72M samples, including COYO, LAION, CC12M, CC3M, SBU, Wukong, GRIT, Objects365, OpenImages, and OCR-related datasets.
-    - Note: In this stage, we load the pretrained weights of the original [InternViT-6B-224px](https://huggingface.co/OpenGVLab/InternViT-6B-224px) and interpolate its position embedding to the size corresponding to 448 x 448 pixels. Moreover, in order to reduce the number of visual tokens, we use a pixel shuffle operation to reduce 1024 tokens to 256 tokens.
-  - Supervised Finetuning Stage
-    - Learnable Component: MLP + LLaMA2-13B
+    - Note: In this stage, we load the pretrained weights of the original [InternViT-6B-224px](https://huggingface.co/OpenGVLab/InternViT-6B-224px) and interpolate its position embedding to the size corresponding to 448 x 448 pixels. Moreover, in order to reduce the number of visual tokens, we use a pixel shuffle (unshuffle) operation to reduce 1024 tokens to 256 tokens.
+  - Supervised Fine-tuning Stage
+    - Learnable Component: MLP + LLM
     - Data: A comprehensive collection of open-source datasets, along with their Chinese translation versions, totaling approximately 6M samples.
 
 ## Performance
@@ -79,8 +90,6 @@ This model can also conduct an in-depth analysis of AAAI's official website and
 
 - Note that we use the [official evaluation server](https://huggingface.co/spaces/whyu/MM-Vet_Evaluator) to test the MMVet scores, with `GPT-4-0613` serving as the judge model. Using different versions of GPT-4 as the judge can result in significant score variations.
 
-Here, we have conducted only a simple performance comparison. For more detailed performance information and additional evaluation metrics, please refer to our performance summary table.
-
 ## Quick Start
 
 We provide an example code to run InternVL-Chat-V1-1 using `transformers`.