init

README.md

@@ -1,5 +1,5 @@
 ---
-license: mit
+license: cc-by-nc-4.0
 pipeline_tag: image-text-to-text
 library_name: transformers
 base_model:
@@ -16,7 +16,8 @@ tags:
 
 # Eagle-2
 
-[\[📂 GitHub\]](https://github.com/NVlabs/EAGLE)   [\[📜 Eagle 2\]](TODO)
+
+[\[📂 GitHub\]](https://github.com/NVlabs/EAGLE)   [\[📜 Eagle2 Tech Report\]](TODO)
 [\[🗨️ Chat Demo\]](http://eagle-vlm.xyz/)  [\[🤗 HF Demo\]](TODO)  
 ## Introduction
 
@@ -57,7 +58,7 @@ We provide the following models:
 |     AI2D<sub>test</sub>      |         57.1         |        64.1        |     69.3    | 74.7 |70.9|
 |      MMMU<sub>val</sub>      |          31.4       |    36.7     | 40.9  |41.1|38.8|
 | MMVet<sub>GPT-4-Turbo</sub>  |         32.2       |        32.7       |    48.8    | 49.5|40.9|             HallBench<sub>avg</sub>    |         27.9      |        34.0       |     39.0     |**41.7**|35.3
-| MathVista<sub>testmini</sub> |         3.8         |        37.7        |     43.2     |43.0|45.3|
+| MathVista<sub>testmini</sub> |         33.8         |        37.7        |     43.2     |43.0|45.3|
 | MMstar |             37.7    |       45.7      |     50.1|48.0|48.5|
 
 
@@ -66,7 +67,7 @@ We provide the following models:
 
 
 
-We provide a [demo inference script](./demo.py) to help you quickly start using the model. We support different input types: 
+We provide a [inference script](./demo.py) to help you quickly start using the model. We support different input types: 
 - pure text input
 - single image input
 - multiple image input

demo.py

@@ -390,7 +390,7 @@ class ModelWorker:
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser()
-    parser.add_argument('--model-path', type=str, default='/home/zhidingy/workspace/eagle-next/internvl_chat/work_dirs/release/Eagle2-1B')
+    parser.add_argument('--model-path', type=str, default='nvidia/Eagle2-1B')
     parser.add_argument('--model-name', type=str, default='Eagle2-1B')
     parser.add_argument('--device', type=str, default='cuda')
     parser.add_argument('--load-8bit', action='store_true')

modeling_eagle_chat.py

@@ -25,6 +25,9 @@ from .flash_attention import *
 from .multi_backbone_channel_concatentation_model import MultiBackboneChannelConcatenationVisionModel
 from .multi_backbone_channel_concatenation_encoder import MultiBackboneChannelConcatenationVisionTower
 from .configuration_multi_backbone_channel_concatentation_model import MultiBackboneChannelConcatenationVisionModelConfig
+from .siglip_vision_tower import SiglipVisionTower
+from .convnext_encoder import ConvNextVisionTower
+from .convnext import ConvNeXt
 
 logger = logging.get_logger(__name__)