Hugging Face's logo

nvidia
/
Eagle2.5-8B

Image-Text-to-Text
Transformers
Safetensors
multilingual
eagle_2_5_vl
feature-extraction
eagle
VLM
conversational
custom_code
Model card Files
xet
Community
14
Eagle2.5-8B / README.md
guochen1177's picture
guochen1177
Update README.md
9094acd verified
preview code
|
raw
history blame
20.7 kB
metadata 
license: cc-by-nc-4.0
pipeline_tag: image-text-to-text
library_name: transformers
base_model:
  - Qwen/Qwen2.5-7B-Instruct
  - google/siglip-so400m-patch14-384
base_model_relation: merge
language:
  - multilingual
tags:
  - eagle
  - VLM

Eagle 2.5

[📂 GitHub] [📜 Eagle 2.5 Tech Report] [🗨️ Chat Demo (NVIDIA Internal Network)]

Introduction

Eagle 2.5 is a family of frontier vision-language models (VLMs) designed for long-context multimodal learning. While most existing VLMs focus on short-context tasks, Eagle 2.5 addresses the challenges of long video comprehension and high-resolution image understanding, providing a generalist framework for both. The Eagle 2.5 training framework introduces two key techniques—Automatic Degrade Sampling (ADS) and Image Area Preservation (IAP)—to preserve contextual integrity and visual details. Additionally, the training pipeline is optimized for efficient long-context data training.

A major contribution of Eagle 2.5 is the introduction of Eagle-Video-110K, a novel dataset with both story-level and clip-level annotations, specifically curated for long video understanding. Eagle 2.5 demonstrates substantial improvements on long-context multimodal benchmarks, offering a robust solution to the limitations of existing VLMs. Notably, Eagle 2.5-8B achieves 72.4% on Video-MME with 512 input frames, matching the results of top-tier commercial models such as GPT-4o and large-scale open-source models like Qwen2.5-VL-72B and InternVL2.5-78B, despite having significantly fewer parameters.

Key Innovations

  • Information-First Sampling:
    • Image Area Preservation (IAP): Optimizes image tiling to retain most of the original image area and aspect ratio, preserving fine-grained details.
    • Automatic Degrade Sampling (ADS): Dynamically balances visual and textual input, ensuring complete text retention while maximizing visual content within context length constraints.
  • Progressive Mixed Post-Training:
    • Gradually increases context length during training, enhancing the model's ability to process varying input sizes and improving information density over static sampling.
  • Diversity-Driven Data Recipe:
    • Combines open-source data (human-annotated and synthetic) with the self-curated Eagle-Video-110K dataset, collected via a diversity-driven strategy and annotated with both story-level and clip-level QA pairs.

Model Details

  • Model Type: Long-context vision-language model
  • Architecture:
    • Vision encoder: SigLIP (with tiling strategies)
    • Language model: Qwen2.5 series
    • Multimodal connector: MLP projection
  • Supported Inputs:
    • Long video sequences (up to 512 frames)
    • High-resolution images
    • Multi-page documents
    • Long text
  • Training Strategy:
    • Progressive mixed post-training, expanding from 32K to 128K context length
    • Information-first sampling for optimal visual and textual information retention
  • Training Data:
    • Open-source video and document datasets
    • Eagle-Video-110K (110K long videos with dual-level annotation)

Released Models

Model Date Download Link Notes
Eagle2.5-8B 2025.04.16 HF link Long video (512 frames), high-res support

Video Benchmarks

Benchmark GPT-4o Gemini-1.5 Pro InternVL2.5-8B Qwen2.5-VL-8B Eagle2.5-8B
MVBenchtest - - 72.0 69.6 74.8
Perception_testval - - - 70.5 82.0
EgoSchemafullset - 72.2 - 65.0 72.2
MMB-Video 1.63 1.30 1.68 1.79 1.94
MLVUval - - 68.9 70.2 77.6
LVBenchval 66.7 64.0 60.0 56.0 66.4
Video-MMEw/o subtitle 71.9 75.0 64.2 65.1 72.4
Video-MMEw subtitle 77.2 81.3 66.9 71.6 75.7
CG-BenchClue 58.6 50.9 - 44.5 55.8
CG-BenchLong 44.9 37.8 - 35.5 46.6
CG-BenchmIoU 5.73 3.85 - 2.48 13.4
HourVideoDev - 37.2 - - 44.5
HourVideoTest - 37.4 - - 41.8
Charade-STAmIoU 35.7 - - 43.6 65.9
HD-EPIC - 37.6 - - 42.9
HRVideoBench - - - - 68.5
EgoPlanval - - - - 45.3

Embodied Benchmarks

Benchmark GPT-4o Gemini-1.5 Pro InternVL2.5-8B Qwen2.5-VL-8B Eagle2.5-8B
OpenEQA - - - - 63.5
ERQA 47.0 41.8 - - 38.3
EgoPlanval - - - - 45.3

Image Benchmarks

Benchmark GPT-4o Gemini-1.5 Pro InternVL2.5-8B Qwen2.5-VL-8B Eagle2.5-8B
DocVQAtest 92.8 93.1 93.0 95.7 94.1
ChartQAtest 85.7 87.2 84.8 87.3 87.5
InfoVQAtest 79.2 81.0 77.6 82.6 80.4
TextVQAval 77.4 78.8 79.1 84.9 83.7
OCRBenchtest 736 754 822 864 869
MMstartest 64.7 59.1 62.8 63.9 66.2
RWQAtest 75.4 67.5 70.1 68.5 76.7
AI2Dtest 84.6 79.1 84.5 83.9 84.5
MMMUval 69.1 62.2 56.0 58.6 55.8
MMBench_V11test 83.1 74.6 83.2 82.6 81.7
MMVetGPT-4-Turbo 69.1 64.0 62.8 67.1 62.9
HallBenchavg 55.0 45.6 50.1 52.9 54.7
MathVistatestmini 63.8 63.9 64.4 68.2 67.8
Avg Score 74.9 71.7 73.1 75.6 75.6

All numbers are directly extracted from Table 2 and Table 3 of the Eagle 2.5 Tech Report.

Eagle 2.5-8B matches or surpasses the performance of much larger models on long-context video and image benchmarks.

Quick Start 

pip install transformers==4.51.0

single image 

from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch
model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://www.ilankelman.org/stopsigns/australia.jpg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)]
image_inputs, video_inputs = processor.process_vision_info(messages)
inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True)
inputs = inputs.to("cuda")
model = model.to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output_text = processor.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

stream generation 

from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel, AutoTokenizer
import torch

from transformers import TextIteratorStreamer
import threading


model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, attn_implementation='flash_attention_2', torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://www.ilankelman.org/stopsigns/australia.jpg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)]
image_inputs, video_inputs = processor.process_vision_info(messages)
inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True)
inputs = inputs.to("cuda")
model = model.to("cuda")

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

generation_kwargs = dict(
    **inputs,
    streamer=streamer,
    max_new_tokens=1024,
    do_sample=True,
    top_p=0.95,
    temperature=0.8
)
thread = threading.Thread(target=model.generate, kwargs=generation_kwargs)
thread.start()


for new_text in streamer:
    print(new_text, end="", flush=True)

multiple-images 

from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch
model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://www.ilankelman.org/stopsigns/australia.jpg",
            },
            {
                "type": "image",
                "image": "https://www.nvidia.com/content/dam/en-zz/Solutions/about-nvidia/logo-and-brand/01-nvidia-logo-vert-500x200-2c50-d@2x.png",
            },
            {"type": "text", "text": "Describe these two images."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)]
image_inputs, video_inputs = processor.process_vision_info(messages)
inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True)
inputs = inputs.to("cuda")
model = model.to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output_text = processor.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

single video 


from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch
model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "../Eagle2-8B/space_woaudio.mp4",
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)]
image_inputs, video_inputs, video_kwargs = processor.process_vision_info(messages, return_video_kwargs=True)

inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True, videos_kwargs=video_kwargs)
inputs = inputs.to("cuda")
model = model.to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output_text = processor.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

multieple videos 

from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch
model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "../Eagle2-8B/space_woaudio.mp4",
                "nframes": 10,
            },
            {
                "type": "video",
                "video": "../Eagle2-8B/video_ocr.mp4",
                "nframes": 10,
            },
            {"type": "text", "text": "Describe these two videos respectively."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)]
image_inputs, video_inputs, video_kwargs = processor.process_vision_info(messages, return_video_kwargs=True)
inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True, videos_kwargs=video_kwargs)
inputs = inputs.to("cuda")
model = model.to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output_text = processor.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

batch inference 

from PIL import Image
import requests
from transformers import AutoProcessor, AutoModel
import torch
model = AutoModel.from_pretrained("nvidia/Eagle-2.5-8B",trust_remote_code=True, torch_dtype=torch.bfloat16)
processor = AutoProcessor.from_pretrained("nvidia/Eagle-2.5-8B", trust_remote_code=True, use_fast=True)
processor.tokenizer.padding_side = "left"

messages1 = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://www.ilankelman.org/stopsigns/australia.jpg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

messages2 = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://www.nvidia.com/content/dam/en-zz/Solutions/about-nvidia/logo-and-brand/01-nvidia-logo-vert-500x200-2c50-d@2x.png",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text_list = [processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
) for messages in [messages1, messages2]]
image_inputs, video_inputs = processor.process_vision_info([messages1, messages2])
inputs = processor(text = text_list, images=image_inputs, videos=video_inputs, return_tensors="pt", padding=True)
inputs = inputs.to("cuda")
model = model.to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=1024)
output_text = processor.batch_decode(
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)

Citation

If you use Eagle 2.5 in your research, please cite:

@article{chen2025eagle2.5,
    title={Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Models},
    author={Chen, Guo and Li, Zhiqi and Wang, Shihao and Jiang, Jindong and Liu, Yicheng and Lu, Lidong and Huang, De-An and Byeon, Wonmin and Le, Matthieu and Ehrlich, Max and Lu, Tong and Wang, Limin and Catanzaro, Bryan and Kautz, Jan and Tao, Andrew and Yu, Zhiding and Liu, Guilin},
    journal={arXiv preprint arXiv:2025.xxxx.xxxxx},
year={2025}
}

Acknowledgements

We thank the contributors and collaborators for their valuable discussions and support, including NVIDIA infrastructure and research teams.

License/Terms of Use

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.