README.md

---
license: apache-2.0
tags:
- speculators
- eagle
- speculative-decoding
---

# Eagle Llama 3.1 8B Instruct

This is a converted Eagle speculator checkpoint for Llama 3.1 8B Instruct, compatible with the [speculators](https://github.com/neuralmagic/speculators) library.

## Model Details

- **Original checkpoint**: [yuhuili/EAGLE-LLaMA3.1-Instruct-8B](https://huggingface.co/yuhuili/EAGLE-LLaMA3.1-Instruct-8B)
- **Base model**: [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
- **Architecture**: EAGLE (standard, without layernorms)
- **Size**: 481MB

## Usage

```python
from speculators.models.eagle import EagleSpeculator
from transformers import AutoModelForCausalLM

# Load the Eagle speculator
eagle_model = EagleSpeculator.from_pretrained("nm-testing/eagle-llama3.1-8b-instruct")

# Attach verifier model
verifier = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
eagle_model.attach_verifier(verifier)

# Use for speculative decoding
# ... your speculative decoding code here ...
```

## Conversion Details

This checkpoint was converted using:
```bash
speculators convert --eagle \
  yuhuili/EAGLE-LLaMA3.1-Instruct-8B \
  ./eagle-standard-converted \
  meta-llama/Meta-Llama-3.1-8B-Instruct
```

## License

Apache 2.0