|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- speculators |
|
|
- eagle |
|
|
- speculative-decoding |
|
|
--- |
|
|
|
|
|
# Eagle Llama 3.1 8B Instruct |
|
|
|
|
|
This is a converted Eagle speculator checkpoint for Llama 3.1 8B Instruct, compatible with the [speculators](https://github.com/neuralmagic/speculators) library. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Original checkpoint**: [yuhuili/EAGLE-LLaMA3.1-Instruct-8B](https://huggingface.co/yuhuili/EAGLE-LLaMA3.1-Instruct-8B) |
|
|
- **Base model**: [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) |
|
|
- **Architecture**: EAGLE (standard, without layernorms) |
|
|
- **Size**: 481MB |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from speculators.models.eagle import EagleSpeculator |
|
|
from transformers import AutoModelForCausalLM |
|
|
|
|
|
# Load the Eagle speculator |
|
|
eagle_model = EagleSpeculator.from_pretrained("nm-testing/eagle-llama3.1-8b-instruct") |
|
|
|
|
|
# Attach verifier model |
|
|
verifier = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct") |
|
|
eagle_model.attach_verifier(verifier) |
|
|
|
|
|
# Use for speculative decoding |
|
|
# ... your speculative decoding code here ... |
|
|
``` |
|
|
|
|
|
## Conversion Details |
|
|
|
|
|
This checkpoint was converted using: |
|
|
```bash |
|
|
speculators convert --eagle \ |
|
|
yuhuili/EAGLE-LLaMA3.1-Instruct-8B \ |
|
|
./eagle-standard-converted \ |
|
|
meta-llama/Meta-Llama-3.1-8B-Instruct |
|
|
``` |
|
|
|
|
|
## License |
|
|
|
|
|
Apache 2.0 |
