Llama 2 (7B): Optimized for AMD ROCm
Llama 2 (7B) is Meta's causal language model, served via vLLM's OpenAI-compatible HTTP API. This repository packages evaluation/inference for causal language modeling using vLLM, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.
This is based on the implementation of Llama 2 found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the Llama-2-7B-hf AMD scripts to reproduce results or export with custom configurations.
Task Overview
Task: Causal language modeling (text generation)
Dataset: HellaSwag, ARC-Challenge, Winogrande, MMLU, GSM8K (via lm-evaluation-harness)
Output metrics: acc / acc_norm / exact_match per task (not perplexity)
Model variants: Only the base 7B model is auto-bootstrapped.
vLLM note: This version of vLLM does not work on kraken2e devices — ROCm has no supporting kernels for gfx1153.
Gated model: Llama 2 weights cannot be downloaded until you accept Meta's license on Hugging Face and authenticate via
hf auth login.
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the Llama-2-7B-hf on GitHub.
Model Details
Model Type: Causal language model (decoder-only transformer)
Base Model: meta-llama/Llama-2-7b-hf
Performance Summary
No results have been recorded in this checkout yet — run make eval-accuracy-all-slow (or eval-accuracy-fast for a quicker subset), then make metrics to populate METRICS_SUMMARY.md. Meta's published numbers below are for sanity-checking a fresh run; values far outside these ranges signal a pipeline issue, not a model issue.
Expected Accuracy Ranges (Meta reference)
| Task | Metric | Expected |
|---|---|---|
| hellaswag (10-shot) | acc_norm |
0.76–0.78 |
| arc_challenge (25-shot) | acc_norm |
0.51–0.54 |
| winogrande (5-shot) | acc |
0.74–0.76 |
| mmlu (5-shot) | acc |
0.45–0.47 |
| gsm8k (5-shot) | exact_match |
0.12–0.15 |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for the supported runners
- Additional model variants and datasets
- Benchmarking and reproduction instructions