---
library_name: vllm
license: other
tags:
  - foundation
  - amd
  - rocm
  - text-generation
pipeline_tag: text-generation
---

![](https://huggingface.co/AMD-PAVS-AI/facebook-opt/resolve/main/facebook-opt.webp)

# OPT: Optimized for AMD ROCm

OPT is Meta's causal language model family, served via [vLLM](https://github.com/vllm-project/vllm)'s OpenAI-compatible HTTP API. This repository packages evaluation/inference for causal language modeling using vLLM, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs.

This is based on the implementation of OPT found [here](https://huggingface.co/facebook/opt-1.3b).
This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [facebook-opt AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/facebook-opt) to reproduce results or export with custom configurations. More details on model performance can be found [here](#accuracy-pipeline).

---

## Task Overview

**Task:** Causal language modeling (text generation)

**Dataset:** HellaSwag, ARC-Challenge, Winogrande, MMLU, GSM8K (via `lm-evaluation-harness`)

**Output metrics:** `acc` / `acc_norm` / `exact_match` per task (not perplexity)

> **Model variants:** Default is **opt-1.3b** (`facebook/opt-1.3b`). Override with `MODEL_SIZE=125m|350m|1.3b|2.7b|6.7b`. Only one variant's weights live on disk at a time; downloading another deletes the previous one's. All OPT variants are not gated on Hugging Face — no token or license acceptance required. All variants share OPT's 2048-token positional-embedding ceiling.

> **vLLM note:** This version of vLLM does not work on kraken2e devices — ROCm has no supporting kernels for gfx1153.

---

## AMD ROCm Optimization

This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**. Key points:

- Validated backends: **vLLM** (ROCm-enabled, OpenAI-compatible HTTP API).
- No code changes required versus the upstream OPT implementation — only environment/runtime configuration differs.
- Not supported on kraken2e devices (gfx1153) — ROCm lacks supporting kernels for this vLLM version.

| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| GPU | — | vLLM | AMD GPU (ROCm) | OpenAI-compatible HTTP API |

---

## Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the [facebook-opt on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/facebook-opt).

---

## Model Details

**Model Type:** Causal language model (decoder-only transformer)

**Base Model:** `facebook/opt-1.3b` (default) — `opt-125m`, `opt-350m`, `opt-2.7b`, `opt-6.7b` also supported

**Model Stats:**
- Model variant: opt-1.3b (default) — opt-125m, opt-350m, opt-2.7b, opt-6.7b also supported
- Positional-embedding ceiling: 2048 tokens (shared across all variants)
- Number of parameters: `1.3B` (default variant; 125M/350M/2.7B/6.7B also available)

---

## Accuracy Pipeline

Accuracy is measured with [EleutherAI's lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) against the server's OpenAI-compatible `/v1/completions` endpoint. Few-shot counts follow the Open LLM Leaderboard convention.

### Metrics Explained

| Metric | Description |
|--------|-------------|
| acc / acc_norm | Accuracy on multiple-choice tasks (HellaSwag, ARC-Challenge, Winogrande, MMLU); `acc_norm` length-normalizes answer choices. |
| exact_match | Exact-match accuracy for generative tasks (GSM8K). |

### Accuracy Results

**Measured results (LIMIT=100 smoke-test subset)** — from the committed `METRICS_SUMMARY.md`, last updated 2026-05-11; run `make eval-accuracy-all-slow` and `make metrics` to refresh:

<!-- accuracy-table-start -->
| Variant | Task | Metric | Value |
|---------|------|--------|-------|
| opt-125m | arc_challenge | acc / acc_norm | 0.1900 / 0.2100 |
| opt-125m | hellaswag | acc / acc_norm | 0.3600 / 0.4200 |
| opt-125m | winogrande | acc | 0.5300 |
| opt-125m | mmlu | acc | 0.2616 |
| opt-125m | gsm8k | exact_match | 0.0000 |
| opt-350m | arc_challenge | acc / acc_norm | 0.1700 / 0.2200 |
| opt-350m | hellaswag | acc / acc_norm | 0.4100 / 0.4700 |
| opt-350m | winogrande | acc | 0.5800 |
<!-- accuracy-table-end -->

opt-1.3b (default), opt-2.7b, and opt-6.7b had not yet been run as of this writing (TBD in the source README).

---

## Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/facebook-opt)**

The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for benchmarking, model switching, and server management
- Additional model variants and datasets
- Benchmarking and reproduction instructions