README.md

---
license: apache-2.0
---
# ZAYA1-74B-Preview

ZAYA1-74B-Preview is a mixture of experts language model with 4B active and 74B total parameters. This is a reasoning-base checkpoint which has not been tuned for chat or undergone RL post-training. ZAYA1-74B-Preview was trained end to end on AMD.

Learn more on our [blog](https://www.zyphra.com/post/zaya1-74b-preview).

## Quickstart
### Prerequisites
We recommend installing the following libraries in a fresh python environment (tested with python 3.12).

To use ZAYA1-74B-preview, install `zaya1-pr` branch from our fork of `vllm` library (the command will trigger a full build of vLLM from source):  
```bash
pip install "vllm @ git+https://github.com/Zyphra/vllm.git@zaya1-pr"
```

If you want to run in transformers, install `zaya1` branch from our fork of `transformers` library as well:  
```bash
pip install "transformers @ git+https://github.com/Zyphra/transformers.git@zaya1"
```

### Deployment
To start vLLM server, run the following command:
```bash
vllm serve Zyphra/ZAYA1-74B-Preview --port 8010 \
   --mamba-cache-dtype float32 --dtype bfloat16 \
   --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser zaya_xml
```

For parallel deployment we recommend using DP with EP as TP for CCA is not supported in the branch above. If running on 8 GPUs, set extra flags `-dp 8 -ep` to run with DP=EP=8.

Once the server is up, you can query a model with `curl` like in the following example:
```bash
curl http://localhost:8010/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Zyphra/ZAYA1-74B-Preview",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello. How is it going?"}
        ]
    }'
```