README.md

---
pipeline_tag: text-generation
license: other
license_name: tongyi-qianwen
license_link: https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE
base_model: Qwen/Qwen2-72B-Instruct
library_name: zeromodels
language:
- en
tags:
- keras
- zeromodels
- qwen2
- text-generation
- pytorch
- jax
- tf
---

Paper: [Qwen2 Technical Report (arXiv:2407.10671)](https://arxiv.org/abs/2407.10671) · [HF Papers](https://huggingface.co/papers/2407.10671)

## ***See [our collection](https://huggingface.co/collections/zeromodels/qwen2-6a8eaf9702bfe6bba801f0bd) for all Qwen2 versions.***

# Run Qwen2 with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-Qwen2-blue)](https://imvision12.github.io/ZeroModels/qwen2/) [![Collection](https://img.shields.io/badge/HF-Qwen2%20collection-yellow)](https://huggingface.co/collections/zeromodels/qwen2-6a8eaf9702bfe6bba801f0bd)

# zeromodels/qwen2-72b-instruct

Qwen2 is Alibaba's decoder-only transformer family: grouped-query attention with q/k/v bias, SwiGLU MLPs, RMSNorm, and rotary positions, in dense 0.5B-72B sizes (plus the Qwen2-57B-A14B mixture-of-experts), as base and instruct variants.

For more details on the model, please see the upstream [model card](https://huggingface.co/Qwen/Qwen2-72B-Instruct).

Pure-**Keras 3** conversion of [`Qwen/Qwen2-72B-Instruct`](https://huggingface.co/Qwen/Qwen2-72B-Instruct) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

This is an **instruct** (chat-tuned) checkpoint; load `Qwen2Tokenizer` so the chat template is applied.

## Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.qwen2 import Qwen2TextGenerate, Qwen2Tokenizer

model = Qwen2TextGenerate.from_weights("zeromodels/qwen2-72b-instruct")
tokenizer = Qwen2Tokenizer.from_weights("zeromodels/qwen2-72b-instruct")

inputs = tokenizer([
    {"role": "user", "content": "Explain rotary embeddings in one sentence."}
])
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))
```

Load any Qwen2 variant the same way with `from_weights("zeromodels/<variant>")`:

| Variant | Hub | Type |
|---|---|---|
| `qwen2-0.5b` | [`zeromodels/qwen2-0.5b`](https://huggingface.co/zeromodels/qwen2-0.5b) | base |
| `qwen2-0.5b-instruct` | [`zeromodels/qwen2-0.5b-instruct`](https://huggingface.co/zeromodels/qwen2-0.5b-instruct) | instruct |
| `qwen2-1.5b` | [`zeromodels/qwen2-1.5b`](https://huggingface.co/zeromodels/qwen2-1.5b) | base |
| `qwen2-1.5b-instruct` | [`zeromodels/qwen2-1.5b-instruct`](https://huggingface.co/zeromodels/qwen2-1.5b-instruct) | instruct |
| `qwen2-7b` | [`zeromodels/qwen2-7b`](https://huggingface.co/zeromodels/qwen2-7b) | base |
| `qwen2-7b-instruct` | [`zeromodels/qwen2-7b-instruct`](https://huggingface.co/zeromodels/qwen2-7b-instruct) | instruct |
| `qwen2-72b` | [`zeromodels/qwen2-72b`](https://huggingface.co/zeromodels/qwen2-72b) | base |
| `qwen2-72b-instruct` | [`zeromodels/qwen2-72b-instruct`](https://huggingface.co/zeromodels/qwen2-72b-instruct) | instruct |
| `qwen2-57b-a14b` | [`zeromodels/qwen2-57b-a14b`](https://huggingface.co/zeromodels/qwen2-57b-a14b) | MoE base |
| `qwen2-57b-a14b-instruct` | [`zeromodels/qwen2-57b-a14b-instruct`](https://huggingface.co/zeromodels/qwen2-57b-a14b-instruct) | MoE instruct |
| `qwen1.5-moe-a2.7b` | [`zeromodels/qwen1.5-moe-a2.7b`](https://huggingface.co/zeromodels/qwen1.5-moe-a2.7b) | MoE base |
| `qwen1.5-moe-a2.7b-chat` | [`zeromodels/qwen1.5-moe-a2.7b-chat`](https://huggingface.co/zeromodels/qwen1.5-moe-a2.7b-chat) | MoE chat |

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `Qwen2Tokenizer.from_weights(...)` so the chat template matches.
- Larger checkpoints: try `load_dtype="bfloat16"` or `quantization="int8"`.
- See [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/) and the [Qwen2 docs](https://imvision12.github.io/ZeroModels/qwen2/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `Qwen2TextGenerate.from_weights("hf:Qwen/Qwen2-72B-Instruct")`.

## Special Thanks

A huge thank you to the Qwen team at Alibaba for creating and releasing these models.

License: Tongyi Qianwen (see the [upstream license](https://huggingface.co/Qwen/Qwen2-72B-Instruct/blob/main/LICENSE)).