File size: 1,769 Bytes
31e83be | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ---
language:
- en
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3-1.7B/blob/main/LICENSE
pipeline_tag: text-generation
base_model: Qwen/Qwen3-1.7B
base_model_relation: quantized
library_name: litert-lm
tags:
- litert-lm
- litertlm
- qwen
- Qwen3
---
# Qwen3-1.7B LiteRT-LM Model
This repository contains LiteRT-LM variants of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) optimized for on-device text generation.
## Available Artifact
| File | Quantization Recipe | Context | Size |
|---|---|---:|---:|
| `Qwen3_1.7B.litertlm` | dynamic_wi8_afp32 | - | 2.1 GB |
## How to Use
### Command-Line Interface
1. Install the prerequisites:
```bash
pip install litert-lm
```
2. Run the command in CLI:
```bash
litert-lm run --from-huggingface-repo=litert-community/Qwen3-1.7B Qwen3_1.7B.litertlm --prompt="Write me a poem on nature"
```
### Python
1. Install the prerequisites:
```bash
pip install litert-lm huggingface_hub
```
2. Download the model file:
```python
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
repo_id="litert-community/Qwen3-1.7B",
filename="Qwen3_1.7B.litertlm"
)
```
3. Run inference:
```python
import litert_lm
litert_lm.set_min_log_severity(litert_lm.LogSeverity.ERROR) # Hide log for TUI app
with litert_lm.Engine(model_path) as engine:
with engine.create_conversation() as conversation:
while True:
user_input = input("\n>>> ")
for chunk in conversation.send_message_async(user_input):
print(chunk["content"][0]["text"], end="", flush=True)
```
## Integration
Ready to integrate this into your product? Get started in the [LiteRT-LM documentation](https://ai.google.dev/edge/litert-lm/overview). |