README.md

---
language:
- en
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3-1.7B/blob/main/LICENSE
pipeline_tag: text-generation
base_model: Qwen/Qwen3-1.7B
base_model_relation: quantized
library_name: litert-lm
tags:
- litert-lm
- litertlm
- qwen
- Qwen3
---
# Qwen3-1.7B LiteRT-LM Model

This repository contains LiteRT-LM variants of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) optimized for on-device text generation.

## Available Artifact

| File | Quantization Recipe | Context | Size |
|---|---|---:|---:|
| `Qwen3_1.7B.litertlm` | dynamic_wi8_afp32 | - | 2.1 GB |

## How to Use

### Command-Line Interface

1. Install the prerequisites:
```bash
   pip install litert-lm
```
2. Run the command in CLI:
```bash
   litert-lm run --from-huggingface-repo=litert-community/Qwen3-1.7B Qwen3_1.7B.litertlm --prompt="Write me a poem on nature"
```

### Python

1. Install the prerequisites:
```bash
   pip install litert-lm huggingface_hub
```
2. Download the model file:
```python
   from huggingface_hub import hf_hub_download

   model_path = hf_hub_download(
       repo_id="litert-community/Qwen3-1.7B",
       filename="Qwen3_1.7B.litertlm"
   )
```
3. Run inference:
```python
   import litert_lm

   litert_lm.set_min_log_severity(litert_lm.LogSeverity.ERROR)  # Hide log for TUI app

   with litert_lm.Engine(model_path) as engine:
     with engine.create_conversation() as conversation:
       while True:
         user_input = input("\n>>> ")
         for chunk in conversation.send_message_async(user_input):
           print(chunk["content"][0]["text"], end="", flush=True)
```

## Integration

Ready to integrate this into your product? Get started in the [LiteRT-LM documentation](https://ai.google.dev/edge/litert-lm/overview).