Qwen3-1.7B LiteRT-LM Model

This repository contains LiteRT-LM variants of Qwen/Qwen3-1.7B optimized for on-device text generation.

Available Artifact

File	Quantization Recipe	Context	Size
Qwen3_1.7B.litertlm	dynamic_wi8_afp32	-	2.1 GB

How to Use

Command-Line Interface

1. Install the prerequisites:

   pip install litert-lm

2. Run the command in CLI:

   litert-lm run --from-huggingface-repo=litert-community/Qwen3-1.7B Qwen3_1.7B.litertlm --prompt="Write me a poem on nature"

Python

1. Install the prerequisites:

   pip install litert-lm huggingface_hub

2. Download the model file:

   from huggingface_hub import hf_hub_download

   model_path = hf_hub_download(
       repo_id="litert-community/Qwen3-1.7B",
       filename="Qwen3_1.7B.litertlm"
   )

3. Run inference:

   import litert_lm

   litert_lm.set_min_log_severity(litert_lm.LogSeverity.ERROR)  # Hide log for TUI app

   with litert_lm.Engine(model_path) as engine:
     with engine.create_conversation() as conversation:
       while True:
         user_input = input("\n>>> ")
         for chunk in conversation.send_message_async(user_input):
           print(chunk["content"][0]["text"], end="", flush=True)

Integration

Ready to integrate this into your product? Get started in the LiteRT-LM documentation.