README.md

metadata

license: apache-2.0
tags:
  - fastapi
  - qwen
  - llm
  - jwt

FastAPI Qwen 0.6B REST API

This repository contains a production-ready (though lightweight) wrapper for the Qwen/Qwen3-0.6B model using FastAPI and JWT authentication.

Features

• FastAPI Backend: Efficient RESTful endpoints.
• JWT Security: Token-based authentication for the /generate endpoint.
• Thinking Toggle: Option to hide or show the model's internal reasoning/thinking blocks via regex filtering.
• OpenAI Chat Format: Compatible with list-of-messages input.

Files

• app.py: The core FastAPI server logic.
• generate_jwt.py: Utility to create access tokens.
• client.py: Demo script for performing inference.

Quick Start

1. Install dependencies: pip install fastapi uvicorn transformers torch pyjwt python-dotenv requests.
2. Set your SECRET_KEY in a .env file.
3. Run python app.py.
4. Use client.py to send requests.