| license: apache-2.0 | |
| tags: | |
| - fastapi | |
| - qwen | |
| - llm | |
| - jwt | |
| # FastAPI Qwen 0.6B REST API | |
| This repository contains a production-ready (though lightweight) wrapper for the **Qwen/Qwen3-0.6B** model using FastAPI and JWT authentication. | |
| ## Features | |
| - **FastAPI Backend**: Efficient RESTful endpoints. | |
| - **JWT Security**: Token-based authentication for the `/generate` endpoint. | |
| - **Thinking Toggle**: Option to hide or show the model's internal reasoning/thinking blocks via regex filtering. | |
| - **OpenAI Chat Format**: Compatible with list-of-messages input. | |
| ## Files | |
| - `app.py`: The core FastAPI server logic. | |
| - `generate_jwt.py`: Utility to create access tokens. | |
| - `client.py`: Demo script for performing inference. | |
| ## Quick Start | |
| 1. Install dependencies: `pip install fastapi uvicorn transformers torch pyjwt python-dotenv requests`. | |
| 2. Set your `SECRET_KEY` in a `.env` file. | |
| 3. Run `python app.py`. | |
| 4. Use `client.py` to send requests. | |