|
|
--- |
|
|
library_name: pytorch |
|
|
license: mit |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
language: |
|
|
- vi |
|
|
- en |
|
|
tags: |
|
|
- automatic-speech-recognition |
|
|
- invoice-extraction |
|
|
- speech |
|
|
--- |
|
|
|
|
|
# ASR + Invoice Extraction Server |
|
|
|
|
|
Standalone packaging of `Server_conformer.py` to transcribe audio and extract invoice JSON from transcript text. This folder now includes a copy of the trained RNNT checkpoint for convenience. |
|
|
|
|
|
## What’s inside |
|
|
- `Server_conformer.py`, `Speech2text.py`, `InformationExtractor.py` |
|
|
- `chunkformer/` code |
|
|
- `chunkformer-model/` |
|
|
- `requirements.txt` |
|
|
|
|
|
## Prerequisites |
|
|
- Python 3.9+ and a CUDA GPU (required for Qwen invoice extraction; CPU will be extremely slow) |
|
|
- Hugging Face token with access to the models you use (`HF_TOKEN`) |
|
|
- Chunkformer RNNT checkpoint available at `chunkformer-model` (copied into this folder). Update `CHUNKFORMER_MODEL_PATH` if you place it elsewhere. |
|
|
|
|
|
## Setup |
|
|
```bash |
|
|
cd Speech2Invoice |
|
|
python3 -m venv .venv |
|
|
source .venv/bin/activate |
|
|
pip install -r requirements.txt |
|
|
``` |
|
|
|
|
|
## Configure environment |
|
|
Create a `.env` (or export env vars) with at least: |
|
|
``` |
|
|
PORT=8000 |
|
|
USE_NGROK=false |
|
|
HF_TOKEN=your_hf_token_here |
|
|
CHUNKFORMER_MODEL_PATH=chunkformer-model |
|
|
LOG_LEVEL=DEBUG |
|
|
DEBUG=true |
|
|
|
|
|
# Optional ngrok |
|
|
NGROK_AUTHTOKEN= |
|
|
NGROK_REGION=ap |
|
|
|
|
|
# Optional invoice LLM overrides (defaults are fast) |
|
|
IE_LLM_MODEL_ID=Qwen/Qwen1.5-7B-Chat |
|
|
IE_MAX_NEW_TOKENS=256 |
|
|
IE_DO_SAMPLE=false |
|
|
IE_TEMPERATURE=0.0 |
|
|
IE_TOP_P=0.8 |
|
|
``` |
|
|
|
|
|
If you move the model elsewhere, set `CHUNKFORMER_MODEL_PATH` to that directory. |
|
|
|
|
|
## Run |
|
|
```bash |
|
|
python3 Server_conformer.py |
|
|
``` |
|
|
|
|
|
## Endpoints |
|
|
- `POST /transcribe` — multipart/form-data with audio file (`wav`, `mp3`, `m4a`, `ogg`, `webm`). Returns JSON with `final_result` and `full_transcription`. |
|
|
- `POST /ticket` — JSON body `{"full_transcription": "<text>"}`. Returns invoice JSON inferred by Qwen. |
|
|
|
|
|
## Notes |
|
|
- The invoice extractor requires GPU and HF download on first run. Use smaller models via `IE_LLM_MODEL_ID` for speed. |
|
|
- Model weights for the RNNT checkpoint are included in `chunkformer-model/`. For large files, consider git-lfs if you plan to push to a remote. |
|
|
|
|
|
## Contact |
|
|
|
|
|
For questions or controlled access requests to Speech2Invoice: |
|
|
|
|
|
* Duc Dat Pham |
|
|
* Email: [ducdatit2002@gmail.com](mailto:ducdatit2002@gmail.com) |
|
|
|
|
|
|
