|
|
--- |
|
|
title: Lt Space |
|
|
emoji: 🗣 ️ |
|
|
colorFrom: yellow |
|
|
colorTo: purple |
|
|
sdk: docker |
|
|
pinned: false |
|
|
license: mit |
|
|
short_description: Language translation space |
|
|
--- |
|
|
# Universal Translator with CTranslate2 Optimization |
|
|
|
|
|
This project adds CTranslate2 optimization to the HF Spaces backend of the Universal Translator application. CTranslate2 significantly improves translation speed and reduces memory usage while maintaining compatibility with the existing API. |
|
|
|
|
|
## Features |
|
|
|
|
|
- **Speed Improvement**: 5-10x faster translation compared to standard transformers |
|
|
- **Reduced Memory Usage**: Lower memory footprint through model quantization |
|
|
- **Graceful Fallback**: Falls back to standard translation if CTranslate2 is unavailable |
|
|
- **API Compatibility**: Maintains the same API contract for seamless integration |
|
|
- **Dravidian Language Support**: Special handling for Tamil, Telugu, Kannada, and Malayalam |
|
|
|
|
|
## Implementation |
|
|
|
|
|
The optimization has been implemented in a way that preserves the existing API contract while providing significant performance improvements: |
|
|
|
|
|
1. Added a new `translation_model_ct2.py` model that uses CTranslate2 |
|
|
2. Updated `api_server.py` to use the new model while maintaining the same API |
|
|
3. Added efficient batch processing for HTML and document translation |
|
|
4. Included a model converter utility for pre-converting models |
|
|
5. Updated requirements and Dockerfile for easy deployment |
|
|
|
|
|
## Installation |
|
|
|
|
|
1. Update the requirements: |
|
|
```bash |
|
|
pip install -r requirements.txt |
|
|
``` |
|
|
|
|
|
2. That's it! The system will automatically use CTranslate2 if available or fall back to standard transformers if not. |
|
|
|
|
|
## Pre-converting Models |
|
|
|
|
|
For faster startup and better performance, you can pre-convert models using the included utility: |
|
|
|
|
|
```bash |
|
|
# Convert a specific model |
|
|
python -m app.models.ct2_model_converter --src en --tgt es |
|
|
|
|
|
# Convert all common language pairs |
|
|
python -m app.models.ct2_model_converter --all |
|
|
|
|
|
# List available options |
|
|
python -m app.models.ct2_model_converter --list |
|
|
``` |
|
|
|
|
|
## Performance Expectations |
|
|
|
|
|
Based on benchmarks with similar implementations, you can expect: |
|
|
- 5-10x faster translation speed on CPU |
|
|
- 3-5x faster translation speed on GPU |
|
|
- Much lower memory usage |
|
|
- Improved batch processing performance |
|
|
|
|
|
## Deployment |
|
|
|
|
|
The updated Dockerfile includes all necessary dependencies for CTranslate2. To deploy: |
|
|
|
|
|
```bash |
|
|
docker build -t universal-translator . |
|
|
docker run -p 7860:7860 -v /path/to/models:/app/.cache/ct2_models universal-translator |
|
|
``` |
|
|
|
|
|
## API Compatibility |
|
|
|
|
|
This implementation maintains complete compatibility with the existing API, so no changes are needed in the Render client application. All endpoints, request formats, and response formats remain unchanged. |
|
|
|
|
|
## Fallback Mechanism |
|
|
|
|
|
If CTranslate2 cannot be installed or encounters an error, the system will automatically fall back to the standard translation model without any configuration changes needed. |
