README.md

metadata

title: Lt Space
emoji: 🗣    ️
colorFrom: yellow
colorTo: purple
sdk: docker
pinned: false
license: mit
short_description: Language translation space

Universal Translator with CTranslate2 Optimization

This project adds CTranslate2 optimization to the HF Spaces backend of the Universal Translator application. CTranslate2 significantly improves translation speed and reduces memory usage while maintaining compatibility with the existing API.

Features

• Speed Improvement: 5-10x faster translation compared to standard transformers
• Reduced Memory Usage: Lower memory footprint through model quantization
• Graceful Fallback: Falls back to standard translation if CTranslate2 is unavailable
• API Compatibility: Maintains the same API contract for seamless integration
• Dravidian Language Support: Special handling for Tamil, Telugu, Kannada, and Malayalam

Implementation

The optimization has been implemented in a way that preserves the existing API contract while providing significant performance improvements:

1. Added a new translation_model_ct2.py model that uses CTranslate2
2. Updated api_server.py to use the new model while maintaining the same API
3. Added efficient batch processing for HTML and document translation
4. Included a model converter utility for pre-converting models
5. Updated requirements and Dockerfile for easy deployment

Installation

1. Update the requirements:

   pip install -r requirements.txt
   

2. That's it! The system will automatically use CTranslate2 if available or fall back to standard transformers if not.

Pre-converting Models

For faster startup and better performance, you can pre-convert models using the included utility:

# Convert a specific model
python -m app.models.ct2_model_converter --src en --tgt es

# Convert all common language pairs
python -m app.models.ct2_model_converter --all

# List available options
python -m app.models.ct2_model_converter --list

Performance Expectations

Based on benchmarks with similar implementations, you can expect:

• 5-10x faster translation speed on CPU
• 3-5x faster translation speed on GPU
• Much lower memory usage
• Improved batch processing performance

Deployment

The updated Dockerfile includes all necessary dependencies for CTranslate2. To deploy:

docker build -t universal-translator .
docker run -p 7860:7860 -v /path/to/models:/app/.cache/ct2_models universal-translator

API Compatibility

This implementation maintains complete compatibility with the existing API, so no changes are needed in the Render client application. All endpoints, request formats, and response formats remain unchanged.

Fallback Mechanism

If CTranslate2 cannot be installed or encounters an error, the system will automatically fall back to the standard translation model without any configuration changes needed.