README.md

metadata

title: LS ML Backend (T5)
emoji: 🕸️
colorFrom: gray
colorTo: blue
sdk: docker
pinned: false
startup_duration_timeout: 2h
app_port: 9090

T5 Model Backend for Label Studio

{ "project_type": "machine_learning_backend", "primary_language": "Python", "frameworks": ["transformers", "torch", "flask", "label-studio-ml"], "architecture": "T5 with LoRA fine-tuning", "key_components": { "model": "T5Model class extending LabelStudioMLBase", "server": "Flask-based API with Gunicorn", "training": "LoRA fine-tuning pipeline with state preservation" } }

Project Overview

This backend integrates T5 language models with Label Studio for text classification tasks. It uses LoRA (Low-Rank Adaptation) for efficient fine-tuning and implements complete model state preservation.

``` project/ ├── model.py # Core T5 model implementation with LoRA ├── _wsgi.py # Server initialization and logging ├── docker-compose.yml # Container orchestration └── requirements.txt # Python dependencies ```

Features

• T5-based text classification with LoRA fine-tuning
• Complete model state preservation and loading
• Real-time model training and prediction
• Configurable model parameters
• Checkpoint management
• GPU support with automatic fallback to CPU

```python # Key implementation pattern for model initialization and saving class T5Model(LabelStudioMLBase): model_name = os.getenv('MODEL_NAME', 'google/flan-t5-base') max_length = int(os.getenv('MAX_LENGTH', '512')) generation_max_length = int(os.getenv('GENERATION_MAX_LENGTH', '128')) num_return_sequences = int(os.getenv('NUM_RETURN_SEQUENCES', '1'))

def save_model(self, model, save_path):
    """Save model with complete state preservation"""
    model.save_pretrained(
        save_path,
        save_function=torch.save,
        safe_serialization=True,
        save_state_dict=True
    )

</cursor-code-patterns>

## Quick Start

### Running with Docker (Recommended)

1. Start the ML backend on `http://localhost:9090`:
```bash
docker-compose up

2. Verify the backend is running:

curl http://localhost:9090/
{"status":"UP"}

3. Connect to Label Studio: Go to Settings -> Machine Learning -> Add Model and enter http://localhost:9090

Configuration

### Model Settings - `MODEL_NAME` - T5 model to use (default: google/flan-t5-base) - `MAX_LENGTH` - Maximum input sequence length (default: 512) - `GENERATION_MAX_LENGTH` - Maximum output sequence length (default: 128) - `NUM_RETURN_SEQUENCES` - Number of predictions to return (default: 1)

LoRA Training Settings

• LORA_R - LoRA rank (default: 16)
• LORA_ALPHA - LoRA alpha parameter (default: 16)
• LORA_DROPOUT - Dropout rate (default: 0.1)
• LORA_TARGET_MODULES - Target modules for LoRA (default: q,v)
• NUM_EPOCHS - Number of training epochs (default: 16)
• LEARNING_RATE - Learning rate (default: 1e-5)
• BATCH_SIZE - Training batch size (default: 1)

Storage Settings

• MODEL_DIR - Directory for saving trained models (default: /data/models)
• HF_CHECKPOINT_DIR - Directory for saving checkpoints (default: /data/checkpoints)

Development Notes

- The model uses class-level configuration to ensure consistent initialization - LoRA parameters are configurable through environment variables - Complete model state is preserved during saving and loading - Logging is structured with detailed context for debugging - Docker setup includes caching for pip and apt for faster builds

Requirements

See requirements.txt for full dependencies. Key requirements:

• transformers>=4.34.0
• torch>=2.0.0
• label-studio-ml>=1.0.9
• flask>=2.0.0
• peft>=0.5.0

Logging

The backend uses structured logging with configurable levels. Log format includes timestamp, level, function name, and line numbers for easy debugging. Model saving and loading operations are explicitly logged for tracking state preservation.