| --- |
| title: Universal Model Trainer |
| emoji: 🚀 |
| colorFrom: blue |
| colorTo: purple |
| sdk: docker |
| pinned: true |
| license: mit |
| short_description: Universal ML training dashboard for HuggingFace |
| tag: |
| - training |
| - machine-learning |
| - transformers |
| - pytorch |
| - docker |
| --- |
| |
| # 🚀 Universal Model Trainer |
|
|
| A comprehensive, production-ready dashboard for training machine learning models on the HuggingFace ecosystem. Supports multiple task types, datasets, and model architectures with full user control. |
|
|
| ## ✨ Features |
|
|
| ### Supported Task Types |
| - **Causal Language Modeling** - Text generation (GPT-style) |
| - **Masked Language Modeling** - BERT-style pretraining |
| - **Sequence-to-Sequence** - Translation, summarization |
| - **Token Classification** - NER, POS tagging |
| - **Text Classification** - Sentiment, topic classification |
| - **Question Answering** - Extractive QA |
| - **Image Classification** - Vision tasks |
| - **Audio Classification** - Sound classification |
|
|
| ### Training Features |
| - **PEFT/LoRA Support** - Memory-efficient fine-tuning |
| - **DeepSpeed Integration** - Distributed training |
| - **Quantization Options** - 4-bit, 8-bit training |
| - **Custom Hyperparameters** - Full control over training config |
| - **Real-time Progress** - Live training metrics |
| - **Job Queue System** - Manage multiple training jobs |
| - **Model Versioning** - Track experiment history |
|
|
| ### Dataset Support |
| - **HuggingFace Hub** - 100,000+ datasets |
| - **Custom Upload** - CSV, JSON, JSONL support |
| - **Dataset Preprocessing** - Automatic tokenization |
|
|
| ## 🔧 API Endpoints |
|
|
| ### Training |
| ``` |
| POST /api/train/start - Start a new training job |
| GET /api/train/status - Get job status |
| POST /api/train/stop - Stop running job |
| GET /api/train/history - View training history |
| ``` |
|
|
| ### Models & Datasets |
| ``` |
| GET /api/models/search - Search HuggingFace models |
| GET /api/datasets/search - Search HuggingFace datasets |
| GET /api/models/info - Get model info |
| ``` |
|
|
| ### System |
| ``` |
| GET /api/system/status - System health and resources |
| GET /api/system/gpus - Available GPU info |
| ``` |
|
|
| ## 🏗️ Architecture |
|
|
| - **FastAPI Backend** - Modern async Python web framework |
| - **Redis Queue** - Background job processing |
| - **SQLite/PostgreSQL** - Job and experiment persistence |
| - **HuggingFace Hub** - Model and dataset hosting |
|
|
| ## 📝 Configuration |
|
|
| Set the following environment variables: |
| - `HF_TOKEN` - Your HuggingFace API token (required for pushing models) |
| - `WANDB_API_KEY` - Weights & Biases API key (optional, for experiment tracking) |
|
|
| ## 📚 Documentation |
|
|
| Full API documentation available at `/docs` when the Space is running. |
|
|
| ## 🤝 Contributing |
|
|
| Contributions welcome! See issues for planned features. |
|
|
| ## 📄 License |
|
|
| MIT License - See LICENSE file for details. |
