metadata
title: README
emoji: 📚
colorFrom: purple
colorTo: indigo
sdk: static
pinned: false
🚀 Caca Transformers by Lyon
Custom Transformer architecture with modern features for efficient language modeling.
🎯 Features
- 30+ Model Variants (10M - 1T parameters)
- Grouped Query Attention (GQA)
- RoPE Positional Encoding
- SwiGLU Activation
- Flash Attention Support
📦 Installation
pip install caca-transformers
🤗 Available Models
Browse all models below ↓