Spaces:
Sleeping
Sleeping
| license: mit | |
| title: Veylon | |
| sdk: docker | |
| emoji: 🔥 | |
| colorFrom: blue | |
| colorTo: purple | |
| pinned: false | |
| Veylon Alpha 1 Preview | |
| Veylon Alpha 1 Preview is a lightweight experimental language model designed for efficient training and fast inference. | |
| Highlights | |
| - 7M parameter prototype | |
| - Trained with Keras 3 + JAX | |
| - Optimized for TPU v5e training | |
| - Sliding Window Attention (SWA) | |
| - Grouped Query Attention (GQA) | |
| - Fast training throughput | |
| - Research-focused architecture | |
| Performance | |
| - Context Length: 1024 tokens | |
| - Vocabulary Size: 8000 | |
| - TPU Throughput: 200K+ tokens/sec during training | |
| - Lightweight checkpoint size | |
| Try It | |
| Type a prompt and generate text directly in the demo. | |
| Roadmap | |
| - Veylon Alpha 10M | |
| - Veylon Alpha 30M | |
| - Veylon Alpha 100M | |
| - Advanced memory systems | |
| - Longer context support | |
| Author | |
| Created by IconicDev. | |
| Disclaimer | |
| This is a research preview and may generate inaccurate or nonsensical outputs. |