Spaces:
Sleeping
Sleeping
File size: 926 Bytes
8e749f7 6f918ec 907b8b0 6f918ec 470e2bf 8e749f7 6f918ec 470e2bf 8e749f7 6f918ec | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | ---
license: mit
title: Veylon
sdk: docker
emoji: 🔥
colorFrom: blue
colorTo: purple
pinned: false
---
Veylon Alpha 1 Preview
Veylon Alpha 1 Preview is a lightweight experimental language model designed for efficient training and fast inference.
Highlights
- 7M parameter prototype
- Trained with Keras 3 + JAX
- Optimized for TPU v5e training
- Sliding Window Attention (SWA)
- Grouped Query Attention (GQA)
- Fast training throughput
- Research-focused architecture
Performance
- Context Length: 1024 tokens
- Vocabulary Size: 8000
- TPU Throughput: 200K+ tokens/sec during training
- Lightweight checkpoint size
Try It
Type a prompt and generate text directly in the demo.
Roadmap
- Veylon Alpha 10M
- Veylon Alpha 30M
- Veylon Alpha 100M
- Advanced memory systems
- Longer context support
Author
Created by IconicDev.
Disclaimer
This is a research preview and may generate inaccurate or nonsensical outputs. |