---
license: mit
title: Veylon
sdk: docker
emoji: 🔥
colorFrom: blue
colorTo: purple
pinned: false
---
Veylon Alpha 1 Preview

Veylon Alpha 1 Preview is a lightweight experimental language model designed for efficient training and fast inference.

Highlights

- 7M parameter prototype
- Trained with Keras 3 + JAX
- Optimized for TPU v5e training
- Sliding Window Attention (SWA)
- Grouped Query Attention (GQA)
- Fast training throughput
- Research-focused architecture

Performance

- Context Length: 1024 tokens
- Vocabulary Size: 8000
- TPU Throughput: 200K+ tokens/sec during training
- Lightweight checkpoint size

Try It

Type a prompt and generate text directly in the demo.

Roadmap

- Veylon Alpha 10M
- Veylon Alpha 30M
- Veylon Alpha 100M
- Advanced memory systems
- Longer context support

Author

Created by IconicDev.

Disclaimer

This is a research preview and may generate inaccurate or nonsensical outputs.