README.md

metadata

license: mit
title: Veylon
sdk: docker
emoji: 🔥
colorFrom: blue
colorTo: purple
pinned: false

Veylon Alpha 1 Preview

Veylon Alpha 1 Preview is a lightweight experimental language model designed for efficient training and fast inference.

Highlights

• 7M parameter prototype
• Trained with Keras 3 + JAX
• Optimized for TPU v5e training
• Sliding Window Attention (SWA)
• Grouped Query Attention (GQA)
• Fast training throughput
• Research-focused architecture

Performance

• Context Length: 1024 tokens
• Vocabulary Size: 8000
• TPU Throughput: 200K+ tokens/sec during training
• Lightweight checkpoint size

Try It

Type a prompt and generate text directly in the demo.

Roadmap

• Veylon Alpha 10M
• Veylon Alpha 30M
• Veylon Alpha 100M
• Advanced memory systems
• Longer context support

Author

Created by IconicDev.

Disclaimer

This is a research preview and may generate inaccurate or nonsensical outputs.