Appreciate the model drop!
#6
by Nitral-AI - opened
But why is it only 4k? Its 2025 man, those are rookie numbers.
Agree
We understand the demand for longer context windows and our roadmap includes multiple possible approaches to increase it. Extending the context length involves trade-offs in training efficiency, memory usage, and model performance, we are working on how to do it as efficient as possible.
If you now need a model with a longer context, consider using our instructed Salamandra-7b, it might be more suitable for you.
mapama247 changed discussion status to closed
Hi,
The current ALIA-40B model supports up to 32K tokens, and we're actively working on extending its capabilities even further. Stay tuned for updates, we hope you enjoy using it!