Hugging Face's logo
Pranav Upadhyaya's picture

Pranav Upadhyaya

pranavupadhyaya52
6
webxos's profile picture cascade-legend's profile picture Quazim0t0's profile picture
·

AI & ML interests

None yet

Recent Activity

reacted to LH-Tech-AI's post with 🔥 about 22 hours ago
Supra2-100M is out! Go check it out: - https://www.reddit.com/r/LocalLLaMA/comments/1velyl9/new_models_supra2100m_base_and_instruct_go_check/ - https://huggingface.co/SupraLabs/Supra2-100M - https://huggingface.co/SupraLabs/Supra2-100M-Instruct Give us a like and a follow!! HAVE FUN 🤗🔥🚀 more coming soon...
reacted to ucr-max's post with 🔥 13 days ago
Introducing Limen0.2B We are releasing Limen0.2B, a 222.5M-parameter base language model developed as a research platform for efficient pretraining and superword tokenization at smaller scales. Limen0.2B was trained from scratch on 50B tokens and uses a compact 16K BoundlessBPE vocabulary. The project explores whether SuperBPE-style tokenization can remain effective in a substantially smaller model and vocabulary regime than those examined in earlier large-scale experiments. The model also combines a deep-and-narrow transformer design with Exclusive Self-Attention, grouped-query attention, and tied embeddings. Its compact vocabulary reduces the embedding footprint and leaves a larger share of the parameter budget available to the transformer layers. Despite its relatively modest training budget, Limen0.2B achieves competitive results for its scale across the reported language understanding, commonsense reasoning, and grammatical evaluation tasks. Comparisons with other compact models are provided as context rather than strict rankings, as their training data, token budgets, architectures, and evaluation settings differ. The release includes the model weights, implementation, training configuration, checkpoint progression, and evaluation results, all under Apache 2.0. https://huggingface.co/UniversalComputingResearch/Limen0.2B Technical feedback, independent evaluations, and further experiments with the model and tokenizer are welcome.
reacted to ucr-max's post with 🔥 13 days ago
Introducing Limen0.2B We are releasing Limen0.2B, a 222.5M-parameter base language model developed as a research platform for efficient pretraining and superword tokenization at smaller scales. Limen0.2B was trained from scratch on 50B tokens and uses a compact 16K BoundlessBPE vocabulary. The project explores whether SuperBPE-style tokenization can remain effective in a substantially smaller model and vocabulary regime than those examined in earlier large-scale experiments. The model also combines a deep-and-narrow transformer design with Exclusive Self-Attention, grouped-query attention, and tied embeddings. Its compact vocabulary reduces the embedding footprint and leaves a larger share of the parameter budget available to the transformer layers. Despite its relatively modest training budget, Limen0.2B achieves competitive results for its scale across the reported language understanding, commonsense reasoning, and grammatical evaluation tasks. Comparisons with other compact models are provided as context rather than strict rankings, as their training data, token budgets, architectures, and evaluation settings differ. The release includes the model weights, implementation, training configuration, checkpoint progression, and evaluation results, all under Apache 2.0. https://huggingface.co/UniversalComputingResearch/Limen0.2B Technical feedback, independent evaluations, and further experiments with the model and tokenizer are welcome.
View all activity

Organizations

Fermac AI Systems's profile picture