Article
Waseem AlShikh
wassemgtk
AI & ML interests
Multi-modal, Palmyra LLMs, Knowledge Graph
Recent Activity
published
an
article
1 day ago
What If LLMs Could Imagine Consequences?
updated
a model
1 day ago
wassemgtk/jepa_llm_prototypes
published
a model
1 day ago
wassemgtk/jepa_llm_prototypes
Organizations
published
an
article
1 day ago
updated
a
model
1 day ago
published
a
model
1 day ago
updated
a
Space
4 months ago
published
a
Space
4 months ago
upvoted
an
article
5 months ago
Article
Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!
•
58
updated
3
models
5 months ago
published
a
model
5 months ago
authored
a
paper
8 months ago
upvoted
a
paper
8 months ago
updated
2
Spaces
8 months ago
published
a
Space
8 months ago
updated
a
model
10 months ago
published
a
model
10 months ago
replied to
their
post
10 months ago
Applied iRoPE to any model like LLaMA 3.2-3B VERY possible! Interleaved local (RoPE) & global (temp-scaled) attention boosts long-context (10M tokens) handling. With chunking & weight transfer, it’s adaptable to "any" transformer model.
Infinite context feels closer 🤯
posted
an
update
10 months ago
Post
3272
I’ve been diving into the iRoPE architecture from Llama 4—a game-changer for long-context models! It interleaves local attention (with RoPE) for short contexts and global attention (with inference-time temp scaling) for long-range reasoning, aiming for infinite context. I’m going to try writing iRoPE—who wants to help?
Code: https://github.com/wassemgtk/iRoPE-try/blob/main/iRoPE.ipynb
Code: https://github.com/wassemgtk/iRoPE-try/blob/main/iRoPE.ipynb
-
- 1 reply
updated
a
model
10 months ago