Food Extract
🍗
Michael Peres
makiisthebes
AI & ML interests
TinyML | Transformers | Reinforcement Learning | Learning through practice, MSc AI Student
Recent Activity
updated a Space 10 days ago
makiisthebes/gemma-3-270M-FoodExtract-demo published a Space 10 days ago
makiisthebes/gemma-3-270M-FoodExtract-demo liked a model 10 days ago
deepseek-ai/DeepSeek-V4-ProOrganizations
None yet
updated a
Space 10 days ago
published a
Space 10 days ago
liked a
model 10 days ago
updated a
model 10 days ago
published a
model 10 days ago
liked 2
models over 1 year ago
updated 2
models over 1 year ago
liked 2
datasets over 1 year ago
liked a
model over 1 year ago
liked a
dataset over 1 year ago
liked a
model over 1 year ago
liked a
dataset over 1 year ago
updated a
dataset over 1 year ago
I am sure more work on inference will be done, looks pretty exciting, possibly reducing the model sizes quite a bunch!
reacted to Sentdex's post with 🔥 almost 2 years ago
Post
10512
Okay, first pass over KAN: Kolmogorov–Arnold Networks, it looks very interesting!
Interpretability of KAN model:
May be considered mostly as a safety issue these days, but it can also be used as a form of interaction between the user and a model, as this paper argues and I think they make a valid point here. With MLP, we only interact with the outputs, but KAN is an entirely different paradigm and I find it compelling.
Scalability:
KAN shows better parameter efficiency than MLP. This likely translates also to needing less data. We're already at the point with the frontier LLMs where all the data available from the internet is used + more is made synthetically...so we kind of need something better.
Continual learning:
KAN can handle new input information w/o catastrophic forgetting, which helps to keep a model up to date without relying on some database or retraining.
Sequential data:
This is probably what most people are curious about right now, and KANs are not shown to work with sequential data yet and it's unclear what the best approach might be to make it work well both in training and regarding the interpretability aspect. That said, there's a rich long history of achieving sequential data in variety of ways, so I don't think getting the ball rolling here would be too challenging.
Mostly, I just love a new paradigm and I want to see more!
KAN: Kolmogorov-Arnold Networks (2404.19756)
Interpretability of KAN model:
May be considered mostly as a safety issue these days, but it can also be used as a form of interaction between the user and a model, as this paper argues and I think they make a valid point here. With MLP, we only interact with the outputs, but KAN is an entirely different paradigm and I find it compelling.
Scalability:
KAN shows better parameter efficiency than MLP. This likely translates also to needing less data. We're already at the point with the frontier LLMs where all the data available from the internet is used + more is made synthetically...so we kind of need something better.
Continual learning:
KAN can handle new input information w/o catastrophic forgetting, which helps to keep a model up to date without relying on some database or retraining.
Sequential data:
This is probably what most people are curious about right now, and KANs are not shown to work with sequential data yet and it's unclear what the best approach might be to make it work well both in training and regarding the interpretability aspect. That said, there's a rich long history of achieving sequential data in variety of ways, so I don't think getting the ball rolling here would be too challenging.
Mostly, I just love a new paradigm and I want to see more!
KAN: Kolmogorov-Arnold Networks (2404.19756)
- 5 replies
updated a
model almost 2 years ago
updated a
model about 2 years ago