idiot
just saying, maybe calling asperger "idiot intelligence" is maybe unintendedly offensive? :)
Okay
I'll change the name to Dumb because it might be considered discriminatory.
Other than the concept of it being really small. i was wondering if it would be possible to squeeze and LLM into less than say 32k. Though the constraints for an 8bit machine would likely be very few layers, and very tight set of questions/concepts for it to work with, very few token options.
Personally i'd give it some subset of knowledge and train it off that, like trying to have it get good at chess, or answering any and all chess questions. Outside of that topic it would be useless.
This model does not aim to make it high-performance.
As the name suggests, it's pretty stupid. And it's high speed.
Indeed. But if it's incredibly stupid, than other than for Shits and giggles, it's utterly pointless.
Though i suppose i COULD use it make clickbait replies if i made an agent to reply on X/Twitter or something...
Like this, it's extremely stupid
(I ran this on the iPhone 17)
test 2
running an llm on a c64 that gave this output would be a major achievement :)
running an llm on a c64 that gave this output would be a major achievement :)
Agreed. Though just having 32k of memory, maybe more like 20k. Somehow the specs i can think of would have a 32-64 token context window, and 32 tokens (letters) to work with. maybe 64. I doubt you could do more than say 20-30 tensor layers, and they'd have to be 8bit or less.
No one would expect such a small space or 8bit to do anything revolutionary. But maybe it could upscale pixel art... or pixel to ascii. Or as i said, working on a singular narrow topic like chess. Although maybe it would work better at 'playing' chess than at talking about it. That might actually work.
Okay, I'll make a chess dumb too
Okay, I'll make a chess dumb too
Good luck. But a model that can play (or maybe be an expert?) at certain board games would still be cool to see. Though i wouldn't expect much.
It's not that I'm bad at chess; I might be better suited to Othello Dumb AI
My AI might be better suited to Othello than chess.
My AI might be better suited to Othello than chess.
Then go Othello, Reverse/Reversi, Connect 4, and tic-tac-toe (2D and 3D), and maybe Tetris.
Ultimately it's more a proof of concept i would think. Most of those games have pre-programmed rules that makes for a challenging opponent to start.
I'm creating arithmetic AI before Othello-LLM