Add a bitnet step to the workflow
You make a lot of models, I download a lot of quants from you....... some of these models are really large though, and people don't necessarily have the hardware spec to run them..... but you and other quantinizers could just add the bitnet workflow to your workflows, which would reduce the ram required to run the models, potentially opening up more ai accessibility since the models could then run on more devices with less ram.
thx
I'd do it myself but there are so many that need it done and I have a 3050 laptop 4gb vram 😒 plz brudder, be the one the start making ai more accessible
Hi, to do bitnet we will need to train models. We cannot train thousands of models. We are like you just a few people running this project on our own computers. Noone is paying us, it's more like we are paying for this with our hardware and electricity. We quant with main llamacpp, which provides a large range of different precisions, including very small ones
I am aware of that, however SOME folk do have access to what we need to get this done, they just for whatever reason are not doing it, when all models should be attempted to be bitnet'd and output compared to original...... what sense does it make to require everyone to need 10s to 100s of gbs of vram to run models, when we know each model could be ran faster with minimal difference just by bitnetting it..... also @mradermacher himself might have access from his job, I'd say start with the largest possible and work your way down, base and selected finetunes only.....
Hi, to do bitnet we will need to train models. We cannot train thousands of models. We are like you just a few people running this project on our own computers. Noone is paying us, it's more like we are paying for this with our hardware and electricity. We quant with main llamacpp, which provides a large range of different precisions, including very small ones
I just looked at the alleged hardware you have listed as having on your profile 💀 brudder wtf are you talking about get to work you have hella flops allegedly for this kind of project (note, all hardware listed on my page are cloud assets, for seeing what models I can run on vast.ai and Google colab)