CPU is slow, GPU support?
hello sir,
first of all, thanks for making this available.
I tried artifacts/int4-generic/translategemma-4b-it-int4-generic.litertlm on a Samsung S23. It works and the translations look good, but it seems to run on CPU only, so it is pretty slow.
On the same phone, the official Gemma 4 E2B .litertlm runs much faster with GPU acceleration, so I was wondering:
If possible would you please make an Android GPU / ML Drift compatible .litertlm build for TranslateGemma 4B INT4?
I have neither knowledge nor HW..
Thanks!
Those are compiled for your device specifically, see also https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/blob/main/gemma-4-E2B-it_qualcomm_sm8750.litertlm and gemma-4-E2B-it_Google_Tensor_G5.litertlm . I rented the hardware for int4 and int8, and included a script for others to do the same themselves. I don't intend on re-exporting for Samsung, but it's open source for DIY