32 Bits deployment, it is possible?
Hi everyone! I’m an ML researcher. Do you know if it’s possible to deploy this model on a 32-bit CPU? I’m currently working with an edge device.
Hi @mhsiqueira ,
Thanks for reaching out to us!
While the google/functiongemma-270m-it is compact and could technically fit within a 32-bit system’s total memory, the deployment would likely be a bit difficult. In addition to model weights, memory is also needed for runtime, activations, and KV cache, which adds significantly to the total memory required. Additionally, most modern frameworks and runtimes are designed for 64-bit systems, so a 32-bit CPU would require a custom runtime. Using LiteRT might help in this case.
So, does LiteRT support 32‑bit? I tried to deploy it, but I didn’t find any information about that. My device is armeabi‑v7a.
Thank you so much!
Hi @mhsiqueira ,
Yes, LiteRT supports 32-bit ARM (armeabi-v7a), You don’t need a custom runtime, but you may need to build the LiteRT shared library from source, because many prebuilt genai binaries default to 64-bit. You can refer to this official documentation for details on ARM boards: https://ai.google.dev/edge/litert/build/arm
Thanks!
Tks!!