AppleLampsX's picture
Round 3: 701/746 held-out (94.0%), browser build 95/100
97ead18 verified
|
Raw History Blame Contribute Delete
1.15 kB
---
license: gemma
base_model: google/functiongemma-270m-it
library_name: transformers.js
tags: [function-calling, prism-os, webgpu]
---
# Prism OS command center model
FunctionGemma 270M fine-tuned on the commands of
[Prism OS](https://github.com/AppleLamps/prism-os), a desktop in the browser whose window
manager is a handwired numeric program. It turns a sentence into Prism OS's function
calls, or a short reply. It runs in the browser with Transformers.js on WebGPU
(`onnx/model_q4f16.onnx`, the graph of
[onnx-community/functiongemma-270m-it-ONNX](https://huggingface.co/onnx-community/functiongemma-270m-it-ONNX) with these weights). The
PyTorch weights are in `pytorch/`.
## Results on held-out phrasings
Every eval sentence uses a phrasing template that never appears in training. The
checkpoint kept was the best on a separate dev set of eval sentences, which these scores
leave out.
| Model | Correct |
| --- | --- |
| Base FunctionGemma | 108/746 |
| Fine-tuned (PyTorch, bf16) | 701/746 |
| Fine-tuned, browser build (q4f16), first 100 | 95/100 |
Use is subject to the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).