| ---
|
| language:
|
| - my
|
| - en
|
| license: apache-2.0
|
| tags:
|
| - functiongemma
|
| - function-calling
|
| - tool-use
|
| - burmese
|
| - myanmar
|
| - on-device
|
| - gemma
|
| base_model: google/functiongemma-270m-it
|
| datasets:
|
| - thantzinphyo/Hey-Function-Burmese
|
| pipeline_tag: text-generation
|
| ---
|
|
|
| <div align="center">
|
| <img src="hey_avatar.png" width="108" alt="Hey-Function-270M Avatar">
|
| <h1>Hey-Function-270M</h1>
|
| <p><strong>A Lightweight On-Device Burmese & English Function-Calling Agent</strong></p>
|
| </div>
|
|
|
| ---
|
|
|
| ## Overview
|
|
|
| Hey-Function-270M Alpha Version is an on-device function-calling agent designed for lightweight platforms (smartphones, IoT, edge boards). It enables applications to map Burmese, English, and Code-Switching natural language instructions into structured tool calls.
|
|
|
| > [!NOTE]
|
| > **Release:** This checkpoint serves as an initial proof-of-concept to evaluate Burmese function-calling feasibility on a compact 270M-parameter architecture.
|
|
|
| - **Base Model:** Google/FunctionGemma-270M-IT
|
| - **Dataset:** [thantzinphyo/Hey-Function-Burmese](https://huggingface.co/datasets/thantzinphyo/Hey-Function-Burmese)
|
| - **Training Stage:** Initial Baseline (1 Epoch)
|
|
|
| ---
|
|
|
| ## Weight Adaptation & Training Methodology
|
|
|
| How the model weights were updated and retrained:
|
|
|
| - **Full-Parameter Tuning:**
|
| - All 270M transformer decoder weights were unfrozen and updated without parameter-efficient adapters (such as LoRA).
|
| - This was done because the original base model's tool-calling capability was trained almost exclusively in English.
|
| - Full adaptation allowed attention representations to align with Myanmar Subject-Object-Verb (SOV) sentence structures and entity schemas.
|
| - **Dynamic Candidate Collation:**
|
| - Paired target ground-truth tools with 2 negative distractors per sample to prevent 61-tool schema overflow.
|
| - Fitted comfortably within an extended 2,800-token context budget.
|
| - Achieved 0.0% sequence truncation across the entire 42.6K dataset.
|
| - **Response-Only Loss Masking:**
|
| - Masked all system prompt, tool schema, and user query tokens with label -100.
|
| - Gradients were computed strictly on the model's generated function call tokens (<start_function_call>call:...<end_function_call>).
|
| - **Optimization:**
|
| - Trained on a NVIDIA Tesla T4 GPU (16 GB VRAM).
|
| - Effective batch size of 32 (micro-batch size 1 with 32 gradient accumulation steps).
|
| - AdamW optimizer with FP16 mixed precision.
|
| - Learning rate of 2e-5 with 36-step linear warmup and linear decay across 1 full epoch (1,214 steps).
|
| - Zero-copy chunked cross-entropy (chunk_size=256) kept peak VRAM usage under 5.0 GB.
|
| ---
|
|
|
| ## Evaluation Benchmark
|
|
|
| | Register / Language | Samples | Fn Acc (%) | Arg Acc (%) | Exact Match (EM) |
|
| | :--- | :---: | :---: | :---: | :---: |
|
| | **English** | 71 | 69.0% | 67.6% | 67.6% |
|
| | **Burmese** | 47 | 44.7% | 44.7% | 44.7% |
|
| | **Code-Switch** | 73 | 52.1% | 52.1% | 52.1% |
|
| | **Overall** | **191** | **56.5%** | **56.0%** | **56.0%** |
|
|
|
| - **Fn Acc (Function Accuracy):** Percentage of samples where all predicted tool names match ground truth.
|
| - **Arg Acc (Argument Accuracy):** Percentage of samples where tool names AND all arguments match ground truth.
|
| - **Exact Match (EM):** Strict end-to-end match.
|
|
|
| ### Per-Tool Accuracy
|
| *Evaluated at tool invocation level (minimum 3 samples):*
|
|
|
| | Tool Name | Correct | Total | Accuracy (%) |
|
| | :--- | :---: | :---: | :---: |
|
| | `turn_off_flashlight` | 12 | 12 | 100.0% |
|
| | `check_traffic` | 3 | 3 | 100.0% |
|
| | `open_website` | 4 | 4 | 100.0% |
|
| | `open_wifi_settings` | 8 | 9 | 88.9% |
|
| | `create_contact` | 30 | 35 | 85.7% |
|
| | `turn_on_flashlight` | 15 | 18 | 83.3% |
|
| | `navigate_to` | 13 | 16 | 81.2% |
|
| | `send_email` | 18 | 26 | 69.2% |
|
| | `show_map` | 17 | 29 | 58.6% |
|
| | `create_calendar_event` | 33 | 61 | 54.1% |
|
| | `set_volume_level` | 2 | 4 | 50.0% |
|
| | `play_music` | 12 | 26 | 46.2% |
|
| | `control_media` | 1 | 6 | 16.7% |
|
|
|
| ---
|
|
|
| ## Future Roadmap & Planned Improvements
|
|
|
| As this is an initial release, future iterations are planned:
|
|
|
| - **Dataset Expansion:**
|
| Expanding [thantzinphyo/Hey-Function-Burmese](https://huggingface.co/datasets/thantzinphyo/Hey-Function-Burmese) with more conversational Burmese dialogue, regional slang, complex compound commands, and multi-turn function calls.
|
| - **Improving Argument Extraction:**
|
| Training for additional epochs (3–5 epochs) to boost Argument Accuracy (Arg Acc), enabling precise extraction of Burmese dates, times, currency amounts, and entity names into JSON arguments.
|
| - **On-Device Quantization:**
|
| Exporting to GGUF and LiteRT formats (int4 / int8 quantization) for offline execution inside Android and Flutter mobile apps.
|
|
|
| ---
|
|
|
| ## License
|
|
|
| This project is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0), in accordance with the base model's [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
|
|
|
| ---
|
|
|
| ## Citation & Resources
|
|
|
| - **Model Repository:** [thantzinphyo/Hey-Function-270M](https://huggingface.co/thantzinphyo/Hey-Function-270M)
|
| - **Dataset:** [thantzinphyo/Hey-Function-Burmese](https://huggingface.co/datasets/thantzinphyo/Hey-Function-Burmese)
|
| - **Codebase:** [GitHub](https://github.com/thantzinphyo-gg/Hey-Function-Burmese)
|
|
|
| ```bibtex
|
| @misc{hey_270m_alpha_2026,
|
| title={Hey-Function-270M: A Lightweight On-Device Burmese & English Function-Calling Agent},
|
| author={Thant Zin Phyo},
|
| year={2026},
|
| publisher={Hugging Face},
|
| howpublished={\url{https://huggingface.co/thantzinphyo/Hey-Function-270M}}
|
| }
|
| ```
|
|
|
