added github link of app
Browse files
README.md
CHANGED
|
@@ -20,9 +20,10 @@ tags:
|
|
| 20 |
|
| 21 |
**Build Small Hackathon β Backyard AI.**
|
| 22 |
|
| 23 |
-
Turn everyday phone requests into structured Android automation skills. A fine-tuned **Qwen2.5-3B-Instruct** model (β€4B params) classifies natural language into one of 15 skills β alarms, Spotify, WhatsApp, Slack, Uber, and more β then loads the matching **UI automation trajectory** recorded on a real Android device.
|
| 24 |
|
| 25 |
- **Demo video:** [YouTube Short](https://youtube.com/shorts/IQRHf7HfTDA?si=9kEoZKRus6FzHYbB)
|
|
|
|
| 26 |
- **Social post:** _add your post link here_
|
| 27 |
- **Space:** [android-skill-router](https://huggingface.co/spaces/build-small-hackathon/android-skill-router)
|
| 28 |
|
|
@@ -36,6 +37,10 @@ Voice assistants route to cloud APIs. This project shows a **small, local-first
|
|
| 36 |
|
| 37 |
Each trajectory is a step-by-step UI tree capture (app package, taps, screens) ready for an accessibility or agent runtime.
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## Tech stack
|
| 40 |
|
| 41 |
| Piece | What |
|
|
@@ -44,6 +49,7 @@ Each trajectory is a step-by-step UI tree capture (app package, taps, screens) r
|
|
| 44 |
| **Fine-tune** | 4-bit QLoRA + SFT with [Unsloth](https://github.com/unslothai/unsloth) on Modal (`modal_apps/train_modal.py`) |
|
| 45 |
| **Inference** | Modal GPU API (`modal_apps/predict_api.py`) β keeps the Gradio Space lightweight |
|
| 46 |
| **Demo UI** | Gradio (`app.py`) |
|
|
|
|
| 47 |
| **Data** | 15 Android trajectories β `data/skills.jsonl` β ~510 prompt variations in `data/train.jsonl` |
|
| 48 |
|
| 49 |
## Hackathon entry
|
|
@@ -102,7 +108,7 @@ scripts/
|
|
| 102 |
generate_skill_dataset.py
|
| 103 |
generate_training_data.py
|
| 104 |
train.py # Local GPU training (optional)
|
| 105 |
-
trajectories/ #
|
| 106 |
trained_model/ # Local model weights (gitignored)
|
| 107 |
```
|
| 108 |
|
|
|
|
| 20 |
|
| 21 |
**Build Small Hackathon β Backyard AI.**
|
| 22 |
|
| 23 |
+
Turn everyday phone requests into structured Android automation skills. A fine-tuned **Qwen2.5-3B-Instruct** model (β€4B params) classifies natural language into one of 15 skills β alarms, Spotify, WhatsApp, Slack, Uber, and more β then loads the matching **UI automation trajectory** recorded on a real Android device. Classifier demo for the **[Pocket Automator](https://github.com/kriyanshii/pocket-automator)** stack.
|
| 24 |
|
| 25 |
- **Demo video:** [YouTube Short](https://youtube.com/shorts/IQRHf7HfTDA?si=9kEoZKRus6FzHYbB)
|
| 26 |
+
- **Android recorder:** [Pocket Automator](https://github.com/kriyanshii/pocket-automator)
|
| 27 |
- **Social post:** _add your post link here_
|
| 28 |
- **Space:** [android-skill-router](https://huggingface.co/spaces/build-small-hackathon/android-skill-router)
|
| 29 |
|
|
|
|
| 37 |
|
| 38 |
Each trajectory is a step-by-step UI tree capture (app package, taps, screens) ready for an accessibility or agent runtime.
|
| 39 |
|
| 40 |
+
## Recording trajectories
|
| 41 |
+
|
| 42 |
+
UI traces in `trajectories/` were captured with **[Pocket Automator](https://github.com/kriyanshii/pocket-automator)** β an Android accessibility recorder that exports JSON for training and replay. Record a flow on device β export β map to a skill via `scripts/generate_skill_dataset.py`.
|
| 43 |
+
|
| 44 |
## Tech stack
|
| 45 |
|
| 46 |
| Piece | What |
|
|
|
|
| 49 |
| **Fine-tune** | 4-bit QLoRA + SFT with [Unsloth](https://github.com/unslothai/unsloth) on Modal (`modal_apps/train_modal.py`) |
|
| 50 |
| **Inference** | Modal GPU API (`modal_apps/predict_api.py`) β keeps the Gradio Space lightweight |
|
| 51 |
| **Demo UI** | Gradio (`app.py`) |
|
| 52 |
+
| **Recorder** | [Pocket Automator](https://github.com/kriyanshii/pocket-automator) β Android accessibility capture & replay |
|
| 53 |
| **Data** | 15 Android trajectories β `data/skills.jsonl` β ~510 prompt variations in `data/train.jsonl` |
|
| 54 |
|
| 55 |
## Hackathon entry
|
|
|
|
| 108 |
generate_skill_dataset.py
|
| 109 |
generate_training_data.py
|
| 110 |
train.py # Local GPU training (optional)
|
| 111 |
+
trajectories/ # Pocket Automator exports (Android UI automation traces)
|
| 112 |
trained_model/ # Local model weights (gitignored)
|
| 113 |
```
|
| 114 |
|