kriyanshi commited on
Commit
4e1884d
Β·
1 Parent(s): 0fc627c

added github link of app

Browse files
Files changed (1) hide show
  1. README.md +8 -2
README.md CHANGED
@@ -20,9 +20,10 @@ tags:
20
 
21
  **Build Small Hackathon β€” Backyard AI.**
22
 
23
- Turn everyday phone requests into structured Android automation skills. A fine-tuned **Qwen2.5-3B-Instruct** model (≀4B params) classifies natural language into one of 15 skills β€” alarms, Spotify, WhatsApp, Slack, Uber, and more β€” then loads the matching **UI automation trajectory** recorded on a real Android device.
24
 
25
  - **Demo video:** [YouTube Short](https://youtube.com/shorts/IQRHf7HfTDA?si=9kEoZKRus6FzHYbB)
 
26
  - **Social post:** _add your post link here_
27
  - **Space:** [android-skill-router](https://huggingface.co/spaces/build-small-hackathon/android-skill-router)
28
 
@@ -36,6 +37,10 @@ Voice assistants route to cloud APIs. This project shows a **small, local-first
36
 
37
  Each trajectory is a step-by-step UI tree capture (app package, taps, screens) ready for an accessibility or agent runtime.
38
 
 
 
 
 
39
  ## Tech stack
40
 
41
  | Piece | What |
@@ -44,6 +49,7 @@ Each trajectory is a step-by-step UI tree capture (app package, taps, screens) r
44
  | **Fine-tune** | 4-bit QLoRA + SFT with [Unsloth](https://github.com/unslothai/unsloth) on Modal (`modal_apps/train_modal.py`) |
45
  | **Inference** | Modal GPU API (`modal_apps/predict_api.py`) β€” keeps the Gradio Space lightweight |
46
  | **Demo UI** | Gradio (`app.py`) |
 
47
  | **Data** | 15 Android trajectories β†’ `data/skills.jsonl` β†’ ~510 prompt variations in `data/train.jsonl` |
48
 
49
  ## Hackathon entry
@@ -102,7 +108,7 @@ scripts/
102
  generate_skill_dataset.py
103
  generate_training_data.py
104
  train.py # Local GPU training (optional)
105
- trajectories/ # Recorded Android UI automation traces
106
  trained_model/ # Local model weights (gitignored)
107
  ```
108
 
 
20
 
21
  **Build Small Hackathon β€” Backyard AI.**
22
 
23
+ Turn everyday phone requests into structured Android automation skills. A fine-tuned **Qwen2.5-3B-Instruct** model (≀4B params) classifies natural language into one of 15 skills β€” alarms, Spotify, WhatsApp, Slack, Uber, and more β€” then loads the matching **UI automation trajectory** recorded on a real Android device. Classifier demo for the **[Pocket Automator](https://github.com/kriyanshii/pocket-automator)** stack.
24
 
25
  - **Demo video:** [YouTube Short](https://youtube.com/shorts/IQRHf7HfTDA?si=9kEoZKRus6FzHYbB)
26
+ - **Android recorder:** [Pocket Automator](https://github.com/kriyanshii/pocket-automator)
27
  - **Social post:** _add your post link here_
28
  - **Space:** [android-skill-router](https://huggingface.co/spaces/build-small-hackathon/android-skill-router)
29
 
 
37
 
38
  Each trajectory is a step-by-step UI tree capture (app package, taps, screens) ready for an accessibility or agent runtime.
39
 
40
+ ## Recording trajectories
41
+
42
+ UI traces in `trajectories/` were captured with **[Pocket Automator](https://github.com/kriyanshii/pocket-automator)** β€” an Android accessibility recorder that exports JSON for training and replay. Record a flow on device β†’ export β†’ map to a skill via `scripts/generate_skill_dataset.py`.
43
+
44
  ## Tech stack
45
 
46
  | Piece | What |
 
49
  | **Fine-tune** | 4-bit QLoRA + SFT with [Unsloth](https://github.com/unslothai/unsloth) on Modal (`modal_apps/train_modal.py`) |
50
  | **Inference** | Modal GPU API (`modal_apps/predict_api.py`) β€” keeps the Gradio Space lightweight |
51
  | **Demo UI** | Gradio (`app.py`) |
52
+ | **Recorder** | [Pocket Automator](https://github.com/kriyanshii/pocket-automator) β€” Android accessibility capture & replay |
53
  | **Data** | 15 Android trajectories β†’ `data/skills.jsonl` β†’ ~510 prompt variations in `data/train.jsonl` |
54
 
55
  ## Hackathon entry
 
108
  generate_skill_dataset.py
109
  generate_training_data.py
110
  train.py # Local GPU training (optional)
111
+ trajectories/ # Pocket Automator exports (Android UI automation traces)
112
  trained_model/ # Local model weights (gitignored)
113
  ```
114