Add repository README with app LoRA descriptions and usage instructions
Browse files
README.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Qwen2-VL-7B App LoRA Adapters
|
| 2 |
+
|
| 3 |
+
This repository contains LoRA (Low-Rank Adaptation) adapters for the [Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct) model, fine-tuned on different mobile application datasets.
|
| 4 |
+
|
| 5 |
+
## Model Description
|
| 6 |
+
|
| 7 |
+
These LoRA adapters were trained using federated learning approaches on app-specific datasets from the FedMABench benchmark. Each app contains:
|
| 8 |
+
|
| 9 |
+
- **v0**: Initial training checkpoint (ๅบ็ก่ฎญ็ปๅ็ๅๅงLoRA)
|
| 10 |
+
- **global_lora_10**: Global aggregated LoRA after 10 rounds of federated learning (็ฌฌ10่ฝฎ่้ฆๅญฆไน ่ๅๅ็ๅ
จๅฑLoRA)
|
| 11 |
+
|
| 12 |
+
## Available Apps (14 total)
|
| 13 |
+
|
| 14 |
+
| App | v0 Checkpoint | Global LoRA (Round 10) | Training Version |
|
| 15 |
+
|-----|---------------|------------------------|------------------|
|
| 16 |
+
| adidas | โ
checkpoint-108 | โ
global_lora_10 | v0-20260131-111220 |
|
| 17 |
+
| amazon | โ
checkpoint-312 | โ
global_lora_10 | v0-20260130-190721 |
|
| 18 |
+
| calendar | โ
checkpoint-129 | โ
global_lora_10 | v0-20260131-131350 |
|
| 19 |
+
| clock | โ
checkpoint-198 | โ
global_lora_10 | v0-20260131-052630 |
|
| 20 |
+
| decathlon | โ
checkpoint-63 | โ
global_lora_10 | v0-20260131-122711 |
|
| 21 |
+
| ebay | โ
checkpoint-219 | โ
global_lora_10 | v0-20260130-223012 |
|
| 22 |
+
| etsy | โ
checkpoint-60 | โ
global_lora_10 | v0-20260131-202612 |
|
| 23 |
+
| flipkart | โ
checkpoint-174 | โ
global_lora_10 | v0-20260131-010305 |
|
| 24 |
+
| gmail | โ
checkpoint-201 | โ
global_lora_10 | v0-20260131-025059 |
|
| 25 |
+
| google_drive | โ
checkpoint-63 | โ
global_lora_10 | v0-20260131-103300 |
|
| 26 |
+
| google_maps | โ
checkpoint-60 | โ
global_lora_10 | v0-20260131-150323 |
|
| 27 |
+
| kitchen_stories | โ
checkpoint-75 | โ
global_lora_10 | v0-20260131-155209 |
|
| 28 |
+
| reminder | โ
checkpoint-138 | โ
global_lora_10 | v0-20260131-073315 |
|
| 29 |
+
| youtube | โ
checkpoint-78 | โ
global_lora_10 | v0-20260131-093519 |
|
| 30 |
+
|
| 31 |
+
## Directory Structure
|
| 32 |
+
|
| 33 |
+
```
|
| 34 |
+
qwen2vl-7b-lora-apps/
|
| 35 |
+
โโโ adidas/
|
| 36 |
+
โ โโโ v0/ # Initial checkpoint LoRA
|
| 37 |
+
โ โโโ global_lora_10/ # Round 10 federated LoRA
|
| 38 |
+
โโโ amazon/
|
| 39 |
+
โ โโโ v0/
|
| 40 |
+
โ โโโ global_lora_10/
|
| 41 |
+
โโโ calendar/
|
| 42 |
+
โ โโโ v0/
|
| 43 |
+
โ โโโ global_lora_10/
|
| 44 |
+
... (14 apps total)
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
## Usage
|
| 48 |
+
|
| 49 |
+
### Loading with PEFT
|
| 50 |
+
|
| 51 |
+
```python
|
| 52 |
+
from peft import PeftModel
|
| 53 |
+
from transformers import Qwen2VLForConditionalGeneration
|
| 54 |
+
|
| 55 |
+
# Load base model
|
| 56 |
+
base_model = Qwen2VLForConditionalGeneration.from_pretrained(
|
| 57 |
+
"Qwen/Qwen2-VL-7B-Instruct",
|
| 58 |
+
torch_dtype="auto",
|
| 59 |
+
device_map="auto"
|
| 60 |
+
)
|
| 61 |
+
|
| 62 |
+
# Load a specific app LoRA (e.g., amazon global_lora_10)
|
| 63 |
+
model = PeftModel.from_pretrained(
|
| 64 |
+
base_model,
|
| 65 |
+
"bmh201708/qwen2vl-7b-lora-apps",
|
| 66 |
+
subfolder="amazon/global_lora_10"
|
| 67 |
+
)
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
### Loading with Hugging Face Hub
|
| 71 |
+
|
| 72 |
+
```python
|
| 73 |
+
from huggingface_hub import snapshot_download
|
| 74 |
+
|
| 75 |
+
# Download specific app LoRA
|
| 76 |
+
local_path = snapshot_download(
|
| 77 |
+
repo_id="bmh201708/qwen2vl-7b-lora-apps",
|
| 78 |
+
allow_patterns=["amazon/global_lora_10/*"]
|
| 79 |
+
)
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
## Base Model
|
| 83 |
+
|
| 84 |
+
- **Model**: [Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct)
|
| 85 |
+
- **LoRA Library**: PEFT 0.17.1
|
| 86 |
+
|
| 87 |
+
## Training Details
|
| 88 |
+
|
| 89 |
+
These LoRAs were trained as part of the FedMABench (Federated Mobile Agent Benchmark) project for mobile GUI agent tasks. Each app represents a specific mobile application domain.
|
| 90 |
+
|
| 91 |
+
## Related Repositories
|
| 92 |
+
|
| 93 |
+
- **Category LoRAs**: [bmh201708/qwen2vl-7b-lora-categories](https://huggingface.co/bmh201708/qwen2vl-7b-lora-categories) - LoRAs organized by category (Entertainment, Lives, Office, Shopping, Traveling)
|
| 94 |
+
|
| 95 |
+
## License
|
| 96 |
+
|
| 97 |
+
Please refer to the [Qwen2-VL license](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct) for usage terms.
|
| 98 |
+
|
| 99 |
+
## Citation
|
| 100 |
+
|
| 101 |
+
If you use these LoRA adapters, please cite the FedMABench project.
|