File size: 2,607 Bytes
29afc87 e7980f3 29afc87 e7980f3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
base_model: Qwen/Qwen2.5-0.5B-Instruct
library_name: mobiletransformers
pipeline_tag: text-generation
tags:
- mobiletransformers
- onnx
- on-device
- android
- lora
- int4
---
# Qwen/Qwen2.5-0.5B-Instruct — MobileTransformers package
On-device (Android) package exported from **Qwen/Qwen2.5-0.5B-Instruct** with MobileTransformers.
## What this package can do
- `core` — shared files every other group needs
- `inference` — generate or score on device
- `train` — **fine-tune on device**, then merge the adapter back into the base weights
- `rag` — retrieve over documents you ingest, and ground answers in them
## Fine-tuning method
- **lora** — LoRA — low-rank adapters on the attention projections.
- Rank: `8`
- Adapted modules: `q_proj`, `v_proj`
## Provenance
- Base model: `Qwen/Qwen2.5-0.5B-Instruct`
- Selected task: `text-generation-with-past`
- Quantization: int4
- Toolchain: optimum-onnx 0.1.0, transformers 4.57.6, ort-training 1.23.0+cpu
## Licenses
- Framework: not declared in this package — see the repository
- Base model weights: see the base model above (this package redistributes an export of those weights, so their terms govern its contents)
## Android runtime
- Minimum API: 28
- Required ABIs: any
## Variants
| id | EP | quant | engines | features | min API | rec. RAM (MB) |
| --- | --- | --- | --- | --- | --- | --- |
| cpu-int4 | cpu | int4 | native | core, inference, train, rag | 28 | — |
Default variant: `cpu-int4`.
## Running this model
This is a **MobileTransformers package**, not a plain Hugging Face model: it is a manifest plus per-variant ONNX stages and a weight-handoff map. `transformers`, `optimum` and plain `onnxruntime` cannot load it. Use the framework:
**https://github.com/martinkorelic/mobiletransformers**
```kotlin
// Android — pulls, verifies and installs on first use.
val model = MobileTransformers.fromPretrained(
context = context,
repoId = "mobiletransformers/Qwen2.5-0.5B-Instruct",
)
```
```bash
# Host — download and inspect the package without a device.
mobiletransformers pull --repo-id mobiletransformers/Qwen2.5-0.5B-Instruct
```
## Citation
If you are using this framework for your own work, please cite:
```bibtex
@misc{mobiletransformers2025,
author = {Koreli\v{c}, Martin and Pejovi{\'c}, Veljko},
title = {MobileTransformers: An On-Device LLM PEFT Framework for Fine-Tuning and Inference},
year = {2025},
howpublished = {\url{https://gitlab.fri.uni-lj.si/lrk/mobiletransformers}}
}
```
|