v0.47.0
Browse filesSee https://github.com/qualcomm/ai-hub-models/releases/v0.47.0 for changelog.
README.md
CHANGED
|
@@ -11,48 +11,53 @@ pipeline_tag: text-generation
|
|
| 11 |
|
| 12 |

|
| 13 |
|
| 14 |
-
# PLaMo-1B: Optimized for
|
| 15 |
-
## State-of-the-art large language model useful on a variety of language understanding and generation tasks
|
| 16 |
|
| 17 |
PLaMo-1B is the first small language model (SLM) in the PLaMo™ Lite series from Preferred Networks (PFN), designed to power AI applications for edge devices including mobile, automotive, and robots across various industrial sectors. This model builds on the advancements of PLaMo-100B, a 100-billion parameter large language model (LLM) developed from the ground up by PFN’s subsidiary Preferred Elements (PFE). Leveraging high-quality Japanese and English text data generated by PLaMo-100B, PLaMo-1B has been pre-trained on a total of 4 trillion tokens. As a result, it delivers exceptional performance in Japanese benchmarks, outperforming other SLMs with similar parameter sizes. In evaluations such as Jaster 0-shot and 4-shot, PLaMo-1B has demonstrated performance on par with larger LLMs, making it a highly efficient solution for edge-based AI tasks.
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
##
|
| 24 |
|
| 25 |
-
|
| 26 |
-
- **Model Stats:**
|
| 27 |
-
- Input sequence length for Prompt Processor: 128
|
| 28 |
-
- Context length: 4096
|
| 29 |
-
- Number of parameters: 1B
|
| 30 |
-
- Precision: w4a16 + w8a16 (few layers)
|
| 31 |
-
- Use: Initiate conversation with prompt-processor and then token generator for subsequent iterations.
|
| 32 |
-
- Minimum QNN SDK version required: 2.27.7
|
| 33 |
-
- Supported languages: Japanese and English.
|
| 34 |
-
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
|
| 35 |
-
- Response Rate: Rate of response generation after the first response token.
|
| 36 |
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
| PLaMo-1B | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN_CONTEXT_BINARY | 68.21 | 0.031448000000000004 - 1.0063360000000001 | -- | -- |
|
| 40 |
|
| 41 |
-
## Deploying PLaMo-1B on-device
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
|
| 49 |
## Community
|
| 50 |
-
* Join [our AI Hub Slack community](https://qualcomm
|
| 51 |
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
|
| 52 |
|
| 53 |
## Usage and Limitations
|
| 54 |
|
| 55 |
-
|
| 56 |
|
| 57 |
- Accessing essential private and public services and benefits;
|
| 58 |
- Administration of justice and democratic processes;
|
|
|
|
| 11 |
|
| 12 |

|
| 13 |
|
| 14 |
+
# PLaMo-1B: Optimized for Qualcomm Devices
|
|
|
|
| 15 |
|
| 16 |
PLaMo-1B is the first small language model (SLM) in the PLaMo™ Lite series from Preferred Networks (PFN), designed to power AI applications for edge devices including mobile, automotive, and robots across various industrial sectors. This model builds on the advancements of PLaMo-100B, a 100-billion parameter large language model (LLM) developed from the ground up by PFN’s subsidiary Preferred Elements (PFE). Leveraging high-quality Japanese and English text data generated by PLaMo-100B, PLaMo-1B has been pre-trained on a total of 4 trillion tokens. As a result, it delivers exceptional performance in Japanese benchmarks, outperforming other SLMs with similar parameter sizes. In evaluations such as Jaster 0-shot and 4-shot, PLaMo-1B has demonstrated performance on par with larger LLMs, making it a highly efficient solution for edge-based AI tasks.
|
| 17 |
|
| 18 |
+
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/plamo_1b) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
|
| 19 |
|
| 20 |
+
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
|
| 21 |
|
| 22 |
+
## Deploying PLaMo-1B on-device
|
| 23 |
|
| 24 |
+
Please follow the [LLM on-device deployment](https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie) tutorial.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
## Getting Started
|
| 27 |
+
This model is available for purchase. Please [contact us](mailto:ai-hub-support@qti.qualcomm.com) to learn more about licensing options.
|
|
|
|
| 28 |
|
|
|
|
| 29 |
|
| 30 |
+
## Model Details
|
| 31 |
+
|
| 32 |
+
**Model Type:** Model_use_case.text_generation
|
| 33 |
+
|
| 34 |
+
**Model Stats:**
|
| 35 |
+
- Input sequence length for Prompt Processor: 128
|
| 36 |
+
- Context length: 4096
|
| 37 |
+
- Number of parameters: 1B
|
| 38 |
+
- Precision: w4a16 + w8a16 (few layers)
|
| 39 |
+
- Use: Initiate conversation with prompt-processor and then token generator for subsequent iterations.
|
| 40 |
+
- Minimum QNN SDK version required: 2.27.7
|
| 41 |
+
- Supported languages: Japanese and English.
|
| 42 |
+
- TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
|
| 43 |
+
- Response Rate: Rate of response generation after the first response token.
|
| 44 |
+
|
| 45 |
+
## Performance Summary
|
| 46 |
+
| Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
|
| 47 |
+
|---|---|---|---|---|---|---
|
| 48 |
+
| PLaMo-1B | QNN_CONTEXT_BINARY | w4a16 | Snapdragon® 8 Elite Mobile | 4096 | 68.21 | 0.031448000000000004 - 1.0063360000000001
|
| 49 |
|
| 50 |
|
| 51 |
|
| 52 |
|
| 53 |
|
| 54 |
## Community
|
| 55 |
+
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
|
| 56 |
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
|
| 57 |
|
| 58 |
## Usage and Limitations
|
| 59 |
|
| 60 |
+
This model may not be used for or in connection with any of the following applications:
|
| 61 |
|
| 62 |
- Accessing essential private and public services and benefits;
|
| 63 |
- Administration of justice and democratic processes;
|