qaihm-bot commited on
Commit
267958e
·
verified ·
1 Parent(s): a8c0bfe

See https://github.com/qualcomm/ai-hub-models/releases/v0.47.0 for changelog.

Files changed (1) hide show
  1. README.md +28 -23
README.md CHANGED
@@ -11,48 +11,53 @@ pipeline_tag: text-generation
11
 
12
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/plamo_1b/web-assets/model_demo.png)
13
 
14
- # PLaMo-1B: Optimized for Mobile Deployment
15
- ## State-of-the-art large language model useful on a variety of language understanding and generation tasks
16
 
17
  PLaMo-1B is the first small language model (SLM) in the PLaMo™ Lite series from Preferred Networks (PFN), designed to power AI applications for edge devices including mobile, automotive, and robots across various industrial sectors. This model builds on the advancements of PLaMo-100B, a 100-billion parameter large language model (LLM) developed from the ground up by PFN’s subsidiary Preferred Elements (PFE). Leveraging high-quality Japanese and English text data generated by PLaMo-100B, PLaMo-1B has been pre-trained on a total of 4 trillion tokens. As a result, it delivers exceptional performance in Japanese benchmarks, outperforming other SLMs with similar parameter sizes. In evaluations such as Jaster 0-shot and 4-shot, PLaMo-1B has demonstrated performance on par with larger LLMs, making it a highly efficient solution for edge-based AI tasks.
18
 
19
- Please contact us to purchase this model. More details on model performance across various devices, can be found [here](https://aihub.qualcomm.com/models/plamo_1b).
20
 
21
- **WARNING**: The model assets are not readily available for download due to licensing restrictions.
22
 
23
- ### Model Details
24
 
25
- - **Model Type:** Model_use_case.text_generation
26
- - **Model Stats:**
27
- - Input sequence length for Prompt Processor: 128
28
- - Context length: 4096
29
- - Number of parameters: 1B
30
- - Precision: w4a16 + w8a16 (few layers)
31
- - Use: Initiate conversation with prompt-processor and then token generator for subsequent iterations.
32
- - Minimum QNN SDK version required: 2.27.7
33
- - Supported languages: Japanese and English.
34
- - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
35
- - Response Rate: Rate of response generation after the first response token.
36
 
37
- | Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
38
- |---|---|---|---|---|---|
39
- | PLaMo-1B | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN_CONTEXT_BINARY | 68.21 | 0.031448000000000004 - 1.0063360000000001 | -- | -- |
40
 
41
- ## Deploying PLaMo-1B on-device
42
 
43
- Please follow the [LLM on-device deployment](https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie) tutorial.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
 
46
 
47
 
48
 
49
  ## Community
50
- * Join [our AI Hub Slack community](https://qualcomm-ai-hub.slack.com/join/shared_invite/zt-2d5zsmas3-Sj0Q9TzslueCjS31eXG2UA#/shared-invite/email) to collaborate, post questions and learn more about on-device AI.
51
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
52
 
53
  ## Usage and Limitations
54
 
55
- Model may not be used for or in connection with any of the following applications:
56
 
57
  - Accessing essential private and public services and benefits;
58
  - Administration of justice and democratic processes;
 
11
 
12
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/plamo_1b/web-assets/model_demo.png)
13
 
14
+ # PLaMo-1B: Optimized for Qualcomm Devices
 
15
 
16
  PLaMo-1B is the first small language model (SLM) in the PLaMo™ Lite series from Preferred Networks (PFN), designed to power AI applications for edge devices including mobile, automotive, and robots across various industrial sectors. This model builds on the advancements of PLaMo-100B, a 100-billion parameter large language model (LLM) developed from the ground up by PFN’s subsidiary Preferred Elements (PFE). Leveraging high-quality Japanese and English text data generated by PLaMo-100B, PLaMo-1B has been pre-trained on a total of 4 trillion tokens. As a result, it delivers exceptional performance in Japanese benchmarks, outperforming other SLMs with similar parameter sizes. In evaluations such as Jaster 0-shot and 4-shot, PLaMo-1B has demonstrated performance on par with larger LLMs, making it a highly efficient solution for edge-based AI tasks.
17
 
18
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/plamo_1b) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
 
20
+ Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
 
22
+ ## Deploying PLaMo-1B on-device
23
 
24
+ Please follow the [LLM on-device deployment](https://github.com/quic/ai-hub-apps/tree/main/tutorials/llm_on_genie) tutorial.
 
 
 
 
 
 
 
 
 
 
25
 
26
+ ## Getting Started
27
+ This model is available for purchase. Please [contact us](mailto:ai-hub-support@qti.qualcomm.com) to learn more about licensing options.
 
28
 
 
29
 
30
+ ## Model Details
31
+
32
+ **Model Type:** Model_use_case.text_generation
33
+
34
+ **Model Stats:**
35
+ - Input sequence length for Prompt Processor: 128
36
+ - Context length: 4096
37
+ - Number of parameters: 1B
38
+ - Precision: w4a16 + w8a16 (few layers)
39
+ - Use: Initiate conversation with prompt-processor and then token generator for subsequent iterations.
40
+ - Minimum QNN SDK version required: 2.27.7
41
+ - Supported languages: Japanese and English.
42
+ - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
43
+ - Response Rate: Rate of response generation after the first response token.
44
+
45
+ ## Performance Summary
46
+ | Model | Runtime | Precision | Chipset | Context Length | Response Rate (tokens per second) | Time To First Token (range, seconds)
47
+ |---|---|---|---|---|---|---
48
+ | PLaMo-1B | QNN_CONTEXT_BINARY | w4a16 | Snapdragon® 8 Elite Mobile | 4096 | 68.21 | 0.031448000000000004 - 1.0063360000000001
49
 
50
 
51
 
52
 
53
 
54
  ## Community
55
+ * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
56
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
57
 
58
  ## Usage and Limitations
59
 
60
+ This model may not be used for or in connection with any of the following applications:
61
 
62
  - Accessing essential private and public services and benefits;
63
  - Administration of justice and democratic processes;