File size: 4,174 Bytes
8ff51f3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
---
base_model: ehdwns1516/bert-base-uncased_SWAG
library_name: rk-transformers
model_name: bert-base-uncased_SWAG
tags:
- rknn
- rockchip
- npu
- rk-transformers
- rk3588
---
# bert-base-uncased_SWAG (RKNN2)
> This is an RKNN-compatible version of the [ehdwns1516/bert-base-uncased_SWAG](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG) model. It has been optimized for Rockchip NPUs using the [rk-transformers](https://github.com/emapco/rk-transformers) library.
<details><summary>Click to see the RKNN model details and usage examples</summary>
## Model Details
- **Original Model:** [ehdwns1516/bert-base-uncased_SWAG](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG)
- **Target Platform:** rk3588
- **rknn-toolkit2 Version:** 2.3.2
- **rk-transformers Version:** 0.3.0
### Available Model Files
| Model File | Optimization Level | Quantization | File Size |
| :--------- | :----------------- | :----------- | :-------- |
| [model.rknn](./model.rknn) | 0 | float16 | 235.4 MB |
## Usage
### Installation
Install `rk-transformers` with inference dependencies to use this model:
```bash
pip install rk-transformers[inference]
```
#### RK-Transformers API
```python
import numpy as np
from rktransformers import RKModelForMultipleChoice
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("rk-transformers/bert-base-uncased_SWAG")
model = RKModelForMultipleChoice.from_pretrained(
"rk-transformers/bert-base-uncased_SWAG",
platform="rk3588",
core_mask="auto",
)
prompt = "In Italy, pizza is served in slices."
choice0 = "It is eaten with a fork and knife."
choice1 = "It is eaten while held in the hand."
choice2 = "It is blended into a smoothie."
choice3 = "It is folded into a taco."
encoding = tokenizer(
[prompt, prompt, prompt, prompt], [choice0, choice1, choice2, choice3], return_tensors="np", padding=True
)
inputs = {k: np.expand_dims(v, 0) for k, v in encoding.items()}
outputs = model(**inputs)
logits = outputs.logits
print(logits.shape)
```
## Configuration
The full configuration for all exported RKNN models is available in the [config.json](./config.json) file.
</details>
---
# ehdwns1516/bert-base-uncased_SWAG
* This model has been trained as a [SWAG dataset](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG).
* Sentence Inference Multiple Choice DEMO: [Ainize DEMO](https://main-sentence-inference-multiple-choice-ehdwns1516.endpoint.ainize.ai/)
* Sentence Inference Multiple Choice API: [Ainize API](https://ainize.web.app/redirect?git_repo=https://github.com/ehdwns1516/sentence_inference_multiple_choice)
## Overview
Language model: [bert-base-uncased](https://huggingface.co/bert-base-uncased)
Language: English
Training data: [SWAG dataset](https://huggingface.co/datasets/swag)
Code: See [Ainize Workspace](https://ainize.ai/workspace/create?imageId=hnj95592adzr02xPTqss&git=https://github.com/ehdwns1516/Multiple_choice_SWAG_finetunning)
## Usage
## In Transformers
```
from transformers import AutoTokenizer, AutoModelForMultipleChoice
tokenizer = AutoTokenizer.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")
model = AutoModelForMultipleChoice.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")
def run_model(candicates_count, context: str, candicates: list[str]):
assert len(candicates) == candicates_count, "you need " + candicates_count + " candidates"
choices_inputs = []
for c in candicates:
text_a = "" # empty context
text_b = context + " " + c
inputs = tokenizer(
text_a,
text_b,
add_special_tokens=True,
max_length=128,
padding="max_length",
truncation=True,
return_overflowing_tokens=True,
)
choices_inputs.append(inputs)
input_ids = torch.LongTensor([x["input_ids"] for x in choices_inputs])
output = model(input_ids=input_ids)
return {"result": candicates[torch.argmax(output.logits).item()]}
items = list()
count = 4 # candicates count
context = "your context"
for i in range(int(count)):
items.append("sentence")
result = run_model(count, context, items)
```
|