README.md

---
base_model: ehdwns1516/bert-base-uncased_SWAG
library_name: rk-transformers
model_name: bert-base-uncased_SWAG
tags:
- rknn
- rockchip
- npu
- rk-transformers
- rk3588
---
# bert-base-uncased_SWAG (RKNN2)

> This is an RKNN-compatible version of the [ehdwns1516/bert-base-uncased_SWAG](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG) model. It has been optimized for Rockchip NPUs using the [rk-transformers](https://github.com/emapco/rk-transformers) library.

<details><summary>Click to see the RKNN model details and usage examples</summary>

## Model Details

- **Original Model:** [ehdwns1516/bert-base-uncased_SWAG](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG)
- **Target Platform:** rk3588
- **rknn-toolkit2 Version:** 2.3.2
- **rk-transformers Version:** 0.3.0

### Available Model Files

| Model File | Optimization Level | Quantization | File Size |
| :--------- | :----------------- | :----------- | :-------- |
| [model.rknn](./model.rknn) | 0 | float16 | 235.4 MB |

## Usage

### Installation

Install `rk-transformers` with inference dependencies to use this model:

```bash
pip install rk-transformers[inference]
```

#### RK-Transformers API

```python
import numpy as np
from rktransformers import RKModelForMultipleChoice
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("rk-transformers/bert-base-uncased_SWAG")
model = RKModelForMultipleChoice.from_pretrained(
    "rk-transformers/bert-base-uncased_SWAG",
    platform="rk3588",
    core_mask="auto",
)

prompt = "In Italy, pizza is served in slices."
choice0 = "It is eaten with a fork and knife."
choice1 = "It is eaten while held in the hand."
choice2 = "It is blended into a smoothie."
choice3 = "It is folded into a taco."

encoding = tokenizer(
    [prompt, prompt, prompt, prompt], [choice0, choice1, choice2, choice3], return_tensors="np", padding=True
)
inputs = {k: np.expand_dims(v, 0) for k, v in encoding.items()}

outputs = model(**inputs)
logits = outputs.logits
print(logits.shape)
```

## Configuration

The full configuration for all exported RKNN models is available in the [config.json](./config.json) file.

</details>

---
# ehdwns1516/bert-base-uncased_SWAG

* This model has been trained as a [SWAG dataset](https://huggingface.co/ehdwns1516/bert-base-uncased_SWAG).

* Sentence Inference Multiple Choice DEMO: [Ainize DEMO](https://main-sentence-inference-multiple-choice-ehdwns1516.endpoint.ainize.ai/)

* Sentence Inference Multiple Choice API: [Ainize API](https://ainize.web.app/redirect?git_repo=https://github.com/ehdwns1516/sentence_inference_multiple_choice)

## Overview

Language model: [bert-base-uncased](https://huggingface.co/bert-base-uncased)

Language: English

Training data: [SWAG dataset](https://huggingface.co/datasets/swag)

Code: See [Ainize Workspace](https://ainize.ai/workspace/create?imageId=hnj95592adzr02xPTqss&git=https://github.com/ehdwns1516/Multiple_choice_SWAG_finetunning)

## Usage
## In Transformers

```
from transformers import AutoTokenizer, AutoModelForMultipleChoice
  
tokenizer = AutoTokenizer.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")

model = AutoModelForMultipleChoice.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")

def run_model(candicates_count, context: str, candicates: list[str]):
    assert len(candicates) == candicates_count, "you need " + candicates_count + " candidates"
    choices_inputs = []
    for c in candicates:
        text_a = ""  # empty context
        text_b = context + " " + c
        inputs = tokenizer(
            text_a,
            text_b,
            add_special_tokens=True,
            max_length=128,
            padding="max_length",
            truncation=True,
            return_overflowing_tokens=True,
        )
        choices_inputs.append(inputs)

    input_ids = torch.LongTensor([x["input_ids"] for x in choices_inputs])
    output = model(input_ids=input_ids)

    return {"result": candicates[torch.argmax(output.logits).item()]}

items = list()
count = 4 # candicates count
context = "your context"
for i in range(int(count)):
    items.append("sentence")

result = run_model(count, context, items)
```