---
license: apache-2.0
tags:
- unsloth
- trl
- sft
datasets:
- prismdata/KDI-DATASET
base_model:
- beomi/Llama-3-Open-Ko-8B-Instruct-preview
---

Inference sample Code
```
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM
```
```
model = AutoModelForCausalLM.from_pretrained("prismdata/KDI-Llama-3-Open-Ko-8B-Instruct",cache_dir="./", device_map = 'cuda')
tokenizer = AutoTokenizer.from_pretrained("prismdata/KDI-Llama-3-Open-Ko-8B-Instruct",cache_dir="./", device_map = 'cuda')
```
```
prompt_template = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.\nHuman: {prompt}\nAssistant:\n"
text = 'PMDU(prime minister’s delivery unit)가 어떤 역할을 하는 조직인가요?'
model_inputs = tokenizer(prompt_template.format(prompt=text), return_tensors='pt').to("cuda:0")
```

```
outputs = model.generate(**model_inputs, max_new_tokens=256).to("cuda:0")
output_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
print(output_text)
```

```
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
Human: PMDU(prime minister’s delivery unit)가 어떤 역할을 하는 조직인가요?
Assistant:
PMDU는 총리실 산하에 있는 조직으로, 정책효과성 증대를 위한 집행과학에 관한 연구에서 총리실의 정책조정과 집행을 지원하는 역할을 합니다.
```