metadata
license: other
license_name: exaone-license
license_link: >-
https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct/resolve/main/LICENSE
language:
- ko
base_model:
- LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
EXASPO-3.5-7.8B-Instruct
Introduction
EXASPO-3.5-7.8B-Instruct is a language model specifically optimized for the Korean spoken(colloquial) language.
EXASPO-3.5-7.8B-Instruct is based on the LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct and has undergone continual pre-training and instruction tuning using a Korean spoken-language dataset.
You can find the details of the base model here.
This repository contains the instruction-tuned 7.8B language model with the following features:
- Number of Parameters (without embeddings): 6.98B
- Number of Layers: 32
- Number of Attention Heads: GQA with 32 Q-heads and 8 KV-heads
- Vocab Size: 102,400
- Context Length: 32,768 tokens
Quickstart
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "JunHaHwang/EXASPO-3.5-7.8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct")
prompt = "최근 겪은 일 중 재밌는 썰좀 풀어줘"
messages = [
{"role": "system",
"content": "You are a kind and helpful assistant."},
{"role": "user", "content": prompt}
]
input_ids = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
)
output = model.generate(
input_ids.to("cuda"),
eos_token_id=tokenizer.eos_token_id,
max_new_tokens=512,
temperature=0.7,
repetition_penalty =1.2
)
print(tokenizer.decode(output[0]))