README.md

---
license: other
license_name: exaone-license
license_link: >-
  https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct/resolve/main/LICENSE
language:
- ko
base_model:
- LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct
---

# EXASPO-3.5-2.4B-Instruct 

## Introduction

EXASPO-3.5-2.4B-Instruct is a language model specifically optimized for the Korean spoken(colloquial) language.

EXASPO-3.5-2.4B-Instruct is based on the `LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct` and has undergone continual pre-training and instruction tuning using a Korean spoken-language dataset.

You can find the details of the base model [here](https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct).

This repository contains the instruction-tuned 2.4B language model with the following features:

- Number of Parameters (without embeddings): 2.14B
- Number of Layers: 30
- Number of Attention Heads: GQA with 32 Q-heads and 8 KV-heads
- Vocab Size: 102,400
- Context Length: 32,768 tokens
- Tie Word Embeddings: True (unlike 7.8B and 32B models)

## Quickstart

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "JunHaHwang/EXASPO-3.5-2.4B-Instruct"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct")

prompt = "최근 겪은 일 중 재밌는 썰좀 풀어줘"

messages = [
    {"role": "system", 
     "content": "You are a kind and helpful assistant."},
    {"role": "user", "content": prompt}
]
input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)


output = model.generate(
    input_ids.to("cuda"),
    eos_token_id=tokenizer.eos_token_id,
        max_new_tokens=512,
        temperature=0.7,
        repetition_penalty =1.2
)
print(tokenizer.decode(output[0]))
```