---
base_model:
- mkd-hossain/keural-alpha
---
---
library_name: transformers
model_name: keural-alpha


---
language:
- en
- kr
library_name: transformers
pipeline_tag: text-generation
tags:
- causal-lm
- text-generation
- chat
- vllm
- bf16
- keural-alpha
- mkd

---

# Keural-Alpha

**Keural-Alpha-Chat v0.1** is an instruction-tuned causal language model developed by **MKD Co., Ltd.**  
It is optimized for **completion-style prompting** and **local deployment using vLLM** with an OpenAI-compatible API.

This release focuses on **stable, concise assistant responses** using a GPT-style prompt format.

---

## Model Description

- **Model name:** Keural-Alpha-Chat v0.1  
- **Architecture:** Causal Language Model (GPT-style)  
- **Framework:** PyTorch / Transformers  
- **Inference engine:** vLLM  
- **Precision:** bfloat16  
- **Context length:** 2048 tokens  
- **Primary language:** English  

This model is designed for **instruction-following and assistant-style text generation** using explicit prompt formatting rather than a built-in chat template.

---

## Intended Use

Keural-Alpha-Chat v0.1 is intended for:

- Instruction-style prompting  
- Assistant-like conversations (completion-based)  
- Internal chat tools and research  
- API-based inference using vLLM  

This model is **not a native chat-template model**.  
For best results, use `/v1/completions` with a structured prompt.

---

## Prompt Format (Important)

Recommended prompt structure:


### Framework versions

- TRL: 0.27.0
- Transformers: 4.57.6
- Pytorch: 2.9.1
- Datasets: 4.5.0
- Tokenizers: 0.22.1


## vllm run command 
python -m vllm.entrypoints.openai.api_server \
  --model mkd-hossain/keural-alpha-chat-v0.1 \
  --served-model-name keural-alpha-chat-v0.1 \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --max-model-len 2048 \
  --gpu-memory-utilization 0.9 \
  --port 10008


## respose command
HISTORY="System: You are Keural-Alpha, an AI assistant developed by MKD Co., Ltd."
MODEL="keural-alpha-chat-v0.1"

while true; do
  read -p "User: " USER
  [[ "$USER" == "exit" ]] && break

  PROMPT="$HISTORY\nUser: $USER\nAssistant:"

  RESPONSE=$(curl -s http://localhost:10008/v1/completions \
    -H "Content-Type: application/json" \
    -d "{
      \"model\": \"$MODEL\",
      \"prompt\": \"$PROMPT\",
      \"max_tokens\": 120,
      \"temperature\": 0.9,
      \"top_p\": 0.85,
      \"repetition_penalty\": 1.15,
      \"stop\": [\"\\nUser:\"]
    }")

  ASSISTANT=$(echo "$RESPONSE" | python -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['text'].strip())")

  echo "Assistant: $ASSISTANT"

  HISTORY="$HISTORY\nUser: $USER\nAssistant: $ASSISTANT"
done






## start vLLM with a chat template

python -m vllm.entrypoints.openai.api_server \
  --model mkd-hossain/keural-alpha-chat-v0.1 \
  --served-model-name keural-alpha-chat-v0.1 \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --max-model-len 2048 \
  --gpu-memory-utilization 0.9 \
  --port 10008 \
  --chat-template "{% for m in messages %}{% if m.role == 'system' %}System: {{ m.content }}\n{% elif m.role == 'user' %}User: {{ m.content }}\n{% elif m.role == 'assistant' %}Assistant: {{ m.content }}\n{% endif %}{% endfor %}Assistant:"