hossain0909/base-model

---

library_name: transformers model_name: keural-alpha

---

language:

• en
• kr library_name: transformers pipeline_tag: text-generation tags:
• causal-lm
• text-generation
• chat
• vllm
• bf16
• keural-alpha
• mkd

---

Keural-Alpha

Keural-Alpha-Chat v0.1 is an instruction-tuned causal language model developed by MKD Co., Ltd.
It is optimized for completion-style prompting and local deployment using vLLM with an OpenAI-compatible API.

This release focuses on stable, concise assistant responses using a GPT-style prompt format.

---

Model Description

• Model name: Keural-Alpha-Chat v0.1
• Architecture: Causal Language Model (GPT-style)
• Framework: PyTorch / Transformers
• Inference engine: vLLM
• Precision: bfloat16
• Context length: 2048 tokens
• Primary language: English

This model is designed for instruction-following and assistant-style text generation using explicit prompt formatting rather than a built-in chat template.

---

Intended Use

Keural-Alpha-Chat v0.1 is intended for:

• Instruction-style prompting
• Assistant-like conversations (completion-based)
• Internal chat tools and research
• API-based inference using vLLM

This model is not a native chat-template model.
For best results, use /v1/completions with a structured prompt.

---

Prompt Format (Important)

Recommended prompt structure:

Framework versions

• TRL: 0.27.0
• Transformers: 4.57.6
• Pytorch: 2.9.1
• Datasets: 4.5.0
• Tokenizers: 0.22.1

vllm run command

python -m vllm.entrypoints.openai.api_server
--model mkd-hossain/keural-alpha-chat-v0.1
--served-model-name keural-alpha-chat-v0.1
--dtype bfloat16
--tensor-parallel-size 2
--max-model-len 2048
--gpu-memory-utilization 0.9
--port 10008

respose command

HISTORY="System: You are Keural-Alpha, an AI assistant developed by MKD Co., Ltd." MODEL="keural-alpha-chat-v0.1"

while true; do read -p "User: " USER [[ "$USER" == "exit" ]] && break

PROMPT="$HISTORY\nUser: $USER\nAssistant:"

RESPONSE=$(curl -s http://localhost:10008/v1/completions
-H "Content-Type: application/json"
-d "{ "model": "$MODEL", "prompt": "$PROMPT", "max_tokens": 120, "temperature": 0.9, "top_p": 0.85, "repetition_penalty": 1.15, "stop": ["\nUser:"] }")

ASSISTANT=$(echo "$RESPONSE" | python -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['text'].strip())")

echo "Assistant: $ASSISTANT"

HISTORY="$HISTORY\nUser: $USER\nAssistant: $ASSISTANT" done

start vLLM with a chat template

python -m vllm.entrypoints.openai.api_server
--model mkd-hossain/keural-alpha-chat-v0.1
--served-model-name keural-alpha-chat-v0.1
--dtype bfloat16
--tensor-parallel-size 1
--max-model-len 2048
--gpu-memory-utilization 0.9
--port 10008
--chat-template "{% for m in messages %}{% if m.role == 'system' %}System: {{ m.content }}\n{% elif m.role == 'user' %}User: {{ m.content }}\n{% elif m.role == 'assistant' %}Assistant: {{ m.content }}\n{% endif %}{% endfor %}Assistant:"