HybriKo-117M-LinuxFC-SFT

HybriKo-117M 모델을 리눅스 명령어 함수 호출(Linux Command Function Calling) 데이터셋으로 SFT 학습한 모델입니다.

Model Description

항목	값
Base Model	HybriKo-117M (Griffin-style Hybrid RNN-Attention)
Parameters	117.8M
Architecture	2:1 RNN-to-Attention ratio (Griffin layers)
Max Seq Length	6144
Best Epoch	20 (Eval Loss: 0.0502, PPL: 1.05)

Supported Linux Tools (21개)

ls, cd, mkdir, rm, cp, mv, find, cat, grep,
head, tail, wc, ps, df, du, top, ping, curl,
chmod, tar, Finish

Usage

import torch
import sentencepiece as spm
from hybridko.model import HybriKoModel, HybriKoConfig

# Load tokenizer
sp = spm.SentencePieceProcessor()
sp.Load("HybriKo_tok.model")

# Load model
config = HybriKoConfig(
    d_model=768,
    n_layers=12,
    vocab_size=32000,
    n_heads=12,
    n_kv_heads=3,
    ff_mult=3,
    max_seq_len=6144,
    dropout=0.0
)
model = HybriKoModel(config)

checkpoint = torch.load("pytorch_model.pt", map_location="cpu")
state_dict = checkpoint.get("model_state_dict", checkpoint)
model.load_state_dict(state_dict, strict=False)
model.eval()

# Example prompt (ChatML format)
prompt = """<|im_start|>system
You are an AI assistant that helps users execute Linux commands.
<|im_end|>
<|im_start|>user
현재 폴더의 파일 목록을 보여줘
<|im_end|>
<|im_start|>assistant
"""

# Generate
input_ids = torch.tensor([sp.EncodeAsIds(prompt)])
with torch.no_grad():
    output = model.generate(input_ids, max_new_tokens=256)
print(sp.DecodeIds(output[0].tolist()))

Training Details

• Dataset: 250 training samples, 50 eval samples
• Effective Batch Size: 32 (1 × 8 GPUs × 4 accum)
• Learning Rate: 5e-5
• Hardware: NVIDIA A100 × 8

Training Curve (10-epoch intervals)

Epoch	Train Loss	Eval Loss	PPL
10	0.8649	0.2346	1.26
20	0.0968	0.0502	1.05
30	0.0395	0.0517	1.05
40	0.0249	0.0540	1.06
50	0.0162	0.0549	1.06

Citation

@misc{hybridko-linuxfc-2026,
  title={HybriKo-117M-LinuxFC-SFT: Linux Command Function Calling Model},
  author={Yaongi},
  year={2026},
  url={https://huggingface.co/Yaongi/HybriKo-117M-LinuxFC-SFT}
}

License

Apache 2.0