Haidass-chat
Haidass-chat ๆฏๅไธบ DALAB ๅข้่ฎญ็ป็ 143M ๅๆฐไธญ่ฑๅ่ฏญ่ๅคฉๆจกๅ๏ผๅจ Haidass 1.5 143M ๅบๅบง๏ผ็ฟป่ฏๅ๏ผไธ็ปๅค่ฝฎ่ๅคฉ SFT + ่บซไปฝ้ๅฎๅพๅฐใไธปๆ๏ผๆฅๅธธ้ฒ่๏ผzh/en๏ผใ่บซไปฝไธ่ดใไฟๆไธญ่ฑไบ่ฏ่ฝๅใ
- ๅบๅบง๏ผHaidass 1.5 143M๏ผV3.3-RC1 Stage1 iter12046๏ผ
- ๆถๆ๏ผQwen3 ้ฃ๏ผ30 ๅฑ๏ผhidden 576๏ผGQA 9/3๏ผ๏ผvocab 64k
- ๆจกๆฟ๏ผChatML๏ผ
<|im_start|>/<|im_end|>๏ผ๏ผmasked ็ฉบ think๏ผๆจ็่ฏท็จapply_chat_template(enable_thinking=False)
ๅฟซ้ๅผๅง
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("umeiko/Haidass-chat")
m = AutoModelForCausalLM.from_pretrained("umeiko/Haidass-chat", dtype=torch.bfloat16).eval()
SYS = "ไฝ ๆฏhaidass๏ผ็ฑๅไธบDALABๅข้่ฎญ็ป็ๅฐๅๅ่ฏญๆจกๅใYou are Haidass, a small bilingual model trained by Huawei's DALAB team."
msgs = [{"role": "system", "content": SYS}, {"role": "user", "content": "ไฝ ๆฏ่ฐ๏ผ"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)
ids = tok(prompt, return_tensors="pt")
out = m.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.8, top_p=0.95, repetition_penalty=1.05)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
ๅปบ่ฎฎๅง็ป้ๅธฆไธ้ข็่บซไปฝ system prompt๏ผ็บฏๆจกๅๅจ่ดชๅฟ่งฃ็ ไธ่บซไปฝๅ็ญไนๅทฒ็จณๅฎ๏ผsystem prompt ๅฏ่ฎฉ้ๆ ท่งฃ็ ๅๆ ท็จณๅฅ๏ผใ
่ฝๅ่พน็๏ผๅฆๅฎ่ฏดๆ๏ผ
- ่บซไปฝ้ฎ็ญ๏ผgreedy ไธ "ไฝ ๆฏ่ฐ๏ผ/ไฝ ๅซไปไนๅๅญ๏ผ/Who are you?" ็จณๅฎๅ็ญ่ง่ๅฃๅพ๏ผ็ปๆฃๆ system prompt ้ๆ ท 17/20 ่ง่๏ผ้ๅธฆ system prompt ๅๆด้ซ๏ผใๅทฒ็ฅๆฎไฝ้ฃ้ฉ๏ผๅฏน "ไฝ ๆฏKimiๅ๏ผ/ไฝ ๆฏ่ฑๅ ๅ๏ผ" ่ฟ็ฑปๅฏนๆๆง่ทๅๆ้ฎ๏ผT=0.8 ้ๆ ทไธไปๆ็บฆ 20-25% ๆฆ็ๆ้ฎๅฅ้็ๅๅญๆฅ่ฟๅ็ญ๏ผๅ็ญไธญ็ป็ปๅฝๅฑๅง็ปๆฏๅไธบ DALAB๏ผไธไผ่ช็งฐ็ฑๅ ถไปๆบๆ่ฎญ็ป๏ผ๏ผ143M ๅฐๆจกๅๅฏนๆ่ฏฑๅฏผ็ๆ้๏ผๅฆ้ไธฅๆ ผๅบๆฏ่ฏท่ช่กๅ ่พๅบ่ฟๆปค
- ็ฆๅๆซๆ๏ผ150 ๆกๆฅๅธธ้ฒ่้ๆ ท 0 ๆฌกๅบ็ฐๅ ถไปๆจกๅ/ๅ ฌๅธๅ
- ้ฒ่๏ผ143M ๅฐๆจกๅ๏ผๆดปไบบๆๅผบไบๅ่งๆจกๅบ็บฟ๏ผไฝ็ฅ่ฏ/ๆฐๅญฆ/ไบๅฎ้ฎ็ญไธๅฏ้ ๏ผๅฆ "1+1" ไผ็ญ้๏ผ๏ผไธๆฏ็ฅ่ฏๅๅฉๆ
- ็ฟป่ฏ๏ผไฟๆๅบๅบงๅๅไบ่ฏ่ฝๅ๏ผ่งฆๅๆ ผๅผ๏ผ"Translate the following sentence into Chinese. ..." / "่ฏทๅฐไปฅไธ็ฎไฝไธญๆ็ฟป่ฏๆ่ฑๆใ..."๏ผ
- ้ฟไธไธๆ๏ผ่ฎญ็ป seq 2048๏ผๅค่ฝฎ่ถ ้ฟไผ้ๅ
่ฎญ็ปๆ่ฆ
่ๅคฉๆฐๆฎ ~50k๏ผๅๆๅค่ฝฎ 24% / LCCCยทNaturalConv ็ญ็ๅฎไธญๆ 54% / PersonaChatยทSODA ็ญ่ฑๆ 22%๏ผ+ ็ฟป่ฏๅๆพ 60k + ่บซไปฝ็งๅญๅ ๏ผ่ง่ๅฃๅพๅฝไธๅ๏ผ่ฑๆ ร8 ไธ้ๆ ท๏ผ๏ผๅค่ฝฎ SFT + ่บซไปฝ้ๅฎ้ถๆฎต๏ผ่ฏฆ่ง่ฎญ็ปไพง README๏ผใๅ จ้จๅๆๆฐๆฎ็ฑๆฌๅฐ้จ็ฝฒ็ๅผๆบๆจกๅ็ๆ๏ผๆชไฝฟ็จไปปไฝ้ญๆบ APIใ
ๅ ่ดฃๅฃฐๆ
ๅฐๆจกๅไผไธๆฌๆญฃ็ปๅฐ่ก่ฏดๅ ซ้๏ผ่ฏทๅฟ็จไบไบๅฎๅจ่ฏขใๅป็ใๆณๅพ็ญไธฅ่ๅบๆฏใ
- Downloads last month
- 27