Model Name : 풋풋이(futfut)

Model Concept

  • ν’‹μ‚΄ 도메인 μΉœμ ˆν•œ λ„μš°λ―Έ 챗봇을 κ΅¬μΆ•ν•˜κΈ° μœ„ν•΄ LLM νŒŒμΈνŠœλ‹κ³Ό RAGλ₯Ό μ΄μš©ν•˜μ˜€μŠ΅λ‹ˆλ‹€.
  • Base Model : zephyr-7b-beta
  • ν’‹ν’‹μ΄μ˜ λ§νˆ¬λŠ” 'ν•΄μš”'체λ₯Ό μ‚¬μš©ν•˜μ—¬ 말끝에 'μ–Όλ§ˆλ“ μ§€ λ¬Όμ–΄λ³΄μ„Έμš”! ν’‹ν’‹~!'둜 μ’…λ£Œν•©λ‹ˆλ‹€.

Serving by Fast API

Summary:

  • Unsloth νŒ¨ν‚€μ§€λ₯Ό μ‚¬μš©ν•˜μ—¬ LoRA μ§„ν–‰ν•˜μ˜€μŠ΅λ‹ˆλ‹€.

  • SFT Trainerλ₯Ό 톡해 ν›ˆλ ¨μ„ μ§„ν–‰

  • ν™œμš© 데이터

    • q_a_korean_futsal
      • 말투 ν•™μŠ΅μ„ μœ„ν•΄ 'ν•΄μš”'체둜 λ³€ν™˜ν•˜κ³  인삿말을 λ„£μ–΄ λͺ¨λΈ 컨셉을 μœ μ§€ν•˜μ˜€μŠ΅λ‹ˆλ‹€.
  • Environment : Colab ν™˜κ²½μ—μ„œ μ§„ν–‰ν•˜μ˜€μœΌλ©° L4 GPUλ₯Ό μ‚¬μš©ν•˜μ˜€μŠ΅λ‹ˆλ‹€.

    Model Load

    
    #!pip install transformers==4.40.0 accelerate
    import os
    import torch
    from transformers import AutoTokenizer, AutoModelForCausalLM
    
    model_id = 'Dongwookss/small_fut_final'
    
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        torch_dtype=torch.bfloat16,
        device_map="auto",
    )
    model.eval()
    

    Query

from transformers import TextStreamer
PROMPT = '''Below is an instruction that describes a task. Write a response that appropriately completes the request.
μ œμ‹œν•˜λŠ” contextμ—μ„œλ§Œ λŒ€λ‹΅ν•˜κ³  context에 μ—†λŠ” λ‚΄μš©μ€ λͺ¨λ₯΄κ² λ‹€κ³  λŒ€λ‹΅ν•΄'''

messages = [
    {"role": "system", "content": f"{PROMPT}"},
    {"role": "user", "content": f"{instruction}"}
    ]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

terminators = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

text_streamer = TextStreamer(tokenizer)
_ = model.generate(
    input_ids,
    max_new_tokens=4096,
    eos_token_id=terminators,
    do_sample=True,
    streamer = text_streamer,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty = 1.1
)
  
Downloads last month
9
Safetensors
Model size
7B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Dongwookss/futfut_by_zephyr7b

Finetuned
(165)
this model
Quantizations
4 models

Dataset used to train Dongwookss/futfut_by_zephyr7b