Update readme.md

README.md

@@ -2,4 +2,57 @@
 license: apache-2.0
 language:
 - ko
----
+---
+
+# gemma-7b non IT 버전 채팅 파인 튜닝된 버전
+
+## version history
+- 0.1 : 2024-04-05
+
+## 트레이닝 정보
+- Dataset : maywell/koVast
+- GPU : RTX 3090 24G x 1
+- optimizer : adamw_torch
+- learning rate scheduler type : cosine
+- trai hour : 140 hour
+- num of epoch : 1
+- train loss : 0.8991
+- eval loss : 0.7305
+
+## 사용법
+```
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import transformers
+import torch
+
+checkpoint = "nmj21c/gemma-7b-andj-sft"
+dtype = torch.bfloat16
+model = AutoModelForCausalLM.from_pretrained(checkpoint, attn_implementation="flash_attention_2", device_map={"": 0}, torch_dtype=dtype)
+
+toknizer_checkpoint = "philschmid/gemma-tokenizer-chatml"
+tokenizer = AutoTokenizer.from_pretrained(toknizer_checkpoint)
+
+chat = [
+    {"role": "system", "content": ""},
+    {"role": "user", "content": "서울의 강남역에서 맛집 추천해줘"},   
+]
+
+prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
+
+eos_token_str = "<|im_end|>"
+eos_token = tokenizer(eos_token_str,add_special_tokens=False)["input_ids"][0]
+
+inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to("cuda:0")
+outputs = model.generate(
+    input_ids=inputs.to(model.device), 
+    max_new_tokens=1024,
+    eos_token_id=eos_token,
+    do_sample=True,
+    temperature=0.7,
+    top_k=50,
+    top_p=0.95,
+)
+
+response = tokenizer.decode(outputs[0])[len(prompt):].strip().replace(eos_token_str, '')
+print(response)
+```