Update README.md

README.md

@@ -1,3 +1,133 @@
----
-license: cc-by-nc-nd-4.0
----
+---
+base_model: tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3
+license:
+- llama3.1
+- gemma
+language:
+- ja
+- en
+pipeline_tag: text-generation
+tags:
+  - counseling
+  - dialogue-system
+datasets:
+- UEC-InabaLab/KokoroChat
+---
+
+# 🧠 KokoroChat-Low: Japanese Counseling Dialogue Model
+
+**KokoroChat-Low** is a large-scale Japanese language model fine-tuned on the **entire KokoroChat dataset**—a collection of over 6,000 psychological counseling dialogues conducted via **role-play between trained counselors**. The model is capable of generating **empathetic and context-aware responses** suitable for mental health-related conversational tasks.
+
+---
+
+## 💡 Overview
+
+- ✅ Fine-tuned on **3,870 dialogues** with client feedback scores **below 70**
+- ✅ Data collected through **text-based role-play** by trained counselors 
+- ✅ Covers a wide range of topics: depression, family, school, career, relationships, and more
+- ✅ Base Model: [`tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3`](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3)
+
+---
+
+## ⚙️ Usage Example
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "UEC-InabaLab/KokoroChat-Low"
+
+# Load tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+# Set pad_token_id
+if tokenizer.pad_token_id is None:
+    tokenizer.pad_token = "[PAD]"
+    tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids("[PAD]")
+
+model.config.pad_token_id = tokenizer.pad_token_id
+
+# Build dialogue input
+messages = [
+    {"role": "system", "content": "心理カウンセリングの会話において、対話履歴を考慮し、カウンセラーとして適切に応答してください。"},
+    {"role": "user", "content": "最近、気分が落ち込んでやる気が出ません。"}
+]
+
+# Tokenize with chat template
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+attention_mask = inputs.ne(tokenizer.pad_token_id)
+
+# Generate response
+outputs = model.generate(
+    inputs,
+    attention_mask=attention_mask,
+    pad_token_id=tokenizer.pad_token_id,
+    max_new_tokens=256
+)
+
+# Extract only the newly generated tokens
+response = outputs[0][inputs.shape[-1]:]
+response_text = tokenizer.decode(response, skip_special_tokens=True)
+
+# Print clean response
+print(response_text)
+```
+
+---
+
+## 🛠️ Fine-Tuning Details
+
+Fine-tuning was performed using **QLoRA** with the following configuration:
+
+- **Quantization**: 4-bit NF4 with bfloat16 computation
+- **LoRA target modules**: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
+- **LoRA parameters**:
+  - `r = 8`
+  - `lora_alpha = 16`
+  - `lora_dropout = 0.05`
+
+### Dataset Split
+
+- **Training Data**: 3,870 dialogues with feedback scores < 70  
+- **Train/Validation Split**: 90% train, 10% validation
+
+### Hyperparameter Settings
+
+- **Optimizer**: `adamw_8bit`
+- **Warm-up Steps**: `100`
+- **Learning Rate**: `1e-3`
+- **Epochs**: `5`
+- **Batch Size**: `8`
+- **Validation Frequency**: every 400 steps
+
+---
+
+## 📄 Citation
+
+If you use this model or dataset, please cite the following paper:
+
+```bibtex
+@inproceedings{qi2025kokorochat,
+  title     = {KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors},
+  author    = {Zhiyang Qi and Takumasa Kaneko and Keiko Takamizo and Mariko Ukiyo and Michimasa Inaba},
+  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics},
+  year      = {2025},
+  url       = {https://github.com/UEC-InabaLab/KokoroChat}
+}
+```
+---
+
+## 🔗 Related
+
+- 📁 **Dataset**:  
+  - [KokoroChat on Hugging Face Datasets](https://huggingface.co/datasets/UEC-InabaLab/KokoroChat)  
+  - [KokoroChat on GitHub (UEC-InabaLab)](https://github.com/UEC-InabaLab/KokoroChat)
+- 🤖 **Model Variants**:  
+  - [KokoroChat-High](https://huggingface.co/UEC-InabaLab/KokoroChat-High): fine-tuned on **2,601 dialogues** with client feedback scores between **70 and 98**
+  - [KokoroChat-Full](https://huggingface.co/UEC-InabaLab/KokoroChat-Full): fine-tuned on **6,471 dialogues** with client feedback scores **≤ 98**
+- 📄 **Paper**: [ACL 2025 Paper (PDF)](https://drive.google.com/file/d/1T6XgvZii8rZ1kKLgOUGqm3BMvqQAvxEM/view?usp=sharing)