Update README.md

README.md

@@ -7,4 +7,44 @@ base_model:
 tags:
 - Korean
 - Culture
----
+---
+
+
+# Midm-KCulture-2.0-Base-Instruct
+- This model is fine-tuned from KT/Midm-2.0-Base-Instruct on the 'Korean Culture Q&A Corpus' using the LoRA (Low-Rank Adaptation) methodology.
+
+## Training Hyperparameters
+
+| Hyperparameter                | Value                         |
+| :---------------------------- | :---------------------------- |
+| **SFTConfig** |                               |
+| `torch_dtype`                 | `bfloat16`                    |
+| `seed`                        | `42`                          |
+| `epoch`                       | `3`                           |
+| `per_device_train_batch_size` | `2`                           |
+| `per_device_eval_batch_size`  | `2`                           |
+| `learning_rate`               | `0.0002`                      |
+| `lr_scheduler_type`           | `"linear"`                    |
+| `max_grad_norm`               | `1.0`                         |
+| `neftune_noise_alpha`         | `None`                        |
+| `gradient_accumulation_steps` | `1`                           |
+| `gradient_checkpointing`      | `False`                       |
+| `max_seq_length`              | `1024`                        |
+| **LoraConfig** |                               |
+| `r`                           | `16`                          |
+| `lora_alpha`                  | `16`                          |
+| `lora_dropout`                | `0.1`                         |
+| `target_modules`              | `["q_proj", "v_proj"]`        |
+
+## Usage
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model_name = "jjae/Midm-KCulture-2.0-Base-Instruct"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True,
+    device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+```
+