Update README.md

README.md

@@ -1,3 +1,96 @@
 ---
 license: mit
+datasets:
+- LGAI-EXAONE/KoMT-Bench
+- skt/kobest_v1
+language:
+- ko
+- en
+base_model:
+- K-intelligence/Midm-2.0-Base-Instruct
+tags:
+- LLM
+- Korean
+- AWQ
+- Quantized
+- Mi:dm
+- transformers
+- Safetensors
 ---
+
+# Midm-2.0-Base-Instruct - AWQ 4-bit Quantized Version
+
+This repository contains the AWQ (Activation-aware Weight Quantization) 4-bit quantized version of the **[K-intelligence/Midm-2.0-Base-Instruct](https://huggingface.co/K-intelligence/Midm-2.0-Base-Instruct)** model by KT AI.
+
+This model is the result of a journey to solve real-world performance and cost issues encountered in a production environment. I hope this experience can be a practical guide for other developers facing similar challenges.
+
+## Model Details
+
+* **Base Model:** `K-intelligence/Midm-2.0-Base-Instruct`
+* **Quantization Method:** AWQ (Activation-aware Weight Quantization)
+* **Quantization Config:**
+    * `w_bit`: 4
+    * `q_group_size`: 128
+    * `zero_point`: True
+* **Library:** `AutoAWQ`
+
+## ⚙️ How to Get Started
+
+To use this model, you will need to install the `transformers`, `accelerate`, and `autoawq` libraries.
+
+```bash
+pip install transformers accelerate autoawq
+Usage Example
+Python
+
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "jinkyeongk/Midm-2.0-Base-Instruct-AWQ"
+
+# Load the tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype=torch.float16
+).eval()
+
+# Construct the chat prompt
+messages = [
+    {"role": "user", "content": "Who are you?"}
+]
+input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+
+# Generate a response
+outputs = model.generate(input_ids, max_new_tokens=512, do_sample=True, temperature=0.7)
+response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
+
+print(response)
+```
+## 📊 Quantization Evaluation
+To measure the performance degradation from quantization, the original (FP16) and quantized (AWQ) models were evaluated against two major Korean benchmarks.
+
+* **Ko-Best**: Measures objective knowledge and reasoning skills (Accuracy).
+
+* **Ko-MTBench**: Measures subjective conversational ability (Scores graded by GPT-4o as a judge).
+
+### Final Evaluation Results
+
+
+| Model | Benchmark | Metric | Score / Accuracy |
+|---|---|---|---|
+| `K-intelligence/Midm-2.0-Base-Instruct` (FP16) | skt/kobest_v1 | hellaswag (Accuracy) | 0.4900 |
+| `jinkyeongk/Midm-2.0-Base-Instruct-AWQ` (AWQ) | skt/kobest_v1 | hellaswag (Accuracy) | **0.4800** |
+| `K-intelligence/Midm-2.0-Base-Instruct` (FP16) | LGAI-EXAONE/KoMT-Bench | Avg. Score (by GPT-4o) | 8.50 / 10.0 |
+| `jinkyeongk/Midm-2.0-Base-Instruct-AWQ` (AWQ) | LGAI-EXAONE/KoMT-Bench | Avg. Score (by GPT-4o) | **6.40 / 10.0** |
+
+## Analysis
+
+The results from the Ko-Best (hellaswag) benchmark show that the performance drop in objective reasoning ability due to AWQ 4-bit quantization was a mere 1.0 percentage point, which is a negligible decrease.
+
+However, in the Ko-MTBench subjective evaluation using GPT-4o as a judge, a more significant performance drop of 2.1 points on average was observed.
+
+This suggests that while AWQ quantization maintains performance on well-defined, knowledge-based tasks like multiple-choice questions (Ko-Best), it can lead to some loss in nuance, expressiveness, or the sophistication of reasoning in more open-ended, conversational tasks (Ko-MTBench).
+
+Therefore, this quantized model offers a massive improvement in speed and cost-efficiency at the expense of a slight trade-off in creative or complex conversational abilities. Users should consider this trade-off based on their specific application.