Upload README.md with huggingface_hub

README.md

@@ -1,21 +1,71 @@
 ---
 base_model: sonodd/qwen3-4b-structeval-sft-v6c-yaml-xml-focus-merged
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- qwen3
-license: apache-2.0
+datasets:
+- u-10bei/dpo-dataset-qwen-cot
 language:
 - en
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- dpo
+- unsloth
+- qwen
+- alignment
+- structured-output
 ---
 
-# Uploaded finetuned  model
+# Qwen3-4B StructEval qwen3-4b-structeval-dpo-v6c
+
+This model is a fine-tuned version of **sonodd/qwen3-4b-structeval-sft-v6c-yaml-xml-focus-merged** using **Direct Preference Optimization (DPO)**
+via the **Unsloth** library.
+
+This repository contains the **full-merged 16-bit weights**. No adapter loading is required.
+
+## Training Objective
+
+This model has been optimized using DPO to align its responses with preferred outputs,
+focusing on improving structured output quality (JSON, YAML, XML, TOML, CSV).
+
+## Training Configuration
+
+- **Base model**: sonodd/qwen3-4b-structeval-sft-v6c-yaml-xml-focus-merged
+- **SFT Adapter**: None (merged SFT used as base)
+- **Method**: DPO (Direct Preference Optimization)
+- **Epochs**: 1
+- **Learning rate**: 1e-07
+- **Beta**: 0.1
+- **Max sequence length**: 1024
+- **LoRA Config**: r=8, alpha=16 (merged into base)
+
+## Usage
+
+Since this is a merged model, you can use it directly with `transformers`.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = "sonodd/qwen3-4b-structeval-dpo-v6c"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+```
+
+## Inference with Standard Code 2
 
-- **Developed by:** sonodd
-- **License:** apache-2.0
-- **Finetuned from model :** sonodd/qwen3-4b-structeval-sft-v6c-yaml-xml-focus-merged
+For inference using the competition's standard code 2, set:
+```python
+MODEL_SOURCE = "merged"
+MERGED_MODEL_ID_OR_PATH = "sonodd/qwen3-4b-structeval-dpo-v6c"
+```
 
-This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Sources & License (IMPORTANT)
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+* **Training Data**: [u-10bei/dpo-dataset-qwen-cot](https://huggingface.co/datasets/u-10bei/dpo-dataset-qwen-cot)
+* **License**: MIT License (as per dataset terms)
+* **Compliance**: Users must follow the original base model's license terms.