EunsuKim commited on
Commit
adf1b40
·
verified ·
1 Parent(s): e71bf44

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -3
README.md CHANGED
@@ -1,3 +1,59 @@
1
- ---
2
- license: cc-by-4.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - LLM
7
+ - classification
8
+ - instruction-tuned
9
+ - multi-label
10
+ - qwen
11
+ datasets:
12
+ - custom
13
+ pipeline_tag: text-classification
14
+ ---
15
+
16
+ # BenchHub-Cat-7b
17
+
18
+ **BenchHub-Cat-7b** is a 7B parameter instruction-tuned language model that performs structured classification of natural language queries into three dimensions:
19
+
20
+ - `subject`: Topic domain of the query (e.g., law, health, travel)
21
+ - `skill`: Type of skill or task (e.g., reasoning, explanation, comparison)
22
+ - `target`: General or cultural-specific target audience
23
+
24
+ It is based on the Qwen2.5-7B-Instruct architecture and trained on a mixture of synthetic and GPT-generated instruction data.
25
+
26
+ ## 🔧 Model Details
27
+
28
+ - **Base Model**: Qwen2.5-7B-Instruct
29
+ - **Task**: Structured triple-label classification
30
+ - **Prompt Format**: Instruction-style with output structure
31
+ - **Training Framework**: Axolotl + DeepSpeed ZeRO-3
32
+
33
+ ## 🧪 Training Configuration
34
+
35
+ | Hyperparameter | Value |
36
+ |--------------------------|----------------------|
37
+ | Sequence Length | 8192 |
38
+ | Learning Rate | 2 × 10⁻⁵ |
39
+ | Batch Size (Effective) | 256 |
40
+ | Epochs | 3 |
41
+ | Scheduler | Cosine Decay |
42
+ | Warmup Ratio | 0.05 |
43
+ | Optimizer | Method from [19] |
44
+ | Hardware | 4× A6000 48GB GPUs |
45
+ | Training Time | ~5 hours per run |
46
+
47
+ ## 🧠 Intended Use
48
+
49
+ **Input**: Open-ended natural language queries
50
+ **Output**: Structured classification result with 3 fields
51
+
52
+ ### Example Categories:
53
+ - `subject`: education, health, history, law, etc.
54
+ - `skill`: reasoning, recall, summarization, etc.
55
+ - `target`: general, cultural-specific
56
+
57
+ ### ✨ Example Prompt & Output
58
+
59
+ #### 📝 Prompt