Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,59 @@
|
|
| 1 |
-
---
|
| 2 |
-
license:
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- LLM
|
| 7 |
+
- classification
|
| 8 |
+
- instruction-tuned
|
| 9 |
+
- multi-label
|
| 10 |
+
- qwen
|
| 11 |
+
datasets:
|
| 12 |
+
- custom
|
| 13 |
+
pipeline_tag: text-classification
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# BenchHub-Cat-7b
|
| 17 |
+
|
| 18 |
+
**BenchHub-Cat-7b** is a 7B parameter instruction-tuned language model that performs structured classification of natural language queries into three dimensions:
|
| 19 |
+
|
| 20 |
+
- `subject`: Topic domain of the query (e.g., law, health, travel)
|
| 21 |
+
- `skill`: Type of skill or task (e.g., reasoning, explanation, comparison)
|
| 22 |
+
- `target`: General or cultural-specific target audience
|
| 23 |
+
|
| 24 |
+
It is based on the Qwen2.5-7B-Instruct architecture and trained on a mixture of synthetic and GPT-generated instruction data.
|
| 25 |
+
|
| 26 |
+
## 🔧 Model Details
|
| 27 |
+
|
| 28 |
+
- **Base Model**: Qwen2.5-7B-Instruct
|
| 29 |
+
- **Task**: Structured triple-label classification
|
| 30 |
+
- **Prompt Format**: Instruction-style with output structure
|
| 31 |
+
- **Training Framework**: Axolotl + DeepSpeed ZeRO-3
|
| 32 |
+
|
| 33 |
+
## 🧪 Training Configuration
|
| 34 |
+
|
| 35 |
+
| Hyperparameter | Value |
|
| 36 |
+
|--------------------------|----------------------|
|
| 37 |
+
| Sequence Length | 8192 |
|
| 38 |
+
| Learning Rate | 2 × 10⁻⁵ |
|
| 39 |
+
| Batch Size (Effective) | 256 |
|
| 40 |
+
| Epochs | 3 |
|
| 41 |
+
| Scheduler | Cosine Decay |
|
| 42 |
+
| Warmup Ratio | 0.05 |
|
| 43 |
+
| Optimizer | Method from [19] |
|
| 44 |
+
| Hardware | 4× A6000 48GB GPUs |
|
| 45 |
+
| Training Time | ~5 hours per run |
|
| 46 |
+
|
| 47 |
+
## 🧠 Intended Use
|
| 48 |
+
|
| 49 |
+
**Input**: Open-ended natural language queries
|
| 50 |
+
**Output**: Structured classification result with 3 fields
|
| 51 |
+
|
| 52 |
+
### Example Categories:
|
| 53 |
+
- `subject`: education, health, history, law, etc.
|
| 54 |
+
- `skill`: reasoning, recall, summarization, etc.
|
| 55 |
+
- `target`: general, cultural-specific
|
| 56 |
+
|
| 57 |
+
### ✨ Example Prompt & Output
|
| 58 |
+
|
| 59 |
+
#### 📝 Prompt
|