Update README.md
Browse files
README.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
-
- zh
|
| 5 |
- en
|
|
|
|
| 6 |
base_model:
|
| 7 |
- Qwen/Qwen3-4B-Thinking-2507
|
| 8 |
---
|
| 9 |
|
| 10 |
-
# DMind-2: Advanced Web3 Domain-Specific Large Language Models
|
| 11 |
|
| 12 |
## Model Overview
|
| 13 |
|
|
@@ -64,11 +64,11 @@ $$
|
|
| 64 |
\end{cases}
|
| 65 |
$$
|
| 66 |
|
| 67 |
-
Where
|
| 68 |
|
| 69 |
-
This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep Web3 domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism
|
| 70 |
|
| 71 |
-
Through extensive experimentation, we found optimal hyperparameters:
|
| 72 |
|
| 73 |
### 3. Reinforcement Learning from Human Feedback (RLHF) Optimization
|
| 74 |
|
|
@@ -78,6 +78,24 @@ For professional output formatting, we constructed 4.2K carefully designed profe
|
|
| 78 |
|
| 79 |
Safety alignment is another aspect we particularly emphasize. The Web3 investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
## Application Scenarios
|
| 82 |
|
| 83 |
### 🎯 Edge-Side Web3 Investment Decision Support
|
|
@@ -96,22 +114,6 @@ DMind-2 is not limited to the Web3 domain but also possesses powerful pan-financ
|
|
| 96 |
|
| 97 |
Edge-deployed DMind-2 can monitor market dynamics 24/7, promptly alerting users when important market events or investment opportunities arise. Running locally ensures extremely fast response speeds, providing immediate response recommendations during severe market volatility.
|
| 98 |
|
| 99 |
-
## Performance Metrics
|
| 100 |
-
|
| 101 |
-
### Edge Deployment Performance
|
| 102 |
-
| Hardware Configuration | Initial Load | Single Inference | Memory Usage |
|
| 103 |
-
|------------------------|--------------|------------------|--------------|
|
| 104 |
-
| RTX 3060 (12GB) | 3.2s | 1.8s | 3.8GB |
|
| 105 |
-
| M2 MacBook Pro | 4.1s | 2.3s | 4.2GB |
|
| 106 |
-
| RTX 4090 (24GB) | 1.9s | 0.9s | 3.8GB |
|
| 107 |
-
|
| 108 |
-
### Investment Analysis Accuracy
|
| 109 |
-
| Evaluation Dimension | DMind2-mini | Industry Average |
|
| 110 |
-
|---------------------|-------------|------------------|
|
| 111 |
-
| DeFi Protocol Analysis Accuracy | 91.7% | 73.2% |
|
| 112 |
-
| Market Trend Prediction Accuracy | 84.3% | 68.5% |
|
| 113 |
-
| Risk Identification Completeness | 93.8% | 76.4% |
|
| 114 |
-
| Investment Advice Rationality | 89.2% | 71.8% |
|
| 115 |
|
| 116 |
## Usage Example
|
| 117 |
|
|
@@ -162,16 +164,9 @@ print(response)
|
|
| 162 |
3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
|
| 163 |
4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
|
| 164 |
|
| 165 |
-
## Roadmap
|
| 166 |
-
|
| 167 |
-
- ✅ 2024 Q4: DMind2-mini release, supporting consumer GPU deployment
|
| 168 |
-
- 🚧 2025 Q1: DMind2-base release, enhanced multi-chain analysis capabilities
|
| 169 |
-
- 📅 2025 Q2: DMind2-large release, supporting institutional-grade complex strategies
|
| 170 |
-
- 📅 2025 Q3: Mobile deployment version, supporting smartphone operation
|
| 171 |
-
|
| 172 |
## Acknowledgments
|
| 173 |
|
| 174 |
-
We thank the Qwen
|
| 175 |
|
| 176 |
## License
|
| 177 |
|
|
@@ -181,7 +176,7 @@ This model follows the Apache 2.0 open-source license. Commercial use must compl
|
|
| 181 |
|
| 182 |
```bibtex
|
| 183 |
@misc{dmind2024,
|
| 184 |
-
title={DMind-2:
|
| 185 |
author={DMind Team},
|
| 186 |
year={2024},
|
| 187 |
publisher={Hugging Face}
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
|
|
|
| 4 |
- en
|
| 5 |
+
- zh
|
| 6 |
base_model:
|
| 7 |
- Qwen/Qwen3-4B-Thinking-2507
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation
|
| 11 |
|
| 12 |
## Model Overview
|
| 13 |
|
|
|
|
| 64 |
\end{cases}
|
| 65 |
$$
|
| 66 |
|
| 67 |
+
Where \\(\mathcal{H}(x_t)\\) measures reasoning complexity through chain length and branching factor, \\(\mathcal{S}(c_t)\\) counts domain-specific terms, and \\(|\mathcal{V}_{\text{Web3}}|\\) is the Web3 vocabulary size.
|
| 68 |
|
| 69 |
+
This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep Web3 domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism \\(\mathcal{L}_{\text{QS}}\\) filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold \\(\tau = 0.85\\).
|
| 70 |
|
| 71 |
+
Through extensive experimentation, we found optimal hyperparameters: \\(\lambda_{\text{base}} = 0.3\\), \\(\lambda_{\text{high}} = 0.7\\), \\(\beta = 0.2\\), and \\(\delta = 0.1\\). This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.
|
| 72 |
|
| 73 |
### 3. Reinforcement Learning from Human Feedback (RLHF) Optimization
|
| 74 |
|
|
|
|
| 78 |
|
| 79 |
Safety alignment is another aspect we particularly emphasize. The Web3 investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
|
| 80 |
|
| 81 |
+
## Performance Metrics
|
| 82 |
+
|
| 83 |
+
| Category | Benchmark (Metric) | DeepSeek-R1-0528-Qwen3-8B | gpt-oss-20b | Qwen3-32B | Qwen3-4B(Thinking) | DMind2-mini(4B) |
|
| 84 |
+
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
| 85 |
+
| **General** | | | | | | |
|
| 86 |
+
| | MMLU-Pro (EM) | 84.0 | 85.0 | - | - | 82.1 |
|
| 87 |
+
| | GPQA-Diamond (Pass@1) | 71.5 | 81.0 | - | - | 64.2 |
|
| 88 |
+
| | SimpleQA (Correct) | 30.1 | 27.8 | - | - | - |
|
| 89 |
+
| **Math** | | | | | | |
|
| 90 |
+
| | AIME 2024 (Pass@1) | 79.8 | 91.4 | - | - | 92.7 |
|
| 91 |
+
| | AIME 2025 (Pass@1) | 70.0 | 87.5 | - | - | 81.6 |
|
| 92 |
+
| | CNMO 2024 (Pass@1) | 78.8 | 86.9 | - | - | 82.4 |
|
| 93 |
+
| **Tools** | | | | | | |
|
| 94 |
+
| | BFCL_v3 | - | 37.0 | - | - | 70.2 |
|
| 95 |
+
| **Web3** | | | | | | |
|
| 96 |
+
| | DMind Benchmark | - | - | - | - | - |
|
| 97 |
+
|
| 98 |
+
|
| 99 |
## Application Scenarios
|
| 100 |
|
| 101 |
### 🎯 Edge-Side Web3 Investment Decision Support
|
|
|
|
| 114 |
|
| 115 |
Edge-deployed DMind-2 can monitor market dynamics 24/7, promptly alerting users when important market events or investment opportunities arise. Running locally ensures extremely fast response speeds, providing immediate response recommendations during severe market volatility.
|
| 116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
## Usage Example
|
| 119 |
|
|
|
|
| 164 |
3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
|
| 165 |
4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
|
| 166 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
## Acknowledgments
|
| 168 |
|
| 169 |
+
We thank the Qwen and zai teams for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Web3 community.
|
| 170 |
|
| 171 |
## License
|
| 172 |
|
|
|
|
| 176 |
|
| 177 |
```bibtex
|
| 178 |
@misc{dmind2024,
|
| 179 |
+
title={DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation},
|
| 180 |
author={DMind Team},
|
| 181 |
year={2024},
|
| 182 |
publisher={Hugging Face}
|