Frank commited on
Commit
56cc5e7
·
verified ·
1 Parent(s): 2003672

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -18
README.md CHANGED
@@ -7,26 +7,26 @@ base_model:
7
  - zai-org/GLM-4.5-Air
8
  ---
9
 
10
- # DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation
11
 
12
  ## Model Overview
13
 
14
- DMind-2 is a series of Web3 investment analysis language models designed to provide real-time, professional Web3 investment consulting services for individual investors and professional institutions. Standing on the shoulders of numerous open-source pioneers, we have successfully launched two model variants through innovative post-training techniques. Among these, Dmind-2-Pro demonstrates exceptional depth of understanding and analytical capabilities when addressing complex Web3 ecosystem challenges, delivering comprehensive insights that span from macroeconomic trends to microscopic on-chain behaviors.
15
 
16
  ## Model Variants(DMind-2-107B)
17
  - **Base Model**: GLM-4.5-Air
18
  - **Parameters**: 107B
19
  - **Training Duration**: 1 month of refined post-training
20
  - **Hardware Requirements**:
21
- - **Features**: Its core advantage lies in its ability to deeply integrate macro market trends with micro on-chain activities, possessing a panoramic multi-chain data analysis capability; it can autonomously orchestrate and execute complex on-chain tasks spanning multiple protocols and dozens of steps; and it can synthesize traditional indicators with Web3-native signals such as on-chain data and social sentiment, providing investors with unprecedented deep insights and intelligent decision-making support.
22
 
23
  ## Technical Innovations
24
 
25
  ### 1. Domain-Adaptive Supervised Fine-Tuning (SFT)
26
 
27
- In building DMind-2, we deeply understand the uniqueness of the Web3 investment domain—it requires not only profound blockchain technical understanding but also keen financial market insights, and most importantly, the ability to perform rigorous logical reasoning among complex on-chain data and market signals. Therefore, our domain-adaptive fine-tuning strategy fully considers these requirements from the very beginning of dataset construction. We carefully curated a total of 47.6K high-quality training samples, including 27.8K Web3 domain-specific data points covering comprehensive Web3 investment scenarios from DeFi protocol analysis and NFT valuation models to DAO governance decisions. These data points are not simple Q&A pairs but contain complete investment logic chains, encompassing the entire reasoning process from market observation, data analysis, and risk assessment to investment recommendations.
28
 
29
- To ensure the model maintains fundamental financial analysis capabilities while focusing on the Web3 domain, we specifically incorporated 11.2K high-quality general domain data points and 8.6K pan-financial domain data points. These datasets help the model establish a solid foundation in financial theory and market analysis frameworks, enabling it to creatively apply mature methodologies from traditional finance to the emerging Web3 sector. Through this multi-layered data fusion strategy, DMind-2 can act like a professional investment advisor who understands both technology and finance, providing users with comprehensive and in-depth investment analysis.
30
 
31
  ### 2. 🔥 Core Innovation: Distribution-Preserving Chain-of-Thought Distillation (DPCD)
32
 
@@ -37,7 +37,7 @@ DMind-2's greatest technical breakthrough lies in our innovative Distribution-Pr
37
  The DPCD optimization objective combines domain adaptation with reasoning preservation through the following loss function:
38
 
39
  $$
40
- \mathcal{L}_{\text{DPCD}} = \underbrace{\mathcal{L}_{\text{CE}}(\theta_s, \mathcal{D}_{\text{Web3}})}_{\text{Domain Learning}} + \underbrace{\lambda(t) \cdot \sum_{i=1}^{T} \alpha_i \cdot D_{\text{KL}}(P_{\theta_s}^{(i)} \| P_{\theta_t}^{(i)})}_{\text{Distribution Preservation}} + \underbrace{\beta \cdot \mathcal{L}_{\text{QS}}(\mathcal{C}_{\theta_s})}_{\text{Quality Score}}
41
  $$
42
 
43
  Where:
@@ -58,13 +58,13 @@ $$
58
  \lambda(t) = \begin{cases}
59
  \lambda_{\text{high}} \cdot \left(1 + \tanh\left(\frac{\mathcal{H}(x_t) - \mu_{\mathcal{H}}}{\sigma_{\mathcal{H}}}\right)\right) & \text{if } \mathcal{T}(x_t) \in \{\text{DeFi Analysis, Risk Assessment}\} \\
60
  \lambda_{\text{base}} & \text{if } \mathcal{T}(x_t) \in \{\text{Market Data, Price Query}\} \\
61
- \lambda_{\text{base}} \cdot \left(1 + \frac{\mathcal{S}(c_t)}{|\mathcal{V}_{\text{Web3}}|}\right) & \text{otherwise}
62
  \end{cases}
63
  $$
64
 
65
- Where \\(\mathcal{H}(x_t)\\) measures reasoning complexity through chain length and branching factor, \\(\mathcal{S}(c_t)\\) counts domain-specific terms, and \\(|\mathcal{V}_{\text{Web3}}|\\) is the Web3 vocabulary size.
66
 
67
- This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep Web3 domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism \\(\mathcal{L}_{\text{QS}}\\) filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold \\(\tau = 0.85\\).
68
 
69
  Through extensive experimentation, we found optimal hyperparameters: \\(\lambda_{\text{base}} = 0.3\\), \\(\lambda_{\text{high}} = 0.7\\), \\(\beta = 0.2\\), and \\(\delta = 0.1\\). This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.
70
 
@@ -74,7 +74,7 @@ After completing basic domain fine-tuning, we further optimize the model using t
74
 
75
  For professional output formatting, we constructed 4.2K carefully designed professional format data points. These data samples are sourced from real investment research reports, market analysis documents, and project due diligence reports, covering all aspects of investment analysis. Through RLHF training, the model learned how to organize a professional investment analysis report: starting with an executive summary that clearly articulates investment opportunities and risks; conducting in-depth technical analysis and market evaluation in the main body; and finally providing clear investment recommendations and risk warnings. This structured output not only improves information readability but more importantly helps investors establish systematic analytical frameworks, avoiding impulsive investment decisions due to disorganized information.
76
 
77
- Safety alignment is another aspect we particularly emphasize. The Web3 investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
78
 
79
  ## Performance Metrics
80
 
@@ -90,23 +90,23 @@ Safety alignment is another aspect we particularly emphasize. The Web3 investmen
90
  | | CNMO 2024 (Pass@1) | 78.8 | 86.9 | - | - | 84.1 |
91
  | **Tools** | | | | | | |
92
  | | BFCL_v3 | - | 67.8 | 70.3 | 76.4 | 74.5 |
93
- | **Web3** | | | | | | |
94
  | | DMind Benchmark | 74.1 | 76.3 | 73.4 | 76.8 | 82.2 |
95
 
96
 
97
  ## Application Scenarios
98
 
99
- ### 🎯 Edge-Side Web3 Investment Decision Support
100
 
101
- DMind-2 can provide real-time Web3 investment analysis on users' personal devices, including DeFi yield comparisons, liquidity mining strategy optimization, and NFT valuation analysis. All calculations and analyses are completed locally, ensuring absolute privacy of investment strategies and position information. The model can analyze on-chain data, evaluate project fundamentals, identify market trends, and provide comprehensive support for investment decisions.
102
 
103
  ### 💼 Personalized Financial Advisory Services
104
 
105
- Based on users' risk preferences, investment objectives, and asset allocation needs, DMind-2 can provide customized investment advice. Whether for long-term value investing or short-term arbitrage opportunities, the model can provide professional analysis and recommendations. More importantly, it can explain complex Web3 concepts in plain language, helping investors understand the logic behind every investment decision.
106
 
107
  ### 📊 Comprehensive Financial Investment Computational Analysis
108
 
109
- DMind-2 is not limited to the Web3 domain but also possesses powerful pan-financial computational analysis capabilities. It can perform yield calculations, risk assessments, portfolio optimization, correlation analysis, and other professional financial computations. By integrating traditional financial theory with Web3 innovative mechanisms, the model helps investors find optimal asset allocation solutions between old and new financial systems.
110
 
111
  ### 🔍 Real-Time Market Monitoring and Alerts
112
 
@@ -167,13 +167,13 @@ print(response)
167
  ## Limitations & Disclaimers
168
 
169
  1. **Not Investment Advice**: Model outputs are for reference only; final investment decisions require users' own judgment
170
- 2. **Market Risk**: Web3 markets are highly volatile; please carefully assess risk tolerance
171
  3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
172
  4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
173
 
174
  ## Acknowledgments
175
 
176
- We thank the Qwen and zai teams for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Web3 community.
177
 
178
  ## License
179
 
@@ -183,7 +183,7 @@ This model follows the Apache 2.0 open-source license. Commercial use must compl
183
 
184
  ```bibtex
185
  @misc{dmind2025,
186
- title={DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation},
187
  author={DMind Team},
188
  year={2025},
189
  publisher={Hugging Face}
 
7
  - zai-org/GLM-4.5-Air
8
  ---
9
 
10
+ # DMind-2: Advanced Crypto Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation
11
 
12
  ## Model Overview
13
 
14
+ DMind-2 is a series of crypto investment analysis language models designed to provide real-time, professional crypto investment consulting services for individual investors and professional institutions. Standing on the shoulders of numerous open-source pioneers, we have successfully launched two model variants through innovative post-training techniques. Among these, Dmind-2-Pro demonstrates exceptional depth of understanding and analytical capabilities when addressing complex crypto ecosystem challenges, delivering comprehensive insights that span from macroeconomic trends to microscopic on-chain behaviors.
15
 
16
  ## Model Variants(DMind-2-107B)
17
  - **Base Model**: GLM-4.5-Air
18
  - **Parameters**: 107B
19
  - **Training Duration**: 1 month of refined post-training
20
  - **Hardware Requirements**:
21
+ - **Features**: Its core advantage lies in its ability to deeply integrate macro market trends with micro on-chain activities, possessing a panoramic multi-chain data analysis capability; it can autonomously orchestrate and execute complex on-chain tasks spanning multiple protocols and dozens of steps; and it can synthesize traditional indicators with crypto-native signals such as on-chain data and social sentiment, providing investors with unprecedented deep insights and intelligent decision-making support.
22
 
23
  ## Technical Innovations
24
 
25
  ### 1. Domain-Adaptive Supervised Fine-Tuning (SFT)
26
 
27
+ In building DMind-2, we deeply understand the uniqueness of the crypto investment domain—it requires not only profound blockchain technical understanding but also keen financial market insights, and most importantly, the ability to perform rigorous logical reasoning among complex on-chain data and market signals. Therefore, our domain-adaptive fine-tuning strategy fully considers these requirements from the very beginning of dataset construction. We carefully curated a total of 47.6K high-quality training samples, including 27.8K crypto domain-specific data points covering comprehensive crypto investment scenarios from DeFi protocol analysis and NFT valuation models to DAO governance decisions. These data points are not simple Q&A pairs but contain complete investment logic chains, encompassing the entire reasoning process from market observation, data analysis, and risk assessment to investment recommendations.
28
 
29
+ To ensure the model maintains fundamental financial analysis capabilities while focusing on the crypto domain, we specifically incorporated 11.2K high-quality general domain data points and 8.6K pan-financial domain data points. These datasets help the model establish a solid foundation in financial theory and market analysis frameworks, enabling it to creatively apply mature methodologies from traditional finance to the emerging crypto sector. Through this multi-layered data fusion strategy, DMind-2 can act like a professional investment advisor who understands both technology and finance, providing users with comprehensive and in-depth investment analysis.
30
 
31
  ### 2. 🔥 Core Innovation: Distribution-Preserving Chain-of-Thought Distillation (DPCD)
32
 
 
37
  The DPCD optimization objective combines domain adaptation with reasoning preservation through the following loss function:
38
 
39
  $$
40
+ \mathcal{L}_{\text{DPCD}} = \underbrace{\mathcal{L}_{\text{CE}}(\theta_s, \mathcal{D}_{\text{crypto}})}_{\text{Domain Learning}} + \underbrace{\lambda(t) \cdot \sum_{i=1}^{T} \alpha_i \cdot D_{\text{KL}}(P_{\theta_s}^{(i)} \| P_{\theta_t}^{(i)})}_{\text{Distribution Preservation}} + \underbrace{\beta \cdot \mathcal{L}_{\text{QS}}(\mathcal{C}_{\theta_s})}_{\text{Quality Score}}
41
  $$
42
 
43
  Where:
 
58
  \lambda(t) = \begin{cases}
59
  \lambda_{\text{high}} \cdot \left(1 + \tanh\left(\frac{\mathcal{H}(x_t) - \mu_{\mathcal{H}}}{\sigma_{\mathcal{H}}}\right)\right) & \text{if } \mathcal{T}(x_t) \in \{\text{DeFi Analysis, Risk Assessment}\} \\
60
  \lambda_{\text{base}} & \text{if } \mathcal{T}(x_t) \in \{\text{Market Data, Price Query}\} \\
61
+ \lambda_{\text{base}} \cdot \left(1 + \frac{\mathcal{S}(c_t)}{|\mathcal{V}_{\text{crypto}}|}\right) & \text{otherwise}
62
  \end{cases}
63
  $$
64
 
65
+ Where \\(\mathcal{H}(x_t)\\) measures reasoning complexity through chain length and branching factor, \\(\mathcal{S}(c_t)\\) counts domain-specific terms, and \\(|\mathcal{V}_{\text{crypto}}|\\) is the crypto vocabulary size.
66
 
67
+ This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep crypto domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism \\(\mathcal{L}_{\text{QS}}\\) filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold \\(\tau = 0.85\\).
68
 
69
  Through extensive experimentation, we found optimal hyperparameters: \\(\lambda_{\text{base}} = 0.3\\), \\(\lambda_{\text{high}} = 0.7\\), \\(\beta = 0.2\\), and \\(\delta = 0.1\\). This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.
70
 
 
74
 
75
  For professional output formatting, we constructed 4.2K carefully designed professional format data points. These data samples are sourced from real investment research reports, market analysis documents, and project due diligence reports, covering all aspects of investment analysis. Through RLHF training, the model learned how to organize a professional investment analysis report: starting with an executive summary that clearly articulates investment opportunities and risks; conducting in-depth technical analysis and market evaluation in the main body; and finally providing clear investment recommendations and risk warnings. This structured output not only improves information readability but more importantly helps investors establish systematic analytical frameworks, avoiding impulsive investment decisions due to disorganized information.
76
 
77
+ Safety alignment is another aspect we particularly emphasize. The crypto investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
78
 
79
  ## Performance Metrics
80
 
 
90
  | | CNMO 2024 (Pass@1) | 78.8 | 86.9 | - | - | 84.1 |
91
  | **Tools** | | | | | | |
92
  | | BFCL_v3 | - | 67.8 | 70.3 | 76.4 | 74.5 |
93
+ | **Crypto** | | | | | | |
94
  | | DMind Benchmark | 74.1 | 76.3 | 73.4 | 76.8 | 82.2 |
95
 
96
 
97
  ## Application Scenarios
98
 
99
+ ### 🎯 Edge-Side Crypto Investment Decision Support
100
 
101
+ DMind-2 can provide real-time crypto investment analysis on users' personal devices, including DeFi yield comparisons, liquidity mining strategy optimization, and NFT valuation analysis. All calculations and analyses are completed locally, ensuring absolute privacy of investment strategies and position information. The model can analyze on-chain data, evaluate project fundamentals, identify market trends, and provide comprehensive support for investment decisions.
102
 
103
  ### 💼 Personalized Financial Advisory Services
104
 
105
+ Based on users' risk preferences, investment objectives, and asset allocation needs, DMind-2 can provide customized investment advice. Whether for long-term value investing or short-term arbitrage opportunities, the model can provide professional analysis and recommendations. More importantly, it can explain complex crypto concepts in plain language, helping investors understand the logic behind every investment decision.
106
 
107
  ### 📊 Comprehensive Financial Investment Computational Analysis
108
 
109
+ DMind-2 is not limited to the crypto domain but also possesses powerful pan-financial computational analysis capabilities. It can perform yield calculations, risk assessments, portfolio optimization, correlation analysis, and other professional financial computations. By integrating traditional financial theory with crypto innovative mechanisms, the model helps investors find optimal asset allocation solutions between old and new financial systems.
110
 
111
  ### 🔍 Real-Time Market Monitoring and Alerts
112
 
 
167
  ## Limitations & Disclaimers
168
 
169
  1. **Not Investment Advice**: Model outputs are for reference only; final investment decisions require users' own judgment
170
+ 2. **Market Risk**: Crypto markets are highly volatile; please carefully assess risk tolerance
171
  3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
172
  4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
173
 
174
  ## Acknowledgments
175
 
176
+ We thank the Qwen and zai teams for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Crypto community.
177
 
178
  ## License
179
 
 
183
 
184
  ```bibtex
185
  @misc{dmind2025,
186
+ title={DMind-2: Advanced Crypto Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation},
187
  author={DMind Team},
188
  year={2025},
189
  publisher={Hugging Face}