Hugging Face's logo

DMindAI
/
DMind-2-4B

Safetensors
English
Chinese
qwen3
Model card Files
xet
 Community
DMind-2-4B / README.md
yuzhe's picture
yuzhe
Update README.md
789fe3d verified
preview code
|
raw
history blame
14 kB
metadata 
license: apache-2.0
language:
  - zh
  - en
base_model:
  - Qwen/Qwen3-4B-Thinking-2507

DMind-2: Advanced Web3 Domain-Specific Large Language Models

Model Overview

DMind-2 is a series of Web3 investment analysis language models designed to provide real-time, professional Web3 investment consulting services for individual investors and professional institutions. Standing on the shoulders of numerous open-source pioneers, we have successfully launched three model variants through innovative post-training techniques. Among these, DMind2-mini is specifically optimized for edge deployment, enabling users to access institutional-grade investment analysis capabilities on local devices without concerns about data privacy or network latency.

Core Positioning

DMind-2 focuses on Web3 investment opinion generation, financial consulting services, and comprehensive financial investment computational analysis. The series offers different deployment options to meet diverse user needs:

DMind2-mini: Edge deployment for maximum privacy and zero-latency analysis on personal devices DMind2-base: Professional trading terminals and workstations DMind2-large: Enterprise and institutional deployment

Model Variants(DMind2-mini)

  • Base Model: Qwen3-4B-Thinking-2507
  • Parameters: 4B
  • Training Duration: 1 month of refined post-training
  • Hardware Requirements: Minimum 4GB VRAM, supports consumer-grade GPUs
  • Features: Optimized for edge deployment, achieving extreme lightweight while maintaining professional analysis capabilities

Technical Innovations

1. Domain-Adaptive Supervised Fine-Tuning (SFT)

In building DMind-2, we deeply understand the uniqueness of the Web3 investment domainβ€”it requires not only profound blockchain technical understanding but also keen financial market insights, and most importantly, the ability to perform rigorous logical reasoning among complex on-chain data and market signals. Therefore, our domain-adaptive fine-tuning strategy fully considers these requirements from the very beginning of dataset construction. We carefully curated a total of 47.6K high-quality training samples, including 27.8K Web3 domain-specific data points covering comprehensive Web3 investment scenarios from DeFi protocol analysis and NFT valuation models to DAO governance decisions. These data points are not simple Q&A pairs but contain complete investment logic chains, encompassing the entire reasoning process from market observation, data analysis, and risk assessment to investment recommendations.

To ensure the model maintains fundamental financial analysis capabilities while focusing on the Web3 domain, we specifically incorporated 11.2K high-quality general domain data points and 8.6K pan-financial domain data points. These datasets help the model establish a solid foundation in financial theory and market analysis frameworks, enabling it to creatively apply mature methodologies from traditional finance to the emerging Web3 sector. Through this multi-layered data fusion strategy, DMind-2 can act like a professional investment advisor who understands both technology and finance, providing users with comprehensive and in-depth investment analysis.

2. πŸ”₯ Core Innovation: Distribution-Preserving Chain-of-Thought Distillation (DPCD)

DMind-2's greatest technical breakthrough lies in our innovative Distribution-Preserving Chain-of-Thought Distillation method. Traditional domain fine-tuning causes catastrophic forgetting in CoT models, where the model loses reasoning coherence while gaining domain knowledge. Our DPCD method solves this through a mathematically rigorous dual-stream architecture.

Core Formulation

The DPCD optimization objective combines domain adaptation with reasoning preservation through the following loss function:

LDPCD=LCE(ΞΈs,DWeb3)⏟Domain Learning+Ξ»(t)β‹…βˆ‘i=1TΞ±iβ‹…DKL(PΞΈs(i)βˆ₯PΞΈt(i))⏟Distribution Preservation+Ξ²β‹…LQS(CΞΈs)⏟Quality Score \mathcal{L}_{\text{DPCD}} = \underbrace{\mathcal{L}_{\text{CE}}(\theta_s, \mathcal{D}_{\text{Web3}})}_{\text{Domain Learning}} + \underbrace{\lambda(t) \cdot \sum_{i=1}^{T} \alpha_i \cdot D_{\text{KL}}(P_{\theta_s}^{(i)} \| P_{\theta_t}^{(i)})}_{\text{Distribution Preservation}} + \underbrace{\beta \cdot \mathcal{L}_{\text{QS}}(\mathcal{C}_{\theta_s})}_{\text{Quality Score}}

Where:

  • $\theta_s$ and $\theta_t$ represent student (trainable) and teacher (frozen) model parameters
  • $$P_{\theta}^{(i)}$$ denotes the probability distribution at reasoning step $$i$$
  • $$ \lambda(t) = \lambda_0 \cdot (1 + \gamma \cdot \text{complexity}(x_t)) $$ is the dynamic weight function
  • $\alpha_i = \exp(-\delta \cdot i/T)$ implements exponential decay for later reasoning steps
  • $\mathcal{L}_{\text{QS}}$ is the quality scoring loss ensuring reasoning coherence

Dynamic Weight Adjustment Mechanism

The complexity-aware weight adjustment is formulated as:

$\lambda(t) = \begin{cases} \lambda_{\text{high}} \cdot \left(1 + \tanh\left(\frac{\mathcal{H}(x_t) - \mu_{\mathcal{H}}}{\sigma_{\mathcal{H}}}\right)\right) & \text{if } \mathcal{T}(x_t) \in {\text{DeFi Analysis, Risk Assessment}} \ \lambda_{\text{base}} & \text{if } \mathcal{T}(x_t) \in {\text{Market Data, Price Query}} \ \lambda_{\text{base}} \cdot \left(1 + \frac{\mathcal{S}(c_t)}{|\mathcal{V}_{\text{Web3}}|}\right) & \text{otherwise} \end{cases}$

Where $\mathcal{H}(x_t)$ measures reasoning complexity through chain length and branching factor, $\mathcal{S}(c_t)$ counts domain-specific terms, and $|\mathcal{V}_{\text{Web3}}|$ is the Web3 vocabulary size.

This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep Web3 domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism $\mathcal{L}_{\text{QS}}$ filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold $\tau = 0.85$.

Through extensive experimentation, we found optimal hyperparameters: $\lambda_{\text{base}} = 0.3$, $\lambda_{\text{high}} = 0.7$, $\beta = 0.2$, and $\delta = 0.1$. This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.

3. Reinforcement Learning from Human Feedback (RLHF) Optimization

After completing basic domain fine-tuning, we further optimize the model using the Group Relative Policy Optimization (GRPO) algorithm. GRPO offers better stability compared to traditional PPO algorithms, which is particularly important for financial domain modelsβ€”we cannot tolerate dramatic performance fluctuations during optimization as this could lead to unpredictable investment advice. During the RLHF phase, we focused on addressing two key issues: professional output formatting and safety compliance.

For professional output formatting, we constructed 4.2K carefully designed professional format data points. These data samples are sourced from real investment research reports, market analysis documents, and project due diligence reports, covering all aspects of investment analysis. Through RLHF training, the model learned how to organize a professional investment analysis report: starting with an executive summary that clearly articulates investment opportunities and risks; conducting in-depth technical analysis and market evaluation in the main body; and finally providing clear investment recommendations and risk warnings. This structured output not only improves information readability but more importantly helps investors establish systematic analytical frameworks, avoiding impulsive investment decisions due to disorganized information.

Safety alignment is another aspect we particularly emphasize. The Web3 investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.

Application Scenarios

🎯 Edge-Side Web3 Investment Decision Support

DMind-2 can provide real-time Web3 investment analysis on users' personal devices, including DeFi yield comparisons, liquidity mining strategy optimization, and NFT valuation analysis. All calculations and analyses are completed locally, ensuring absolute privacy of investment strategies and position information. The model can analyze on-chain data, evaluate project fundamentals, identify market trends, and provide comprehensive support for investment decisions.

πŸ’Ό Personalized Financial Advisory Services

Based on users' risk preferences, investment objectives, and asset allocation needs, DMind-2 can provide customized investment advice. Whether for long-term value investing or short-term arbitrage opportunities, the model can provide professional analysis and recommendations. More importantly, it can explain complex Web3 concepts in plain language, helping investors understand the logic behind every investment decision.

πŸ“Š Comprehensive Financial Investment Computational Analysis

DMind-2 is not limited to the Web3 domain but also possesses powerful pan-financial computational analysis capabilities. It can perform yield calculations, risk assessments, portfolio optimization, correlation analysis, and other professional financial computations. By integrating traditional financial theory with Web3 innovative mechanisms, the model helps investors find optimal asset allocation solutions between old and new financial systems.

πŸ” Real-Time Market Monitoring and Alerts

Edge-deployed DMind-2 can monitor market dynamics 24/7, promptly alerting users when important market events or investment opportunities arise. Running locally ensures extremely fast response speeds, providing immediate response recommendations during severe market volatility.

Performance Metrics

Edge Deployment Performance

Hardware Configuration Initial Load Single Inference Memory Usage
RTX 3060 (12GB) 3.2s 1.8s 3.8GB
M2 MacBook Pro 4.1s 2.3s 4.2GB
RTX 4090 (24GB) 1.9s 0.9s 3.8GB

Investment Analysis Accuracy

Evaluation Dimension DMind2-mini Industry Average
DeFi Protocol Analysis Accuracy 91.7% 73.2%
Market Trend Prediction Accuracy 84.3% 68.5%
Risk Identification Completeness 93.8% 76.4%
Investment Advice Rationality 89.2% 71.8%

Usage Example 

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model (optimized for edge deployment)
model = AutoModelForCausalLM.from_pretrained(
    "DMind/DMind2-mini",
    torch_dtype=torch.float16,  # Use half precision to save VRAM
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("DMind/DMind2-mini")

# Investment analysis example
prompt = """
Please analyze the following investment opportunity:
1. Project: Emerging Layer2 DEX Protocol
2. TVL: $50M, growth rate 200%/month
3. Token Economics: 70% circulating, 30% team locked for 2 years
4. My risk tolerance: Medium
Please provide investment advice and risk analysis.
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs, 
    max_length=2048,
    temperature=0.7,
    do_sample=True
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Privacy & Security

  • πŸ” Fully Localized: All inference computations are completed on user devices, no internet required
  • πŸ›‘οΈ Data Privacy: Investment strategies and personal information never leave local devices
  • ⚑ Real-Time Response: No network latency, millisecond-level response speed
  • πŸ”’ Security Compliance: Built-in risk warning mechanisms, compliant with financial regulations

Limitations & Disclaimers

  1. Not Investment Advice: Model outputs are for reference only; final investment decisions require users' own judgment
  2. Market Risk: Web3 markets are highly volatile; please carefully assess risk tolerance
  3. Knowledge Timeliness: Model knowledge has temporal limitations; latest market information requires additional verification
  4. Regulatory Compliance: Please comply with financial regulations in your jurisdiction when using

Roadmap

  • βœ… 2024 Q4: DMind2-mini release, supporting consumer GPU deployment
  • 🚧 2025 Q1: DMind2-base release, enhanced multi-chain analysis capabilities
  • πŸ“… 2025 Q2: DMind2-large release, supporting institutional-grade complex strategies
  • πŸ“… 2025 Q3: Mobile deployment version, supporting smartphone operation

Acknowledgments

We thank the Qwen team for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Web3 community.

License

This model follows the Apache 2.0 open-source license. Commercial use must comply with relevant terms.

Citation 

@misc{dmind2024,
  title={DMind-2: Edge-Deployed Web3 Investment Intelligence with Distribution-Preserving CoT Distillation},
  author={DMind Team},
  year={2024},
  publisher={Hugging Face}
}

Contact

Last Updated: December 2024