solidity-vulnerability-detector / README_experts.md
jhsu12's picture
Add multi-expert training scripts and design document
eb08b83 verified
|
Raw
History Blame Contribute Delete
2.49 kB
See design_document.md for architecture details.
## Multi-Expert Smart Contract Vulnerability Detector
This project uses a **Router + Type-Specific Expert Adapters** architecture to improve smart contract vulnerability detection.
### Problem Diagnosis
Current single-model approach achieves only:
- Binary F1: 0.7673 (with 25% parse failures)
- Access Control F1: 0.0235 (essentially undetected)
- Reentrancy precision: 0.0663 (massive hallucinations)
### Root Causes
1. Severe class imbalance: Integer Overflow 3412 vs tx.origin 11
2. Single model trying to learn 6 distinct vulnerability patterns
3. No reasoning chains — model memorizes shortcuts
4. Rigid output format causes 25% parse failures
### Solution: 5 Expert Adapters
Each expert is a LoRA adapter trained to answer:
**"Is this contract vulnerable with MY specific vulnerability type?"**
| Expert | Dataset | Train Pos/Neg |
|--------|---------|---------------|
| Reentrancy | [jhsu12/solidity-vuln-expert-reentrancy](https://hf.co/datasets/jhsu12/solidity-vuln-expert-reentrancy) | 767 / 2301 |
| Access Control | [jhsu12/solidity-vuln-expert-access-control](https://hf.co/datasets/jhsu12/solidity-vuln-expert-access-control) | 219 / 657 |
| Integer Overflow | [jhsu12/solidity-vuln-expert-integer-overflow-underflow](https://hf.co/datasets/jhsu12/solidity-vuln-expert-integer-overflow-underflow) | 3412 / 6670 |
| Timestamp Dependence | [jhsu12/solidity-vuln-expert-timestamp-dependence](https://hf.co/datasets/jhsu12/solidity-vuln-expert-timestamp-dependence) | 858 / 2574 |
| Unchecked Low-Level Calls | [jhsu12/solidity-vuln-expert-unchecked-low-level-calls](https://hf.co/datasets/jhsu12/solidity-vuln-expert-unchecked-low-level-calls) | 215 / 645 |
### Training
```bash
# Train each expert (can run in parallel)
python train_expert.py --expert "Reentrancy"
python train_expert.py --expert "Access Control"
python train_expert.py --expert "Integer Overflow/Underflow"
python train_expert.py --expert "Timestamp Dependence"
python train_expert.py --expert "Unchecked Low-Level Calls"
```
### Evaluation
```bash
python evaluate_experts.py --max_samples 200
```
### Research References
- **VulnLLM-R** (arXiv:2512.07533): Reasoning models with distillation
- **Smart-LLaMA-DPO** (arXiv:2506.18245): Balanced detection+explanation loss
- **SmartLLM** (arXiv:2502.13167): Multi-role pipeline (Detector→Reasoner→Verificator)
- **SmartVD** (arXiv:2409.10574): Composite function F(C)=(binary, type, severity)