Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- minimind
|
| 5 |
+
- science
|
| 6 |
+
- chemistry
|
| 7 |
+
- biology
|
| 8 |
+
- kanna
|
| 9 |
+
- sceletium-tortuosum
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# MiniMind-Science
|
| 13 |
+
|
| 14 |
+
This repository contains **MiniMind** models (Small and MoE versions) trained on a curated mix of scientific datasets.
|
| 15 |
+
|
| 16 |
+
## Models
|
| 17 |
+
* **`full_sft_science_512.pth`**: MiniMind-Small (26M params, dim=512). **Recommended**.
|
| 18 |
+
* Pretrained on: Biology, Botany, and Kanna (Sceletium tortuosum) texts.
|
| 19 |
+
* Fine-tuned on: Chemistry QA and PubMed Summarization.
|
| 20 |
+
* **`full_sft_science_moe_640_moe.pth`**: MiniMind-MoE (145M params, dim=640, 8 layers). Mixture-of-Experts version.
|
| 21 |
+
|
| 22 |
+
## Training Data
|
| 23 |
+
* **Sceletium Tortuosum (Kanna)**: Custom dataset (`SAINTHALF/kanna_chunks_v2`).
|
| 24 |
+
* **Biology/Botany**: Text corpus from `rag-datasets/rag-mini-bioasq`.
|
| 25 |
+
* **Chemistry**: Conversational QA from `camel-ai/chemistry`.
|
| 26 |
+
* **Medical**: Summarization data from `ccdv/pubmed-summarization`.
|
| 27 |
+
|
| 28 |
+
## Usage
|
| 29 |
+
These models are native PyTorch weights compatible with the [MiniMind](https://github.com/jingyaogong/minimind) architecture.
|
| 30 |
+
|
| 31 |
+
```python
|
| 32 |
+
# Example loading (requires MiniMind code)
|
| 33 |
+
model.load_state_dict(torch.load('full_sft_science_512.pth'))
|
| 34 |
+
```
|