SAINTHALF commited on
Commit
9b47998
·
verified ·
1 Parent(s): 707e4a2

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - minimind
5
+ - science
6
+ - chemistry
7
+ - biology
8
+ - kanna
9
+ - sceletium-tortuosum
10
+ ---
11
+
12
+ # MiniMind-Science
13
+
14
+ This repository contains **MiniMind** models (Small and MoE versions) trained on a curated mix of scientific datasets.
15
+
16
+ ## Models
17
+ * **`full_sft_science_512.pth`**: MiniMind-Small (26M params, dim=512). **Recommended**.
18
+ * Pretrained on: Biology, Botany, and Kanna (Sceletium tortuosum) texts.
19
+ * Fine-tuned on: Chemistry QA and PubMed Summarization.
20
+ * **`full_sft_science_moe_640_moe.pth`**: MiniMind-MoE (145M params, dim=640, 8 layers). Mixture-of-Experts version.
21
+
22
+ ## Training Data
23
+ * **Sceletium Tortuosum (Kanna)**: Custom dataset (`SAINTHALF/kanna_chunks_v2`).
24
+ * **Biology/Botany**: Text corpus from `rag-datasets/rag-mini-bioasq`.
25
+ * **Chemistry**: Conversational QA from `camel-ai/chemistry`.
26
+ * **Medical**: Summarization data from `ccdv/pubmed-summarization`.
27
+
28
+ ## Usage
29
+ These models are native PyTorch weights compatible with the [MiniMind](https://github.com/jingyaogong/minimind) architecture.
30
+
31
+ ```python
32
+ # Example loading (requires MiniMind code)
33
+ model.load_state_dict(torch.load('full_sft_science_512.pth'))
34
+ ```