| | --- |
| | language: |
| | - en |
| | license: apache-2.0 |
| | tags: |
| | - text-generation-inference |
| | - transformers |
| | - unsloth |
| | - qwen3_moe |
| | - scientific-reasoning |
| | - common-sense-reasoning |
| | - instruction-tuning |
| | - open-source |
| | library_name: transformers |
| | base_model: Qwen/Qwen3-30B-A3B-Thinking-2507 |
| | model_name: Daemontatox/HydraMind |
| | trained_with: |
| | - Unsloth |
| | - Hugging Face TRL |
| | --- |
| |  |
| | # Daemontatox/HydraMind |
| |
|
| | **HydraMind** is a fine-tuned Mixture-of-Experts model based on [Qwen/Qwen3-30B-A3B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Thinking-2507), built to excel at **scientific reasoning** and **common sense inference** tasks. |
| |
|
| | This model is especially well-suited for applications that require multi-step deduction, causal analysis, hypothesis generation, or intelligent response in ambiguous or knowledge-driven scenarios. |
| |
|
| | ## 🧠 Intended Use |
| |
|
| | HydraMind is intended for: |
| |
|
| | - Scientific Q&A |
| | - Hypothesis validation and falsifiability testing |
| | - Deductive and abductive reasoning |
| | - Multi-hop common sense logic tasks |
| | - Research assistance for STEM-related queries |
| | - Intelligent tutoring systems |
| |
|
| | ## 🚀 Model Highlights |
| |
|
| | - **Architecture**: Based on Qwen3 30B A3B Mixture-of-Experts, using only a subset of active experts per forward pass (efficient inference). |
| | - **Training**: Fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) for 2x faster training and optimized with Hugging Face's [TRL](https://github.com/huggingface/trl). |
| | - **Domains**: Trained on curated corpora focused on physics, biology, mathematics, cognitive science, and OpenBookQA-style commonsense reasoning. |
| | - **Instruction-tuned**: Accepts natural language instructions and responds with structured and coherent reasoning. |
| |
|
| | ## 📊 Evaluation (Qualitative) |
| |
|
| | HydraMind has demonstrated strong zero- and few-shot capabilities on tasks such as: |
| |
|
| | - **ARC (AI2 Reasoning Challenge)** |
| | - **OpenBookQA** |
| | - **CommonsenseQA** |
| | - **SciQ** |
| | - **StrategyQA** |
| |
|
| | ### Example Prompt: |
| | **Q:** "Why does salt melt ice on the road in winter?" |
| | **A:** "Salt lowers the freezing point of water. When added to ice, it causes the ice to melt even though the temperature is below 0°C. This process is known as freezing point depression." |
| |
|
| | ## ⚙️ Technical Specifications |
| |
|
| | | Property | Value | |
| | |------------------------|-------------------------------------------| |
| | | Base Model | Qwen/Qwen3-30B-A3B-Thinking-2507 | |
| | | Fine-tuned Model | Daemontatox/HydraMind | |
| | | Model Type | MoE (Mixture-of-Experts, 2/8 active) | |
| | | Language | English | |
| | | Parameters | ~30 Billion (active ~7.5B per step) | |
| | | Training Libraries | Unsloth, TRL, Hugging Face Transformers | |
| | | Format | HF Transformers + text-generation-inference compatible | |
| |
|
| | ## 🛠️ Training Details |
| |
|
| | - **Batch Size**: Adaptive via Unsloth memory optimization |
| | - **Precision**: bfloat16 / fp16 |
| | - **Loss Function**: Supervised fine-tuning (SFT) |
| | - **Epochs**: Varies (early stopping used) |
| | - **Optimizer**: AdamW with warmup and cosine decay |
| |
|
| | ## 📁 Files and Artifacts |
| |
|
| | - `pytorch_model.bin`: Main weights |
| | - `config.json`: Model configuration |
| | - `generation_config.json`: Decoding settings |
| | - `tokenizer.model`: Tokenizer (aligned with Qwen3) |
| |
|
| | ## 💡 Limitations |
| |
|
| | - May hallucinate facts if domain context is missing. |
| | - Not suitable for real-time critical applications (e.g., medical diagnosis, legal advice). |
| | - Limited multilingual support (English primarily). |
| |
|
| | ## ✅ License |
| |
|
| | This model is released under the **Apache 2.0 License**—free for research and commercial use, subject to attribution. |
| |
|
| | ## ✍️ Author |
| |
|
| | - **Model Developer**: [Daemontatox](https://huggingface.co/Daemontatox) |
| | - **Base Model Author**: [Qwen Team](https://huggingface.co/Qwen) |
| | - **Training Tools**: [Unsloth](https://github.com/unslothai/unsloth), [Hugging Face TRL](https://github.com/huggingface/trl) |
| |
|
| | --- |
| |
|
| | [](https://github.com/unslothai/unsloth) |
| |
|
| | --- |
| |
|
