metadata
base_model:
- LiquidAI/LFM2-1.2B
tags:
- text-generation-inference
- transformers
- unsloth
- lfm2
license: apache-2.0
language:
- en
datasets:
- MegaScience/MegaScience
🧪 MegaSciMoE‑1.2B
A lightweight open MoE language model focused on science reasoning and instruction-following tasks. Fine-tuned using Unsloth on top of LiquidAI’s LFM2‑1.2B architecture.
🔬 Model Overview
- Model Type: Sparse Mixture-of-Experts (MoE)
- Base: LiquidAI LFM2‑1.2B
- Fine-tuned with: Unsloth LoRA
- Focus Areas: Physics, Chemistry, Biology, Scientific QA
- Parameter Efficiency: Only 2 experts active (~250M active params)
- Design Goal: Run efficiently on consumer GPUs and edge devices
🧠 Capabilities
- Respond to structured science-related instructions
- Generate concise, factual explanations in STEM topics
- Follow multiturn question-answer patterns
- Summarize technical descriptions in plain language
🚀 Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model = AutoModelForCausalLM.from_pretrained("yasserrmd/MegaSciMoE-1.2B", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained("yasserrmd/MegaSciMoE-1.2B")
prompt = "What is the difference between an ionic and covalent bond?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📚 Training Data
MegaSciMoE-1.2B was fine-tuned on a blend of:
- Multiturn scientific dialogues
- Instruction-based question answering
- Simulated domain-specific prompts in physics, chemistry, and biology
- Scientific corpus extracted from open datasets
📎 Sample Prompts
Q: What is a neutron made of?
A: A neutron is composed of three quarks (two down quarks and one up quark)...
---
Q: Define osmosis in simple terms.
A: Osmosis is the movement of water across a membrane...
---
Q: What is the role of mitochondria?
A: Mitochondria are the powerhouses of the cell...
🧪 Benchmark Intent
Planned evaluations on:
- ARC-Challenge
- PubMedQA
- ScienceQA (Note: No math-based datasets included in this build)
⚙️ Technical Details
- LoRA fine-tuning with Unsloth on 1.2B MoE model
- Memory-efficient inference (fits on T4, L4, RTX 3060, even CPU)
- GGUF and FP16 variants to be released
📄 License
Apache 2.0 — free to use and modify with attribution. Ensure upstream license compatibility before commercial use.
🤝 Acknowledgements
- Thanks to Unsloth, LiquidAI, and Hugging Face for enabling this build
- Visuals rendered with scientific prompts and surreal-futuristic style
