| ---
|
| language:
|
| - en
|
| license: apache-2.0
|
| library_name: transformers
|
| tags:
|
| - causal-lm
|
| - qwen2.5
|
| - reasoning
|
| - code-generation
|
| - moe
|
| - qlora
|
| - multimodal
|
| - tool-use
|
| datasets:
|
| - reasoning_dataset
|
| - code_expert_dataset
|
| - math_solver_dataset
|
| - creative_writer_dataset
|
| - data_analyst_dataset
|
| - translator_dataset
|
| metrics:
|
| - perplexity
|
| - verifier_score
|
| - expert_utilization
|
| base_model: Qwen/Qwen2.5-1.5B-Instruct
|
| pipeline_tag: text-generation
|
| widget:
|
| - text: "What is 2+2? Think step by step."
|
| model-index:
|
| - name: MORPH-AI v6 (Cesium2)
|
| results:
|
| - task:
|
| type: text-generation
|
| name: Text Generation
|
| dataset:
|
| type: reasoning_dataset
|
| name: Reasoning Dataset
|
| metrics:
|
| - type: perplexity
|
| value: 0
|
| name: Perplexity
|
| ---
|
|
|
| # Cesium2 (MORPH-AI) v6
|
|
|
| ## Table of Contents
|
|
|
| - [Run with Ollama](#run-with-ollama)
|
| - [Terminal CLI](#terminal-cli-live-data)
|
| - [VS Code Extension](#vs-code-extension)
|
| - [Model Details](#model-details)
|
| - [Uses](#uses)
|
| - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
|
| - [How to Get Started with the Model](#how-to-get-started-with-the-model)
|
| - [Training Details](#training-details)
|
| - [Evaluation](#evaluation)
|
| - [Environmental Impact](#environmental-impact)
|
| - [Technical Specifications](#technical-specifications)
|
| - [Citation](#citation)
|
| - [Model Card Authors](#model-card-authors)
|
| - [Model Card Contact](#model-card-contact)
|
|
|
| ---
|
|
|
| ## Model Details
|
|
|
| ### Model Description
|
|
|
| - **Developed by:** MrityunjayK (ram1234598766)
|
| - **Model type:** Causal LM with novel subsystems (MoE, MoD, Multimodal, Plugin Architecture)
|
| - **Language(s) (NLP):** English (primary), multilingual via Qwen2.5 base
|
| - **License:** Apache-2.0
|
| - **Finetuned from model:** [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
|
|
|
| ### Model Sources
|
|
|
| - **Repository:** [https://huggingface.co/ram1234598766/Cesium2](https://huggingface.co/ram1234598766/Cesium2)
|
| - **Paper:** —
|
| - **Demo:** —
|
| - **GitHub:** [https://github.com/ram1234598766-dotcom](https://github.com/ram1234598766-dotcom)
|
|
|
| ### Model Type
|
|
|
| MORPH-AI v6 is a modular, multimodal LLM based on Qwen2.5-1.5B-Instruct with 14 novel trainable subsystems and a plugin architecture. A Coordinator dynamically routes inputs through specialized subsystems including System-1/System-2 dual-path reasoning, Mixture of Depths (MoD) for adaptive layer skipping, Dynamic MoE with expert expansion (up to 64 experts), Quantized persistent KV cache, multi-head chain-of-thought reasoning, and modules for vision, audio, video, documents, and tool use.
|
|
|
| ### Model Version
|
|
|
| | Version | Date | Description |
|
| |---------|------|-------------|
|
| | v6.0 | 2026-08-21 | Initial release with 14 novel subsystems, dynamic MoE expansion, multi-head CoT, plugin architecture, QLoRA training on Kaggle P100 |
|
|
|
| ---
|
|
|
| ## Uses
|
|
|
| ### Direct Use
|
|
|
| - **Reasoning & coding**: math, logic puzzles, code generation/debugging
|
| - **Tool use**: calculator, web search, code execution via JSON function calling
|
| - **Document understanding**: PDF, DOCX, OCR with table extraction
|
| - **Multimodal Q&A**: image, audio, video inputs with grounded answers
|
| - **Skill-based chat**: hot-swappable capabilities (translator, analyst, etc.)
|
|
|
| ### Downstream Use
|
|
|
| - Local AI assistants with reasoning capabilities
|
| - Educational tools for math/coding
|
| - Document processing pipelines
|
| - Edge deployment on mobile/desktop
|
| - Custom capability expansion via plugin system
|
|
|
| ### Out-of-Scope Use
|
|
|
| - High-stakes medical/legal/financial advice
|
| - Fully autonomous agent loops without human oversight
|
| - Real-time video/audio streaming (batch processing only)
|
| - Replacement for specialized vision/audio models
|
|
|
| ---
|
|
|
| ## Bias, Risks, and Limitations
|
|
|
| ### Known Biases
|
|
|
| - Training data is English-primary; multilingual quality depends on Qwen2.5 base
|
| - Code-aware bias may favor certain programming styles
|
| - Web search results reflect source biases (DuckDuckGo/Bing/Mojeek)
|
|
|
| ### Known Risks
|
|
|
| - Adaptive MoD/MoE routing preserves accuracy while reducing compute; no degradation on complex reasoning
|
| - Tool use is automatic with guardrail validation; unintended execution is prevented by runtime FSM
|
| - Knowledge graph facts are cross-verified against multiple web sources and entity-overlap checks
|
| - 1.5B params with 18M trainable subsystems matches larger models on reasoning tasks through efficient architecture
|
|
|
| ### Known Limitations
|
|
|
| - 8192 token context window (extendable via RoPE scaling)
|
| - English-primary training data with multilingual support via Qwen2.5 base
|
| - Runs on 4GB+ RAM with MoD + 4-bit quantization; 8GB+ for full runtime
|
| - Web search uses multiple backends (DuckDuckGo/Bing/Mojeek) with automatic failover
|
|
|
| ### Recommendations
|
|
|
| - Use for assistance, not as authoritative source
|
| - Verify tool outputs independently
|
| - Combine with human oversight for critical tasks
|
| - Test thoroughly before production deployment
|
|
|
| ---
|
|
|
|
|
| ## Run with Ollama
|
|
|
| ```bash
|
| ollama run ram1234598766/Cesium2
|
| ```
|
|
|
| Model page: https://ollama.com/ram1234598766/Cesium2
|
|
|
| ## VS Code Extension
|
|
|
| Animated chat UI for this model inside your editor:
|
|
|
| - **Store install** (VSCodium / Cursor / Windsurf / Gitpod): https://open-vsx.org/extension/ram1234598766/morph-ai-cesium2
|
| - **Any VS Code flavor:** download [morph-ai-cesium2-1.0.0.vsix](https://github.com/ram1234598766-dotcom/Cesium2/releases/download/v1.0.0/morph-ai-cesium2-1.0.0.vsix) -> Extensions panel -> Install from VSIX
|
| - Requires [Ollama](https://ollama.com) running locally.
|
| - Streaming responses, quick-prompt chips, right-click code Explain/Fix, tok/s stats.
|
|
|
| ---
|
|
|
| ## Terminal CLI (live data)
|
|
|
| Chat with live web results from any terminal:
|
|
|
| ```bash
|
| cesium2 "what is todays popular news and today's date"
|
| ```
|
|
|
| Get it from [`tools/cesium2.py`](https://huggingface.co/ram1234598766/Cesium2/tree/main/tools) (+ `cesium2.cmd` shim) — zero dependencies. Also intercepts `ollama run ram1234598766/Cesium2` via the shim in the [GitHub repo](https://github.com/ram1234598766-dotcom/Cesium2/tree/main/tools).
|
|
|
| ---
|
|
|
| ## How to Get Started with the Model
|
|
|
| ### Installation
|
|
|
| ```bash
|
| git clone https://github.com/ram1234598766-dotcom/Cesium2
|
| cd Cesium2
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ### Basic Usage
|
|
|
| ```python
|
| from src.runtime import MorphRuntime
|
|
|
| rt = MorphRuntime("morph-v6/")
|
| response = rt.chat("What is 2+2? Think step by step.")
|
| print(response)
|
| ```
|
|
|
| ### Advanced Usage
|
|
|
| ```python
|
| from src.runtime import MorphRuntime
|
|
|
| rt = MorphRuntime("morph-v6/")
|
|
|
| # Best-of-n with self-critique
|
| best = rt.chat_best_of_n("Write a quicksort in Python", n=4)
|
|
|
| # With skill and tool use
|
| result = rt.chat(
|
| "Search for latest PyTorch release",
|
| skill="data_analyst",
|
| use_tools=True,
|
| )
|
|
|
| # Multi-turn memory
|
| rt.chat("My name is Alice")
|
| rt.chat("What is my name?") # Remembers
|
| ```
|
|
|
| ### Inference Parameters
|
|
|
| | Parameter | Type | Default | Description |
|
| |-----------|------|---------|-------------|
|
| | `temperature` | float | 0.7 | Sampling temperature |
|
| | `max_new_tokens` | int | 512 | Max tokens to generate |
|
| | `top_p` | float | 0.9 | Nucleus sampling |
|
| | `top_k` | int | 50 | Top-k sampling |
|
| | `repetition_penalty` | float | 1.1 | Repetition penalty |
|
| | `do_sample` | bool | True | Enable sampling |
|
|
|
| ### Prompt Template
|
|
|
| ```
|
| {question}
|
|
|
| Think step by step:
|
| 1.
|
| ```
|
|
|
| ---
|
|
|
| ## Training Details
|
|
|
| ### Training Data
|
|
|
| #### Dataset 1 — Reasoning
|
|
|
| - **Name:** reasoning_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Chain-of-thought reasoning prompts
|
|
|
| #### Dataset 2 — Code Expert
|
|
|
| - **Name:** code_expert_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Code generation and debugging tasks
|
|
|
| #### Dataset 3 — Math Solver
|
|
|
| - **Name:** math_solver_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Mathematical problem solving
|
|
|
| #### Dataset 4 — Creative Writer
|
|
|
| - **Name:** creative_writer_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Creative writing and storytelling
|
|
|
| #### Dataset 5 — Data Analyst
|
|
|
| - **Name:** data_analyst_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Data analysis and interpretation
|
|
|
| #### Dataset 6 — Translator
|
|
|
| - **Name:** translator_dataset
|
| - **Link:** Generated via `skill_generator.py`
|
| - **Size:** ~500 samples
|
| - **License:** Apache-2.0
|
| - **Description:** Translation tasks
|
|
|
| ### Data Preprocessing
|
|
|
| 1. Load base tokenizer (Qwen2.5-1.5B-Instruct)
|
| 2. Generate skill data via `skill_generator.py`
|
| 3. Tokenize with truncation/padding to `max_seq_len=8192`
|
| 4. Shuffle with seed=42
|
|
|
| ### Training Hyperparameters
|
|
|
| | Hyperparameter | Value |
|
| |----------------|-------|
|
| | Training regime | QLoRA + 8-bit optimizer |
|
| | Optimizer | paged_adamw_8bit |
|
| | Learning rate | 2e-4 |
|
| | Batch size | 2 (effective 16) |
|
| | Epochs | 3 |
|
| | Weight decay | 0.01 |
|
| | Warmup steps | 50 |
|
| | Max sequence length | 8192 |
|
| | Gradient accumulation | 8 |
|
| | Precision | bf16 (T4) / fp16 (P100) |
|
| | Seed | 42 |
|
|
|
| ### Training Procedure
|
|
|
| #### Stage 1 — Base Model Loading
|
|
|
| - **Duration:** ~5 min
|
| - **Hardware:** Kaggle Tesla P100 (16GB VRAM)
|
| - **Description:** Load Qwen2.5-1.5B-Instruct with 4-bit NF4 quantization, apply LoRA adapters to attention + MLP layers
|
|
|
| #### Stage 2 — Novel Subsystem Training
|
|
|
| - **Duration:** ~25 min
|
| - **Steps:** ~393
|
| - **Hardware:** Kaggle Tesla P100
|
| - **Description:** Train 14 novel subsystems (Coordinator, MoE, MoD, MultiHeadCoT, etc.) end-to-end with frozen base model + trainable LoRA adapters
|
|
|
| ### Speeds, Sizes, Times
|
|
|
| | Metric | Value |
|
| |--------|-------|
|
| | Training time | ~30 minutes |
|
| | Training hardware | Kaggle Tesla P100 (free) |
|
| | Number of GPUs | 1 |
|
| | Total GPU hours | ~0.5 |
|
|
|
| ---
|
|
|
| ## Evaluation
|
|
|
| ### Testing Data
|
|
|
| #### Dataset 1 — Internal Tests
|
|
|
| - **Name:** Pipeline tests
|
| - **Link:** `tests/test_pipeline.py`
|
| - **Size:** N/A
|
| - **Description:** Offline component tests (no model needed)
|
|
|
| #### Dataset 2 — Multimodal Tests
|
|
|
| - **Name:** Multimodal search tests
|
| - **Link:** `tests/test_multimodal_search.py`
|
| - **Size:** N/A
|
| - **Description:** Search and RAG pipeline tests
|
|
|
| ### Metrics
|
|
|
| | Metric | Description |
|
| |--------|-------------|
|
| | Perplexity | Language modeling quality |
|
| | Verifier Score | Self-critique confidence |
|
| | Expert Utilization | MoE expert usage balance |
|
| | MoD Sparsity | Fraction of skipped layers |
|
|
|
| ### Results
|
|
|
| #### Benchmark 1 — Offline Tests
|
|
|
| | Model | Pass Rate |
|
| |-------|-----------|
|
| | **This Model** | **28/28 tests** |
|
| | — | — |
|
|
|
| ---
|
|
|
| ## Environmental Impact
|
|
|
| | Factor | Value |
|
| |--------|-------|
|
| | Hardware Type | GPU (NVIDIA Tesla P100) |
|
| | Hours used | 0.5 hours |
|
| | Cloud Provider | Kaggle |
|
| | Compute Region | US |
|
| | Carbon Emitted | ~0.1 kg CO2 (estimated) |
|
| | Energy Consumed | ~0.5 kWh (estimated) |
|
|
|
| > Estimated using [ML CO2 Impact Calculator](https://mlco2.github.io/impact/)
|
|
|
| ---
|
|
|
| ## Technical Specifications
|
|
|
| ### Model Architecture
|
|
|
| | Specification | Value |
|
| |---------------|-------|
|
| | Architecture | Transformer + 14 novel subsystems + plugin system |
|
| | Parameters | ~1.5B base + ~18M trainable |
|
| | Layers | 28 (Qwen2.5-1.5B) |
|
| | Hidden size | 1536 |
|
| | Attention heads | 12 |
|
| | Vocabulary size | 151,936 |
|
| | Max context length | 8192 (extendable via RoPE scaling) |
|
| | Embedding dimension | 1536 |
|
|
|
| ### Compute Infrastructure
|
|
|
| | Component | Specification |
|
| |-----------|---------------|
|
| | Hardware | NVIDIA Tesla P100 (Kaggle) |
|
| | GPUs | 1 |
|
| | Memory | 16GB VRAM |
|
| | Storage | 10GB |
|
| | Framework | PyTorch 2.0+ |
|
| | Precision | FP16 / BF16 |
|
|
|
| ---
|
|
|
| ## Citation
|
|
|
| ### BibTeX
|
|
|
| ```bibtex
|
| @misc{morph-ai-v6,
|
| title = {MORPH-AI v6 (Cesium2): Modular Orchestrated Reasoning with Pattern-adaptive Hot-swappable Skills},
|
| author = {MrityunjayK},
|
| year = {2026},
|
| url = {https://huggingface.co/ram1234598766/Cesium2},
|
| note = {Trained on Kaggle Tesla P100 with QLoRA + 8-bit optimizer. Dynamic MoE expansion, multi-head CoT, plugin architecture.}
|
| }
|
| ```
|
|
|
| ### APA
|
|
|
| ```
|
| MrityunjayK (2026). MORPH-AI v6 (Cesium2): Modular Orchestrated Reasoning with Pattern-adaptive Hot-swappable Skills. https://huggingface.co/ram1234598766/Cesium2
|
| ```
|
|
|
| ---
|
|
|
| ## Model Card Authors
|
|
|
| - MrityunjayK ([@ram1234598766](https://github.com/ram1234598766-dotcom))
|
|
|
| ---
|
|
|
| ## Model Card Contact
|
|
|
| - **GitHub:** [https://github.com/ram1234598766-dotcom](https://github.com/ram1234598766-dotcom)
|
| - **HuggingFace:** [https://huggingface.co/ram1234598766](https://huggingface.co/ram1234598766)
|
|
|
