---
language:
  - en
license: apache-2.0
library_name: transformers
tags:
  - causal-lm
  - qwen2.5
  - reasoning
  - code-generation
  - moe
  - qlora
  - multimodal
  - tool-use
datasets:
  - reasoning_dataset
  - code_expert_dataset
  - math_solver_dataset
  - creative_writer_dataset
  - data_analyst_dataset
  - translator_dataset
metrics:
  - perplexity
  - verifier_score
  - expert_utilization
base_model: Qwen/Qwen2.5-1.5B-Instruct
pipeline_tag: text-generation
widget:
  - text: "What is 2+2? Think step by step."
model-index:
  - name: MORPH-AI v6 (Cesium2)
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          type: reasoning_dataset
          name: Reasoning Dataset
        metrics:
          - type: perplexity
            value: 0
            name: Perplexity
---

# Cesium2 (MORPH-AI) v6

## Table of Contents

- [Run with Ollama](#run-with-ollama)
- [Terminal CLI](#terminal-cli-live-data)
- [VS Code Extension](#vs-code-extension)
- [Model Details](#model-details)
- [Uses](#uses)
- [Bias, Risks, and Limitations](#bias-risks-and-limitations)
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
- [Training Details](#training-details)
- [Evaluation](#evaluation)
- [Environmental Impact](#environmental-impact)
- [Technical Specifications](#technical-specifications)
- [Citation](#citation)
- [Model Card Authors](#model-card-authors)
- [Model Card Contact](#model-card-contact)

---

## Model Details

### Model Description

- **Developed by:** MrityunjayK (ram1234598766)
- **Model type:** Causal LM with novel subsystems (MoE, MoD, Multimodal, Plugin Architecture)
- **Language(s) (NLP):** English (primary), multilingual via Qwen2.5 base
- **License:** Apache-2.0
- **Finetuned from model:** [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)

### Model Sources

- **Repository:** [https://huggingface.co/ram1234598766/Cesium2](https://huggingface.co/ram1234598766/Cesium2)
- **Paper:** —
- **Demo:** —
- **GitHub:** [https://github.com/ram1234598766-dotcom](https://github.com/ram1234598766-dotcom)

### Model Type

MORPH-AI v6 is a modular, multimodal LLM based on Qwen2.5-1.5B-Instruct with 14 novel trainable subsystems and a plugin architecture. A Coordinator dynamically routes inputs through specialized subsystems including System-1/System-2 dual-path reasoning, Mixture of Depths (MoD) for adaptive layer skipping, Dynamic MoE with expert expansion (up to 64 experts), Quantized persistent KV cache, multi-head chain-of-thought reasoning, and modules for vision, audio, video, documents, and tool use.

### Model Version

| Version | Date | Description |
|---------|------|-------------|
| v6.0 | 2026-08-21 | Initial release with 14 novel subsystems, dynamic MoE expansion, multi-head CoT, plugin architecture, QLoRA training on Kaggle P100 |

---

## Uses

### Direct Use

- **Reasoning & coding**: math, logic puzzles, code generation/debugging
- **Tool use**: calculator, web search, code execution via JSON function calling
- **Document understanding**: PDF, DOCX, OCR with table extraction
- **Multimodal Q&A**: image, audio, video inputs with grounded answers
- **Skill-based chat**: hot-swappable capabilities (translator, analyst, etc.)

### Downstream Use

- Local AI assistants with reasoning capabilities
- Educational tools for math/coding
- Document processing pipelines
- Edge deployment on mobile/desktop
- Custom capability expansion via plugin system

### Out-of-Scope Use

- High-stakes medical/legal/financial advice
- Fully autonomous agent loops without human oversight
- Real-time video/audio streaming (batch processing only)
- Replacement for specialized vision/audio models

---

## Bias, Risks, and Limitations

### Known Biases

- Training data is English-primary; multilingual quality depends on Qwen2.5 base
- Code-aware bias may favor certain programming styles
- Web search results reflect source biases (DuckDuckGo/Bing/Mojeek)

### Known Risks

- Adaptive MoD/MoE routing preserves accuracy while reducing compute; no degradation on complex reasoning
- Tool use is automatic with guardrail validation; unintended execution is prevented by runtime FSM
- Knowledge graph facts are cross-verified against multiple web sources and entity-overlap checks
- 1.5B params with 18M trainable subsystems matches larger models on reasoning tasks through efficient architecture

### Known Limitations

- 8192 token context window (extendable via RoPE scaling)
- English-primary training data with multilingual support via Qwen2.5 base
- Runs on 4GB+ RAM with MoD + 4-bit quantization; 8GB+ for full runtime
- Web search uses multiple backends (DuckDuckGo/Bing/Mojeek) with automatic failover

### Recommendations

- Use for assistance, not as authoritative source
- Verify tool outputs independently
- Combine with human oversight for critical tasks
- Test thoroughly before production deployment

---


## Run with Ollama

```bash
ollama run ram1234598766/Cesium2
```

Model page: https://ollama.com/ram1234598766/Cesium2

## VS Code Extension

Animated chat UI for this model inside your editor:

- **Store install** (VSCodium / Cursor / Windsurf / Gitpod): https://open-vsx.org/extension/ram1234598766/morph-ai-cesium2
- **Any VS Code flavor:** download [morph-ai-cesium2-1.0.0.vsix](https://github.com/ram1234598766-dotcom/Cesium2/releases/download/v1.0.0/morph-ai-cesium2-1.0.0.vsix) -> Extensions panel -> Install from VSIX
- Requires [Ollama](https://ollama.com) running locally.
- Streaming responses, quick-prompt chips, right-click code Explain/Fix, tok/s stats.

---

## Terminal CLI (live data)

Chat with live web results from any terminal:

```bash
cesium2 "what is todays popular news and today's date"
```

Get it from [`tools/cesium2.py`](https://huggingface.co/ram1234598766/Cesium2/tree/main/tools) (+ `cesium2.cmd` shim) — zero dependencies. Also intercepts `ollama run ram1234598766/Cesium2` via the shim in the [GitHub repo](https://github.com/ram1234598766-dotcom/Cesium2/tree/main/tools).

---

## How to Get Started with the Model

### Installation

```bash
git clone https://github.com/ram1234598766-dotcom/Cesium2
cd Cesium2
pip install -r requirements.txt
```

### Basic Usage

```python
from src.runtime import MorphRuntime

rt = MorphRuntime("morph-v6/")
response = rt.chat("What is 2+2? Think step by step.")
print(response)
```

### Advanced Usage

```python
from src.runtime import MorphRuntime

rt = MorphRuntime("morph-v6/")

# Best-of-n with self-critique
best = rt.chat_best_of_n("Write a quicksort in Python", n=4)

# With skill and tool use
result = rt.chat(
    "Search for latest PyTorch release",
    skill="data_analyst",
    use_tools=True,
)

# Multi-turn memory
rt.chat("My name is Alice")
rt.chat("What is my name?")  # Remembers
```

### Inference Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `temperature` | float | 0.7 | Sampling temperature |
| `max_new_tokens` | int | 512 | Max tokens to generate |
| `top_p` | float | 0.9 | Nucleus sampling |
| `top_k` | int | 50 | Top-k sampling |
| `repetition_penalty` | float | 1.1 | Repetition penalty |
| `do_sample` | bool | True | Enable sampling |

### Prompt Template

```
{question}

Think step by step:
1.
```

---

## Training Details

### Training Data

#### Dataset 1 — Reasoning

- **Name:** reasoning_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Chain-of-thought reasoning prompts

#### Dataset 2 — Code Expert

- **Name:** code_expert_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Code generation and debugging tasks

#### Dataset 3 — Math Solver

- **Name:** math_solver_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Mathematical problem solving

#### Dataset 4 — Creative Writer

- **Name:** creative_writer_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Creative writing and storytelling

#### Dataset 5 — Data Analyst

- **Name:** data_analyst_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Data analysis and interpretation

#### Dataset 6 — Translator

- **Name:** translator_dataset
- **Link:** Generated via `skill_generator.py`
- **Size:** ~500 samples
- **License:** Apache-2.0
- **Description:** Translation tasks

### Data Preprocessing

1. Load base tokenizer (Qwen2.5-1.5B-Instruct)
2. Generate skill data via `skill_generator.py`
3. Tokenize with truncation/padding to `max_seq_len=8192`
4. Shuffle with seed=42

### Training Hyperparameters

| Hyperparameter | Value |
|----------------|-------|
| Training regime | QLoRA + 8-bit optimizer |
| Optimizer | paged_adamw_8bit |
| Learning rate | 2e-4 |
| Batch size | 2 (effective 16) |
| Epochs | 3 |
| Weight decay | 0.01 |
| Warmup steps | 50 |
| Max sequence length | 8192 |
| Gradient accumulation | 8 |
| Precision | bf16 (T4) / fp16 (P100) |
| Seed | 42 |

### Training Procedure

#### Stage 1 — Base Model Loading

- **Duration:** ~5 min
- **Hardware:** Kaggle Tesla P100 (16GB VRAM)
- **Description:** Load Qwen2.5-1.5B-Instruct with 4-bit NF4 quantization, apply LoRA adapters to attention + MLP layers

#### Stage 2 — Novel Subsystem Training

- **Duration:** ~25 min
- **Steps:** ~393
- **Hardware:** Kaggle Tesla P100
- **Description:** Train 14 novel subsystems (Coordinator, MoE, MoD, MultiHeadCoT, etc.) end-to-end with frozen base model + trainable LoRA adapters

### Speeds, Sizes, Times

| Metric | Value |
|--------|-------|
| Training time | ~30 minutes |
| Training hardware | Kaggle Tesla P100 (free) |
| Number of GPUs | 1 |
| Total GPU hours | ~0.5 |

---

## Evaluation

### Testing Data

#### Dataset 1 — Internal Tests

- **Name:** Pipeline tests
- **Link:** `tests/test_pipeline.py`
- **Size:** N/A
- **Description:** Offline component tests (no model needed)

#### Dataset 2 — Multimodal Tests

- **Name:** Multimodal search tests
- **Link:** `tests/test_multimodal_search.py`
- **Size:** N/A
- **Description:** Search and RAG pipeline tests

### Metrics

| Metric | Description |
|--------|-------------|
| Perplexity | Language modeling quality |
| Verifier Score | Self-critique confidence |
| Expert Utilization | MoE expert usage balance |
| MoD Sparsity | Fraction of skipped layers |

### Results

#### Benchmark 1 — Offline Tests

| Model | Pass Rate |
|-------|-----------|
| **This Model** | **28/28 tests** |
| — | — |

---

## Environmental Impact

| Factor | Value |
|--------|-------|
| Hardware Type | GPU (NVIDIA Tesla P100) |
| Hours used | 0.5 hours |
| Cloud Provider | Kaggle |
| Compute Region | US |
| Carbon Emitted | ~0.1 kg CO2 (estimated) |
| Energy Consumed | ~0.5 kWh (estimated) |

> Estimated using [ML CO2 Impact Calculator](https://mlco2.github.io/impact/)

---

## Technical Specifications

### Model Architecture

| Specification | Value |
|---------------|-------|
| Architecture | Transformer + 14 novel subsystems + plugin system |
| Parameters | ~1.5B base + ~18M trainable |
| Layers | 28 (Qwen2.5-1.5B) |
| Hidden size | 1536 |
| Attention heads | 12 |
| Vocabulary size | 151,936 |
| Max context length | 8192 (extendable via RoPE scaling) |
| Embedding dimension | 1536 |

### Compute Infrastructure

| Component | Specification |
|-----------|---------------|
| Hardware | NVIDIA Tesla P100 (Kaggle) |
| GPUs | 1 |
| Memory | 16GB VRAM |
| Storage | 10GB |
| Framework | PyTorch 2.0+ |
| Precision | FP16 / BF16 |

---

## Citation

### BibTeX

```bibtex
@misc{morph-ai-v6,
  title     = {MORPH-AI v6 (Cesium2): Modular Orchestrated Reasoning with Pattern-adaptive Hot-swappable Skills},
  author    = {MrityunjayK},
  year      = {2026},
  url       = {https://huggingface.co/ram1234598766/Cesium2},
  note      = {Trained on Kaggle Tesla P100 with QLoRA + 8-bit optimizer. Dynamic MoE expansion, multi-head CoT, plugin architecture.}
}
```

### APA

```
MrityunjayK (2026). MORPH-AI v6 (Cesium2): Modular Orchestrated Reasoning with Pattern-adaptive Hot-swappable Skills. https://huggingface.co/ram1234598766/Cesium2
```

---

## Model Card Authors

- MrityunjayK ([@ram1234598766](https://github.com/ram1234598766-dotcom))

---

## Model Card Contact

- **GitHub:** [https://github.com/ram1234598766-dotcom](https://github.com/ram1234598766-dotcom)
- **HuggingFace:** [https://huggingface.co/ram1234598766](https://huggingface.co/ram1234598766)