SOLARIS (PoC)

PAPER Selective On-demand Learning Architecture with Residual Intelligence and Sparse experts.

⚠️ Proof-of-Concept Disclaimer

This repository implements a research-oriented Proof of Concept (PoC) of the SOLARIS pipeline, including when executed on Modal infrastructure.

It is not a complete or production-optimized implementation of the full SOLARIS architecture as described in the paper. In particular, several components may be approximated, simplified, or lack low-level optimizations (e.g., custom kernels, fused operations, or hardware-specific tuning).

As such, all reported results should be interpreted as algorithmic validation signals, not as definitive performance benchmarks.

Do not treat current outputs as representative of final architecture-level performance, efficiency, or scaling behavior.

The purpose of this repository is to validate conceptual correctness, modular integration, and reproducibility, serving as a foundation for future large-scale and fully optimized implementations.

This repository is a runnable proof-of-concept (PoC) for a SOLARIS-style pipeline:

1. environment check
2. short LoRA training
3. evaluation on fixed benchmark slices
4. artifact packaging and manifesting

It is designed for practical iteration and reproducible artifacts, not for paper-scale final claims.

What This Repo Is

• A working engineering PoC you can run locally or on Modal.
• Focused on pipeline reliability, artifact auditability, and side-by-side model comparisons.
• Built around a small model workflow (TinyLlama/TinyLlama-1.1B-Chat-v1.0 by default).

What This Repo Is Not

• Not a full production 30B training stack.
• Not a benchmark-claim repo for state-of-the-art numbers.
• Not a substitute for the full research paper methodology.

Project Layout

• solaris_poc/modal_system.py: main end-to-end pipeline (env -> train -> evaluate -> bundle)
• solaris_poc/modal_fireproof.py: reproducible wrapper commands (prepare, manifest, pipeline)
• solaris_poc/modal_versus_report.py: generates detailed plots/tables for tinyllama_default vs solaris_system
• solaris_poc/attention/, moe/, compression/, continual/, world/: core PoC modules
• solaris_poc/tests/: tests
• artifacts/: generated run outputs and runbook files

Setup

Python virtualenv:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Optional plotting extras:

pip install matplotlib numpy

If you use uv, these commands also work via uv run ... as shown below.

Quick Start (Local)

Write starter assets and run the full local pipeline:

uv run python solaris_poc/modal_fireproof.py prepare
uv run python solaris_poc/modal_fireproof.py pipeline

Outputs are written to:

• artifacts/modal_runs/<run_id>/

Quick Start (Modal)

Run pipeline remotely on Modal:

uv run python -m modal run -m solaris_poc.modal_system

Artifacts are written inside the Modal artifacts volume under:

• /runs/<run_id>/

Download Artifacts From Modal

Download full run directory (recommended):

uv run python -m modal volume get solaris-modal-artifacts /runs/<run_id>/ ./artifacts/<run_id>/

Download only tar bundle:

uv run python -m modal volume get solaris-modal-artifacts /runs/<run_id>/artifacts_bundle.tar.gz ./artifacts/<run_id>/artifacts_bundle.tar.gz

If you only fetch manifest.json, you will only get metadata. Use the full run directory or the bundle to get CSV/JSONL/SVG artifacts too.

Typical Output Files

Inside artifacts/modal_runs/<run_id>/ (local) or downloaded run dir:

• environment/: hardware/runtime checks
• train/summary/: train logs, train/eval CSV, metrics JSON, SVG charts
• evaluate/: benchmark JSONL/CSV, summaries, comparison charts
• manifest.json: hash + size registry of recorded files
• artifacts_bundle.tar.gz: packaged run artifacts

Key evaluation outputs:

• evaluate/tinyllama_default_benchmark.csv
• evaluate/solaris_system_benchmark.csv
• evaluate/benchmark_slices.csv
• evaluate/comparison.csv

Run Detailed Versus Report

For a specific run:

python solaris_poc/modal_versus_report.py --run-dir artifacts/modal_runs/<run_id>

Or latest run auto-detection:

python solaris_poc/modal_versus_report.py

Generates:

• artifacts/modal_runs/<run_id>/evaluate/versus/
• figures (*.png) + tabular summaries (prompt_level_versus.csv, aggregate_versus.csv)

Current Benchmark Snapshot (PoC)

From artifacts/modal_runs/<run_id>/evaluate/versus/aggregate_versus.csv:

metric	value
slice_count	11
tinyllama_latency_mean_s	0.308945
solaris_latency_mean_s	1.642673
latency_mean_delta_s	1.333727
tinyllama_tokens_mean	21.818
solaris_tokens_mean	65.455
token_mean_delta	43.636
tinyllama_keywords_total	16
solaris_keywords_total	16
keyword_total_delta	0
solaris_faster_prompt_count	0
solaris_higher_keyword_prompt_count	0

Interpretation (current regime):

• SOLARIS is currently slower and more verbose on this benchmark.
• Keyword quality is flat versus baseline in this run.

Why quality did not increase in this PoC run:

• In low-capacity regimes (around 1B parameters), error is often dominated by approximation limits, not representational inefficiency.
• In that setting, compression/transform pipelines can be approximately information-preserving (ΔE ≈ 0) but still not add quality.
• With short sequence lengths and small workloads, constant overhead dominates wall-clock latency, masking asymptotic gains from KV compression/quantization.

What this implies for next experiments:

• This PoC result is useful, but it is not the target regime for SOLARIS benefits.
• The next step is evaluation in higher-context and/or higher-capacity setups, where representational redundancy is larger.
• Planned direction: test larger base models and longer-context runs to measure whether quality/efficiency tradeoffs improve in the intended regime.

Useful Environment Variables

Main knobs:

• SOLARIS_MODAL_MODEL_ID
• SOLARIS_MODAL_RUN_ID
• SOLARIS_MODAL_TRAINING_MAX_STEPS
• SOLARIS_MODAL_TRAINING_SEQ_LEN
• SOLARIS_MODAL_TRAINING_GPU
• SOLARIS_MODAL_EVAL_GPU
• SOLARIS_MODAL_LORA_RANK
• SOLARIS_MODAL_DATASET_ID
• SOLARIS_MODAL_DATASET_MAX_SAMPLES
• SOLARIS_MODAL_SECRETS

Example (Modal, A100-80GB):

SOLARIS_MODAL_TRAINING_GPU=A100-80GB \
SOLARIS_MODAL_EVAL_GPU=A100-80GB \
uv run python -m modal run -m solaris_poc.modal_system

Reliability Notes

• The pipeline writes a manifest plus a tar bundle for reproducibility.
• The manifest records file paths, sizes, and SHA-256 hashes.
• The bundle is the easiest single-file handoff for downstream analysis.

Troubleshooting

No CSVs after Modal run:

1. Confirm you downloaded the full run directory, not only manifest.json.
2. Check <run_id>/manifest.json for expected evaluate/*.csv entries.
3. If needed, pull artifacts_bundle.tar.gz and inspect contents locally.

Missing authenticated model access:

1. Set SOLARIS_MODAL_SECRETS with your Modal secret name.
2. Re-run pipeline and confirm environment section in output summary.

Tests

Run tests with:

uv run pytest -q

Status Summary

Current status is PoC-complete for:

• pipeline orchestration
• artifact generation and packaging
• baseline vs adapted evaluation comparisons
• module-level implementation across attention/routing/compression/continual/world helpers

Use this repo as an engineering scaffold for fast experiments, not as a final benchmark authority.