README.md

# Lite LLM

Lite LLM is a deterministic, tiered-parameter, hierarchical sparse expert (HSER) language model runtime designed to scale from **1B → 1T parameters** and beyond (up to quadrillion-scale parameter universes) while keeping **active compute bounded** per token.

The Github organization hosts the **specification corpus**, **reference implementations**, and **operational tooling** for building and deploying Lite LLM as an enterprise / reference-grade system.

Model optimization for the LiteCore Coherent Silicon Photonic Complex Multiply-Accumulate (CSP-cMAC) Unit Cell Hardware focuses on maximizing inference efficiency under tight memory and power constraints by combining compression, quantization, and memory aware execution.  LiteCore is a fundamental photonic compute primitive purpose-built for large language model (LLM) inference at quadrillion-parameter scales.  LiteCore leverages silicon-on-insulator (SOI) photonics to perform complex-valued multiply-accumulate operations at <1 fJ energy and 1–10 ps latency—representing 500–2,000× energy and 1,000–10,000× latency improvements over state-of-the-art electronic GPUs.

---

## What makes Lite LLM different

### Deterministic by design
Lite LLM treats determinism as a first-class requirement:
- Stable top‑k routing with seeded tie‑breaking
- Deterministic collectives and reproducible distributed execution
- Deterministic audit logs and replayable training runs

### Tiered Parameter Architecture (TPA)
Parameters are partitioned across storage tiers:
- **Hot** (HBM / GPU)
- **Warm** (DRAM)
- **Cold** (NVMe)
- **Archive** (Object Store)

Only the TierSet for a request is eligible for routing; everything else has **zero activation probability**.

### Hierarchical Sparse Expert Routing (HSER)
Routing is hierarchical:
**Tier → Group → Expert**
with bounded activation:
`k_tier × k_group × k_expert` experts per token per layer.

This enables extreme parameter scaling while keeping per-token compute predictable.

### Enterprise runtime focus
Lite LLM is not only a model architecture—it is a runtime system:
- Distributed execution protocols
- Storage hierarchy and prefetching
- Secure loading and integrity verification
- Multi-tenant isolation, quotas, and compliance readiness

---

## Repositories 

### Specifications (authoritative)
- `lite-llm-specs` — Enterprise Runtime Engineering Specification Corpus (SPEC‑001…SPEC‑060)
- `lite-llm-schemas` — JSON/YAML schemas for manifests, telemetry, policies
- `lite-llm-rfcs` — Design proposals and evolution process (RFCs)

### Reference implementations
- `lite-llm-runtime` — Rust runtime (routing, caches, dispatch, TierSet engine)
- `lite-llm-train` — Training orchestration, checkpointing, determinism harness
- `lite-llm-kernels` — Device kernels + safe wrappers (CUDA/HIP/Metal/CPU)
- `lite-llm-comm` — Transport abstraction (RDMA / NCCL / QUIC), collectives
- `lite-llm-storage` — Shards, manifests, tier placement, streaming + prefetch

### Tooling
- `lite-llm-cli` — Operator CLI (inspect checkpoints, tier policies, telemetry)
- `lite-llm-observability` — Metrics exporters, dashboards, tracing
- `lite-llm-deploy` — Helm charts, Terraform modules, bare‑metal playbooks

> The organization may not yet contain all repositories listed above; this is the intended long-term structure.

---

## Getting started

### 1) Read the specs
Start with:
- **SPEC‑001** Runtime Architecture Overview
- **SPEC‑003** Deterministic Routing Engine
- **SPEC‑004** Tiered Parameter Architecture (TPA)
- **SPEC‑005** Hierarchical Sparse Expert Routing (HSER)
- **SPEC‑006** Active Compute Bounding Model
- **SPEC‑021…030** Storage hierarchy (hot/warm/cold/archive)
- **SPEC‑041…050** Inference runtime (TierSet selection, dispatch, KV cache)

### 2) Implement the contracts
The specs are written to be directly implementable:
- Deterministic routing + stable sorting
- Tier placement policies and shard formats
- All‑to‑all dispatch and imbalance handling
- Audit logging and integrity verification

### 3) Validate determinism
Before performance optimization:
- Ensure cross-node routing reproducibility
- Validate deterministic collectives
- Use the replay engine during training

---

## Contribution

We welcome contributions in:
- Spec clarifications and testable invariants
- Rust runtime modules (memory model, routing, dispatch, caching)
- Deterministic training harness and replay tooling
- Storage tier orchestration and prefetch algorithms
- Security hardening and audit improvements

Please read:
- `CONTRIBUTING.md` for workflow and standards
- `CODE_OF_CONDUCT.md` for community expectations
- `SECURITY.md` for vulnerability reporting

---

## Security

Lite LLM emphasizes:
- Memory-safe runtime design in Rust
- Secure checkpoint loading and integrity verification
- Encryption at rest for tier storage
- Key management and auditability
- Sandboxing and capability isolation for extensions

See `SECURITY.md` to report vulnerabilities responsibly.

---

## Governance

The specification corpus is the **normative authority**.  
Changes to the corpus should go through the RFC process:
1. Open an RFC in `lite-llm-rfcs`
2. Discuss and iterate
3. Land a spec patch with tests, invariants, and migration notes

---

## License

Lite-LLM is distributed under the Dust Open Source License

license: other
license_name: dosl-iie-1.0
license_link: https://github.com/lite-llm/lite-llm/raw/refs/heads/main/LICENSE

---

## Contact

- Security: see `SECURITY.md`
- General: open an issue in the relevant repository

---