AIOS — CPU-Native LLM Inference Architecture

A memory residency controller and Model Contract for deploying large language models efficiently on CPU hardware.

What We're Building

AIOS addresses the memory bandwidth bottleneck in CPU inference through weight aliasing, sparsity maps, KV cache tiering, and activation chunking — targeting 7B+ models on hardware organizations already own.

Current State

Framework and specification published. Runtime implementation is the primary contribution opportunity.

Links

• Paper: https://ssrn.com/abstract=6467298
• GitHub: https://github.com/acasavaraju/AIOS
• Primary validation target: GitHub Issue #2

How to Contribute

Clone the repo, run validation/compliance.py on any GGUF model, post results to the relevant GitHub issue.