acasavaraju commited on
Commit
0503d3f
·
verified ·
1 Parent(s): ccc6bd7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -3
README.md CHANGED
@@ -1,3 +1,50 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - llm-inference
5
+ - cpu-inference
6
+ - memory-bandwidth
7
+ - transformer
8
+ - quantization
9
+ - research
10
+ ---
11
+
12
+ # AIOS: A CPU-Native Inference Architecture for Large Language Models
13
+
14
+ **This is not a model.** This is the framework paper and specification
15
+ for AIOS — a memory residency controller for CPU-native LLM inference.
16
+
17
+ ## Paper
18
+
19
+ **Title:** AIOS: A CPU-Native Inference Architecture for Large Language Models
20
+ **Author:** Anand Casavaraju
21
+ **Published:** March 2026
22
+ **SSRN:** https://ssrn.com/abstract=6467298
23
+ **GitHub:** https://github.com/acasavaraju/AIOS
24
+
25
+ ## What AIOS Is
26
+
27
+ AIOS is a memory residency controller that sits between inference
28
+ engines (llama.cpp, Ollama, vLLM) and hardware, managing how weight
29
+ data moves from DRAM to CPU. It addresses four resource dimensions:
30
+
31
+ - **Weight reads** — aliasing + sparsity maps
32
+ - **KV cache reads** — MQA/GQA + tiered residency
33
+ - **Activation spill** — chunked prefill
34
+ - **Attention compute** — sparsity map
35
+
36
+ ## Current State
37
+
38
+ Framework and specification published. Runtime not yet implemented.
39
+ All performance projections are analytical. Empirical validation
40
+ tracked at github.com/acasavaraju/AIOS/issues.
41
+
42
+ ## Citation
43
+ ```bibtex
44
+ @misc{casavaraju2026aios,
45
+ title = {AIOS: A CPU-Native Inference Architecture for Large Language Models},
46
+ author = {Casavaraju, Anand},
47
+ year = {2026},
48
+ url = {https://ssrn.com/abstract=6467298}
49
+ }
50
+ ```