Upload 5 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+images/packedllm_architecture.png filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,197 @@
+# PackedLLM Source-Available Research and Controlled Commercial Use License
+
+PackedLicense v1.0
+Copyright (c) [2026] [Chance Brownfield]
+
+This license is intended to govern the use, modification, distribution, and commercial licensing of the PackedLLM system and its protected modules, including but not limited to GATOR, WebSearchModule, and CodeBox.
+
+This license governs the use, modification, distribution, and commercial licensing of the PackedLLM system, including the GATOR, WebSearchModule, and CodeBox modules.
+---
+
+## 1. Definitions
+
+For purposes of this License:
+
+**“Licensed Work”** means the software, source code, model weights, checkpoints, configurations, prompt templates, documentation, build artifacts, scripts, and related materials distributed by the Licensor under this License, including all modifications and derivative works of those materials.
+
+**“PackedLLM System”** means the orchestration framework, routing logic, checkpoint packing and loading format, expert dispatch system, runtime coordination logic, and related components distributed as part of or alongside the Licensed Work.
+
+**“Protected Modules”** means the following components and any modifications or derivative works thereof:
+
+* **GATOR** — including memory bank, semantic tree structures, retrieval logic, profile storage, command registry, and related memory orchestration code;
+* **WebSearchModule** — including multi-engine search, page fetching, content extraction, crawling, ranking, summarization, and related web-research code;
+* **CodeBox** — including sandbox execution, asset registry, virtual environment management, DAG pipeline execution, loader injection, and related code-execution infrastructure.
+
+**“Non-Commercial Use”** means use that is not primarily intended for or directed toward commercial advantage or monetary compensation, including personal, academic, research, hobby, and internal evaluation use.
+
+**“Commercial Use”** means any use, deployment, distribution, offering, hosting, embedding, sublicensing, or other exploitation of the Licensed Work or any derivative work that is directly or indirectly intended for or associated with commercial advantage, monetary compensation, paid access, enterprise use, SaaS, resale, bundling, or product incorporation.
+
+**“Written Commercial License”** means a separate written agreement signed by Licensor expressly authorizing Commercial Use of the Licensed Work or any portion of it.
+
+**“Derivative Work”** means any work based upon, adapted from, translated from, modified from, or otherwise incorporating the Licensed Work or any Protected Module.
+
+---
+
+## 2. Grant of Rights for Non-Commercial Use
+
+Subject to the terms and conditions of this License, Licensor grants you a worldwide, non-exclusive, non-transferable, royalty-free license to:
+
+1. use the Licensed Work for Non-Commercial Use;
+2. copy the Licensed Work for Non-Commercial Use;
+3. modify the Licensed Work for Non-Commercial Use;
+4. create Derivative Works for Non-Commercial Use; and
+5. distribute the Licensed Work or Derivative Works solely for Non-Commercial Use and only if all recipients are bound by this License.
+
+No rights are granted except as expressly stated in this License.
+
+---
+
+## 3. Commercial Use Requires Written Permission
+
+Commercial Use of the Licensed Work, the PackedLLM System, or any Protected Module is prohibited unless you first obtain a Written Commercial License from Licensor.
+
+Without limiting the foregoing, Commercial Use includes, without limitation:
+
+* offering the Licensed Work as a service, hosted endpoint, API, or subscription product;
+* incorporating the Licensed Work into a paid or revenue-generating product;
+* redistributing the Licensed Work to paying customers or enterprise users;
+* using the Licensed Work in internal business operations where the Licensed Work is a material part of a revenue-producing workflow, product, or service;
+* sublicensing the Licensed Work or any Protected Module for commercial purposes.
+
+Licensor may approve, deny, condition, or limit Commercial Use in Licensor’s sole discretion.
+
+Any Commercial Use without a Written Commercial License is a material breach of this License.
+
+---
+
+## 4. Protected Modules and No Extraction or Standalone Reuse
+
+The Protected Modules are expressly identified as core and protected components of the Licensed Work.
+
+You may not, without a Written Commercial License:
+
+1. extract, isolate, copy, port, transplant, or repackage any Protected Module as a standalone library, standalone service, plugin, product, or subsystem;
+2. incorporate any Protected Module into another system, agent framework, model runtime, or commercial product;
+3. distribute a derivative implementation of any Protected Module under a substantially similar interface, architecture, workflow, or operational behavior where such implementation is based on or derived from the Licensed Work;
+4. remove, conceal, or fragment the Protected Modules for the purpose of avoiding the terms of this License.
+
+For avoidance of doubt, independent reimplementation from scratch without copying code, checkpoints, configuration, documentation, or other protectable expression is not governed by this License; however, any direct use of the Licensed Work or its protectable expression remains subject to this License.
+
+---
+
+## 5. Attribution and Credit Requirements
+
+If you use, distribute, modify, or publish any part of the Licensed Work, you must retain and reproduce all copyright notices, license notices, and attribution notices contained in the Licensed Work.
+
+In addition, any public distribution, publication, presentation, product documentation, or user-facing deployment that references the Licensed Work must include prominent attribution substantially in the following form:
+
+> “Includes or is based on PackedLLM, including the GATOR memory system, WebSearchModule, CodeBox sandboxing system, and the PackedLLM checkpoint packing architecture.”
+
+You may not remove, obscure, or alter this attribution in a way that misrepresents authorship or origin.
+
+---
+
+## 6. Modification and Redistribution
+
+You may modify the Licensed Work and create Derivative Works for Non-Commercial Use, provided that:
+
+1. you clearly mark your modifications;
+2. you retain this License and all notices;
+3. you do not misrepresent modified work as the original;
+4. you do not remove attribution required by this License; and
+5. you do not use the modifications for Commercial Use without a Written Commercial License.
+
+Redistribution is permitted only if the recipient receives the Licensed Work under this same License and only for Non-Commercial Use unless otherwise authorized in writing by Licensor.
+
+---
+
+## 7. No Trademark License
+
+This License does not grant any right to use Licensor’s trade names, trademarks, logos, product names, or branding, including the name “PackedLLM,” except as necessary to comply with the attribution requirements of Section 5.
+
+Any goodwill arising from permitted use of the trademarks or product names shall inure to the benefit of Licensor.
+
+---
+
+## 8. Optional Commercial Terms; Revenue Share
+
+If Licensor grants a Written Commercial License, Licensor may impose additional terms, including without limitation:
+
+* a fee schedule;
+* per-seat, per-instance, or per-deployment pricing;
+* revenue share obligations;
+* reporting and audit obligations;
+* attribution requirements;
+* support and maintenance terms;
+* field-of-use restrictions;
+* geographic restrictions;
+* termination rights.
+
+Any such terms must be set out in a separate written agreement signed by Licensor and the commercial user.
+
+---
+
+## 9. Patent Rights
+
+To the extent Licensor holds patent rights that are necessarily infringed by the Licensed Work as distributed by Licensor, Licensor grants you a non-exclusive, worldwide, royalty-free patent license to practice those patent claims solely for Non-Commercial Use under this License.
+
+This patent grant does not apply to Commercial Use unless expressly authorized in a Written Commercial License.
+
+---
+
+## 10. No Warranty
+
+The Licensed Work is provided “AS IS,” without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, non-infringement, title, or quiet enjoyment.
+
+You are solely responsible for evaluating the suitability, safety, legality, and correctness of the Licensed Work for your use.
+
+---
+
+## 11. Limitation of Liability
+
+To the maximum extent permitted by law, Licensor shall not be liable for any damages arising out of or related to the Licensed Work, including without limitation direct, indirect, incidental, special, consequential, exemplary, or punitive damages, or loss of profits, data, business opportunity, goodwill, or reputation.
+
+---
+
+## 12. Termination
+
+This License automatically terminates upon any breach of its terms by you.
+
+Upon termination, you must immediately cease all use, copying, modification, distribution, and Commercial Use of the Licensed Work except to the extent otherwise permitted by applicable law.
+
+Sections relating to attribution, trademark, warranty disclaimer, limitation of liability, and any obligations that by their nature should survive termination shall survive termination.
+
+---
+
+## 13. Reservation of Rights
+
+All rights not expressly granted to you under this License are reserved by Licensor.
+
+No implied rights are granted by estoppel, implication, exhaustion, or otherwise.
+
+---
+
+## 14. Governing Terms for Commercial Access
+
+Any Commercial Use requires a separate written agreement. In the event of conflict between this License and a Written Commercial License, the Written Commercial License controls solely with respect to the covered commercial use and only to the extent of the conflict.
+
+---
+
+## 15. Entire License
+
+This License constitutes the entire public license governing Non-Commercial Use of the Licensed Work unless superseded by a separate written commercial agreement signed by Licensor.
+
+---
+
+## 16. Contact for Commercial Licensing
+
+To request Commercial Use authorization, contact:
+
+**Licensor:** [Chance Brownfield]
+**Email:** [HiMindAi@proton.me]
+
+---
+
+## 17. Acceptance
+
+By using, copying, modifying, or distributing the Licensed Work, you agree to be bound by the terms of this License.

README.md

@@ -1,5 +1,222 @@
----
-license: other
-license_name: packedlicensev1.0
-license_link: LICENSE
----
+---
+license: other
+language:
+- en
+- zh
+tags:
+- routing-of-experts
+- compound-ai
+- multi-expert
+- agentic
+- code-execution
+- web-search
+- persistent-memory
+- persona
+- chain-of-thought
+- llm-orchestration
+- llama-cpp
+- gguf
+- packed-model
+pipeline_tag: text-generation
+---
+# PackedLLM
+
+**~10B total parameters · ~3B active per inference · Routing-of-Experts (RoE) architecture**
+
+PackedLLM is a self-contained multi-expert language model system built around a **Routing-of-Experts (RoE)** mechanism. Rather than mixing expert outputs at the token level inside a shared transformer (Mixture-of-Experts), PackedLLM routes each request — and each stage of a multi-stage reasoning pipeline — to a dedicated, fully independent specialist model. At most one or two experts are active simultaneously, keeping peak memory around 3B parameters regardless of the 10B total footprint.
+
+The system runs entirely on consumer hardware via llama.cpp persists its full state to a single ZIP checkpoint, and integrates persistent vector memory, sandboxed Python execution, and multi-engine web search as first-class pipeline citizens.
+
+---
+
+## Architecture overview
+
+```
+PackedLLMRunner          ← user-facing shell: load, warmup, lifecycle
+      │
+PackedLLM (PackedLLM.pt) ← 9-stage orchestration pipeline
+      │
+Expert dispatch layer    ← 10 specialist models, one active at a time
+      │
+ ┌────┴─────────────────────────────┐
+ │ GATOR/MemoryBank   CodeBox   Web │  ← integrated modules
+ └────┬─────────────────────────────┘
+      │
+PackedLM (LM.pt)         ← llama.cpp inference engine + ExpertHandles
+```
+<img src="images/packedllm_architecture.png" alt="PackedLLM Architecture" width="1000">
+
+---
+
+## How it differs from MoE
+
+| | Standard MoE (Mixtral, DeepSeek) | PackedLLM RoE |
+|---|---|---|
+| **Routing granularity** | Per-token, inside every transformer layer | Per-task and per-pipeline-stage |
+| **What gets routed** | FFN sub-modules sharing one transformer | Separate, fully independent specialist LLMs |
+| **Parameters active** | Top-K experts × FFN size, across all layers | One expert at a time (~3B peak) |
+| **Router mechanism** | Learned linear gating vector | `HeadExpert` — a full LLM returning JSON |
+| **Experts share weights?** | Yes (all attention layers are always shared) | No — complete independence |
+| **Pipeline** | Single transformer forward pass | 9-stage: plan → route → execute → synthesize → persona → review |
+
+The closest related work is **Composition of Experts** (Chai et al., 2024, arXiv:2412.01868), which also routes at the input level to full LLM models. PackedLLM extends this with a multi-stage orchestration pipeline, per-stage retry/detour recovery, integrated memory and execution modules, affective state modelling, and a character persona layer — none of which appear in prior systems of this type.
+
+---
+
+## Pipeline stages
+
+Every call to `forward()` passes through these stages in order:
+
+| Stage | Expert | Temperature | What it does |
+|---|---|---|---|
+| 1. Plan goal | HeadExpert | 0.2 | Parse intent, tone, routing flags (needs_web / needs_action / needs_vision) |
+| 2. Consult memory | GATOR | — | Match registered commands; retrieve relevant memory |
+| 3. Build route | HeadExpert | 0.5 | Generate ordered list of expert steps as JSON |
+| 4. Execute route | various | per-expert | Dispatch each step; retry / detour / skip on failure |
+| 5. Synthesize base | HeadExpert | 1.0 | Combine step outputs into a persona-free prose answer |
+| 6. Affective state | AffectExpert | 0.5 | Generate bot emotional + physical state JSON |
+| 7. Apply persona | RoleExpert | 1.0 | Rewrite base response in character |
+| 8. Review | HeadExpert | 0.0 | Accept / revise / reject; extract memory facts; profile updates |
+| 9. Finalize | GATOR | — | Write memory, update user/bot profiles |
+
+---
+
+## Expert roster
+
+| Expert | Active params | Role | Notes                                     |
+|---|---|---|-------------------------------------------|
+| HeadExpert | ~3B | Orchestrator, router, planner, synthesizer, reviewer | Most-called expert     |
+| LogicExpert | ~1B | Structured reasoning; deep-think CoT; action planning/repair | raw completion with `<think>` blocks |
+| CodeExpert | ~1B | Python script generation for action pipeline | Temperature 0.0; raw code only, no prose  |
+| MathExpert | ~1B | Quantitative reasoning | Post-processes CJK spans; deduplicates repeated lines |
+| AffectExpert | ~0.5B | Emotional state; step quality evaluation | Used as both emotion classifier and pass/fail judge |
+| RoleExpert | ~0.5B | Persona rewriting in character | RP style chat format                      |
+| CreativeExpert | ~1B | Writing and stylistic generation | High temperature defaults (0.9)           |
+| VisionExpert | ~1B | Multimodal image understanding | CLIP projector; local images → data URI   |
+| ToolExpert | ~0.5B | Function-call generation | outputs `{"tool_calls": [...]}` JSON      |
+| TranslationExpert | ~300M | Chinese → English | seq2seq — not an LLM; Chinese regex gate  |
+
+**Total: ~10B · Peak active: ~3B**
+
+---
+
+## Forward modes
+
+### Standard (full pipeline)
+```python
+bot.chat("What is the compound interest on $5000 at 4% over 10 years?")
+```
+All 9 stages. Memory read/write. Web and action pipelines if needed.
+
+### Fast think (minimum latency)
+```python
+bot.chat("What time is it in Tokyo?", fast_think=True)
+```
+Skips planning, routing, memory, web, action, affective state, review. HeadExpert answers directly; RoleExpert applies persona if a bot profile exists. Maximum 2 LLM calls.
+
+### Deep think (CoT scaffolding)
+```python
+bot.chat("Design a Python caching decorator with TTL support.", deep_think=True)
+```
+Before each pipeline stage, `LogicExpert` generates `<think>...</think>` blocks scoped to that stage's specific task and output contract. These blocks are prepended to the stage's prompt as if the executing expert had already done that prior reasoning. Blocks are cached within a single `forward()` call. Translation is excluded (not an LLM).
+
+---
+
+## Integrated modules
+
+### MemoryBank (`GATOR.pt`)
+A multi-tree semantic store built on `PackedTree` — a custom embedding + KMeans clustering retrieval structure. Trees: knowledge, conversation, user profiles, bot profiles, commands, assets, telemetry. Hybrid retrieval scoring: 75% semantic similarity + 20% keyword overlap + 5% importance metadata. Embedding model: Jina Embeddings v3 (GGUF, stored inside the checkpoint). GATOR's own action planner uses `HeadExpert` to decide which memory operations to run. Also contains `DesktopControl` (OS automation) and `CommandRegistry` (text-to-action macros).
+
+### CodeBox (`CodeBox.pt`)
+Persistent Python sandbox with isolated virtual environment management, SHA256-verified asset registry, loader injection (`from _codebox_loader import load_asset` inside sandboxed code), DAG pipeline runner with `$var` reference passing between steps, LRU runner cache for expensive models, and hard RAM/CPU kill thresholds enforced by a monitoring thread.
+
+### Web (`WebSearch.pt`)
+Three search engines (DuckDuckGo HTML, Google, ResultHunter) with embedding-ranked candidate deduplication. Content extraction tries 10 methods: YouTube transcripts → trafilatura → boilerpy3 → readability → newspaper3k → goose3 → inscriptis → lxml → BeautifulSoup → visible text. PDF via PyMuPDF. Summarization via DistilBART. Runs in a separate spawned process; communicates via `multiprocessing.Queue`. Serializes safely — live process handles are stripped on save.
+
+---
+
+## Usage
+
+### Basic
+```python
+from PackedLLM import PackedLLMRunner
+
+bot = PackedLLMRunner("PackedLLM.pt", bot_id="pip", user_id="alice")
+print(bot.chat("Explain gradient descent in one paragraph."))
+```
+
+### Expert shortcuts (bypass full pipeline)
+```python
+bot.creative("Write a haiku about a robot discovering music.")
+bot.code("Implement binary search in Python with comments.")
+bot.math("Solve: integral of x² · sin(x) dx")
+bot.logic("All A are B. Some B are C. What follows?", mode="deep_then_answer")
+bot.translate("人工智能正在改变世界")
+bot.web("Latest developments in solid-state batteries?")
+bot.action("Compute compound interest on $5000 at 4% over 10 years; save to report.txt")
+```
+
+### Memory
+```python
+bot.memory_store("User prefers concise answers under 100 words.")
+results = bot.memory_recall("answer preferences", top_k=3)
+bot.set_user_profile({"name": "Alice", "expertise": "ML"})
+bot.set_bot_profile({"character_card": "You are Pip, a direct and slightly sarcastic assistant."})
+```
+
+### Lifecycle
+```python
+bot.unload_expert("vision_expert")   # free VRAM; reloads lazily on next use
+bot.reload_expert("code_expert")     # hot-reload after checkpoint update
+print(bot.status())                  # full system diagnostic
+
+# Context manager (auto-unload on exit)
+with PackedLLMRunner("PackedLLM.pt", bot_id="pip", user_id="alice") as bot:
+    print(bot.chat("Summarise the Pythagorean theorem."))
+```
+
+---
+
+## Checkpointing
+
+`PackedLLM.pt` is a ZIP archive containing:
+- `manifest.pt` — all metadata, profiles, hardware state, embedded source code
+- `lm_chunk_N.bin` — model weights in 32MB streaming chunks
+- `mem_chunk_N.bin` — GATOR memory store chunks
+- `web_chunk_N.bin` — WebSearch module chunks
+- `box_chunk_N.bin` — CodeBox chunks
+
+
+---
+
+## Hardware
+
+PackedLM detects and uses CUDA, Apple Metal (MPS), WebGPU, or CPU automatically via `HardwareProbe`. For each expert, `_plan_offload()` estimates the GGUF file size and computes how many transformer layers can fit in free VRAM (with a 15% safety margin for CUDA, 40% for WebGPU). If VRAM is insufficient for a full offload, layers are split proportionally between GPU and CPU.
+
+---
+
+## Citation
+
+```bibtex
+@software{packedllm2025,
+  Author     = {Chance Brownfield},
+  title     = {PackedLLM: A Routing-of-Experts System with LLM-Orchestrated Execution Pipeline},
+  year      = {2026},
+  note      = {RoE architecture: task-level routing to fully independent specialist LLMs.
+               Distinct from token-level Mixture-of-Experts.
+               Integrates persistent vector memory (GATOR), sandboxed Python execution (CodeBox),
+               and multi-engine web search in a 9-stage orchestration pipeline.}
+}
+```
+
+---
+
+## License
+
+This project is licensed under PackedLicense v1.0.
+
+Free for personal, educational, research, and other non-commercial use.
+
+Commercial use requires prior written authorization.
+
+The GATOR, WebSearchModule, and CodeBox components are protected under this license and may not be extracted, redistributed, or commercially reused without authorization.

requirements.txt

@@ -0,0 +1,26 @@
+numpy
+psutil
+requests
+torch
+torchvision
+torchaudio
+transformers
+sentence-transformers
+huggingface_hub
+safetensors
+spacy
+Pillow
+PyMuPDF
+scikit-learn
+beautifulsoup4
+lxml
+trafilatura
+readability-lxml
+newspaper3k
+goose3
+boilerpy3
+inscriptis
+llama-cpp-python
+wgpu
+youtube-transcript-api
+tqdm