update the readme file

README.md

@@ -113,8 +113,15 @@ Then access:
 - 🔐 **Fine-Grained Role-Based Access Control (RBAC)** – Four-tier role system (viewer, editor, admin, owner) with dynamic UI visibility and backend permission enforcement; frontend automatically shows/hides features based on role
 - 🔄 **Intelligent Multi-Tool Orchestration** – MCP agent orchestrator autonomously selects optimal tool chains (RAG + Web + LLM, etc.) based on query intent, context, latency predictions, and previous tool outputs. Context-aware routing enables sophisticated tool skipping for efficiency
 - ⚡ **Robust Error Handling** – Structured error responses, retry mechanisms, and graceful fallbacks (e.g., if RAG fails → fallback to LLM-only)
-- 📡 **Streaming Responses** – Chat responses stream word-by-word using Server-Sent Events (SSE) for real-time user experience
+- 📡 **Streaming Responses** – Chat responses stream character-by-character using Server-Sent Events (SSE) for real-time user experience
 - 🎯 **Rule-First Processing** – Admin rules checked before intent classification - rules can trigger brief responses or block requests entirely
+- 🧠 **Advanced Context Engineering** – Implements Anthropic's context engineering strategies:
+  - **High-Fidelity Compaction**: Automatically compresses conversations at 80% token threshold, preserving architectural decisions and unresolved issues
+  - **Tool Result Clearing**: Safest form of compaction - removes large tool outputs while keeping metadata
+  - **Structured Note-Taking**: Tracks objectives, architectural decisions, and unresolved issues outside context window
+  - **XML-Structured Prompts**: All prompts use clear XML sections for better model understanding
+  - **Just-in-Time Context Loading**: Selects only relevant memories and tools for each query
+  - **Progressive Disclosure**: Agents discover context incrementally through exploration
 
 ### Enterprise Features
 
@@ -1006,6 +1013,22 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 - **Massive Accuracy Improvement**: Re-ranking significantly improves relevance of search results
 - **Seamless Integration**: Works transparently with existing RAG search API
 
+### Context Engineering (Latest)
+- **Anthropic-Inspired Strategies**: Implements best practices from Anthropic's context engineering research:
+  - **Compaction**: High-fidelity summarization preserving architectural decisions, unresolved issues, and implementation details
+  - **Tool Result Clearing**: Safest form of compaction - removes large tool outputs once processed
+  - **Structured Note-Taking**: Tracks objectives (like Claude playing Pokémon), architectural decisions, and unresolved issues
+  - **XML-Structured Prompts**: All prompts use clear XML sections (`<system>`, `<background_information>`, `<instructions>`) for better model understanding
+  - **Automatic Compression**: Conversations compressed at 80% token threshold, targeting 60% after compression
+  - **Just-in-Time Context**: Selects only relevant memories and tools for each query
+  - **Progressive Disclosure**: Agents discover context incrementally through exploration
+- **Benefits**: 
+  - Reduced token usage and costs
+  - Longer conversation support
+  - Better agent coherence across extended interactions
+  - Improved performance through structured context
+- **Documentation**: See `ANTHROPIC_CONTEXT_ENGINEERING.md` and `CONTEXT_ENGINEERING_IMPLEMENTATION.md` for detailed implementation
+
 ### UI Improvements
 - **Modern Drag-and-Drop**: Intuitive file upload with visual feedback
 - **Enhanced Status Messages**: Clear success/error messages with icons

backend/README.md

@@ -265,6 +265,41 @@ The Next.js frontend includes three powerful visualization components:
 
 All visualizations are accessible to all roles and automatically populate when agent responses include `reasoning_trace` and `tool_traces` data.
 
+### Context Engineering (Latest)
+
+The system implements comprehensive context engineering strategies based on Anthropic's best practices:
+
+- **ContextEngineer Service** (`backend/api/services/context_engineer.py`):
+  - **ContextScratchpad**: Structured note-taking with objectives, architectural decisions, and unresolved issues
+  - **ContextCompressor**: High-fidelity compaction and tool result clearing
+  - **ContextSelector**: Just-in-time context loading and memory selection
+  - **ContextIsolator**: Isolation of large tool outputs
+
+- **Compaction Strategy**:
+  - Monitors token usage and compresses at 80% threshold
+  - Uses tool result clearing first (safest), then full compaction
+  - Preserves architectural decisions, unresolved issues, and implementation details
+  - Targets 60% token usage after compression
+
+- **Structured Prompts**:
+  - All prompts use XML-style sections (`<system>`, `<background_information>`, `<instructions>`)
+  - Clear organization improves model understanding
+  - Better separation of concerns
+
+- **Integration Points**:
+  - Conversation history compression in `agent_orchestrator.py`
+  - Tool output compression for RAG and web search
+  - Structured scratchpad context in all prompts
+  - Memory selection before tool selection
+
+- **Benefits**:
+  - Reduced token usage and API costs
+  - Support for longer conversations
+  - Better agent coherence across extended interactions
+  - Improved performance through structured context
+
+See `ANTHROPIC_CONTEXT_ENGINEERING.md` and `CONTEXT_ENGINEERING_IMPLEMENTATION.md` in the root directory for detailed documentation.
+
 ## Environment Variables (excerpt)
 
 Defined in `env.example`: