Spaces:
Paused
Paused
WidgeTDC - Enterprise AI Platform Architecture
Last Updated: 2025-12-13 Status: ✅ PRODUCTION LIVE - Cloud Deployment Active
System Overview
WidgeTDC is an Enterprise-grade Autonomous Intelligence Platform featuring:
- Semantic Vector Database with multi-provider embeddings
- Autonomous Task Engine with learning loops
- Distributed Event System for scalability
- Production-ready Infrastructure with monitoring and logging
- Modern Frontend with React 19 and Tailwind CSS v4
🚀 Production Deployment (2025-12-13)
Live URLs
| Service | Platform | URL |
|---|---|---|
| Frontend | Vercel Edge | https://matrix-frontend-claus-krafts-projects.vercel.app |
| Backend API | HuggingFace Spaces | https://kraft102-widgettdc-api.hf.space |
Cloud Infrastructure
┌─────────────────────────────────────────────────────────────────────────┐
│ PRODUCTION ARCHITECTURE │
│ "Headless Ghost" Pattern │
└─────────────────────────────────────────────────────────────────────────┘
Browser ──▶ Vercel CDN ──▶ HuggingFace Spaces ──▶ Cloud Databases
│ │ │
React 19 PWA Node.js 20 ┌────┴────┐
185 Widgets 63 MCP Tools │ Redis │
Zustand State Express API │ Neo4j │
│ Neon PG │
└─────────┘
Database Layer
| Database | Provider | Purpose |
|---|---|---|
| Redis | Redis Cloud (EU-North-1) | Cache, Sessions, Event Bus |
| Neo4j | AuraDB Cloud | Knowledge Graph (4274 nodes, 5252 rels) |
| PostgreSQL | Neon Serverless | Vector Store (pgvector), Prisma ORM |
| SQLite | sql.js (in-memory) | Fallback for HF Spaces free tier |
AI/LLM Providers
| Provider | Use Case |
|---|---|
| Google Gemini Pro | Primary reasoning |
| OpenAI GPT-4 | Fallback, embeddings |
| Anthropic Claude | Analysis tasks |
| DeepSeek V3 | Code generation |
Recent Updates (2025-12-13)
✅ HuggingFace Spaces Deployment Fixed
- Dockerfile Optimized: Skip Puppeteer Chromium download
- Memory Limits: Reduced to 512MB for free tier compatibility
- package-lock.json: Regenerated correct monorepo lockfile
- Data Directories:
/app/data/*created with proper permissions - Environment Variables:
HF_SPACE=true,DOCKER=trueset
✅ Frontend Widgets Fixed
- Icon Imports: Added missing
Target,Videoimports - DataSources: Added
threatIntelto SourceLink.tsx - useEffect Fix: Removed undefined
width/heightfrom dependency array - 185 Widgets: All rendering correctly
✅ Previous Updates (2025-11-26)
- CSS Architecture rebuilt for Tailwind v4
- PowerShell scripts standardized (PSScriptAnalyzer compliant)
- Build system fully functional
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────┐
│ FRONTEND (React 19) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Widget Board │ │ MCP Client │ │ Dashboard │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └──────────────────┴──────────────────┘ │
│ │ WebSocket + REST API │
└────────────────────────────┼─────────────────────────────────────────────┘
│
┌────────────────────────────┼─────────────────────────────────────────────┐
│ BACKEND (Node.js + Express) │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ API LAYER │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │ MCP │ │ SRAG │ │ PAL │ │ AI │ ... │ │
│ │ │ Router │ │ API │ │ API │ │ Proxy │ │ │
│ │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │
│ │ │ │ │ │ │ │
│ └───────┼────────────┼────────────┼─────────────┼─────────────────┘ │
│ │ │ │ │ │
│ ┌───────┴────────────┴────────────┴─────────────┴─────────────────┐ │
│ │ COGNITIVE LAYER │ │
│ │ │ │
│ │ ┌──────────────────┐ ┌──────────────────┐ │ │
│ │ │ Autonomous Agent │ │ UnifiedGraphRAG │ │ │
│ │ │ ┌────────────┐ │ │ Multi-hop RAG │ │ │
│ │ │ │ Task Engine│ │ └────────┬─────────┘ │ │
│ │ │ │ & Learning │ │ │ │ │
│ │ │ └─────┬──────┘ │ │ │ │
│ │ └────────┼─────────┘ │ │ │
│ │ │ │ │ │
│ │ ┌────────┴───────────────────────┴──────────┐ │ │
│ │ │ Unified Memory System │ │ │
│ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐│ │ │
│ │ │ │ Working │ │ Episodic │ │Procedural││ │ │
│ │ │ │ Memory │ │ Memory │ │ Memory ││ │ │
│ │ │ └──────────┘ └──────────┘ └──────────┘│ │ │
│ │ └────────────────────┬───────────────────────┘ │ │
│ │ │ │ │
│ └───────────────────────┼─────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┴─────────────────────────────────────────┐ │
│ │ DATA & EMBEDDING LAYER │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────┐ │ │
│ │ │ Embedding Service (Multi-Provider) │ │ │
│ │ │ ┌─────────┐ ┌────────────┐ ┌──────────────┐ │ │ │
│ │ │ │ OpenAI │ │HuggingFace │ │Transformers.js│ │ │ │
│ │ │ │ 1536D │ │ 768D │ │ 384D │ │ │ │
│ │ │ └────┬────┘ └─────┬──────┘ └──────┬───────┘ │ │ │
│ │ │ └─────────────┴────────────────┴─ Auto-select │ │ │
│ │ └───────────────────────┬──────────────────────────────────┘ │ │
│ │ │ Generate embeddings │ │
│ │ ┌───────────────────────▼──────────────────────────────────┐ │ │
│ │ │ PgVectorStoreAdapter (pgvector) │ │ │
│ │ │ • Auto-embedding generation │ │ │
│ │ │ • Semantic text search │ │ │
│ │ │ • Cosine similarity │ │ │
│ │ │ • Namespace isolation │ │ │
│ │ └───────────────────────┬──────────────────────────────────┘ │ │
│ │ │ │ │
│ └──────────────────────────┼───────────────────────────────────────┘ │
│ │ │
│ ┌──────────────────────────┴───────────────────────────────────────┐ │
│ │ EVENT & INTEGRATION LAYER │ │
│ │ │ │
│ │ ┌──────────────────┐ ┌──────────────────────────────────┐ │ │
│ │ │ Redis EventBus │◄───┤ Ingestion Pipeline │ │ │
│ │ │ (Production) │ │ • Data Sources (20+) │ │ │
│ │ │ OR │ │ • Browser History │ │ │
│ │ │ In-Memory │ │ • Email (Gmail, Outlook) │ │ │
│ │ │ (Development) │ │ • Cloud (Drive, OneDrive) │ │ │
│ │ └──────────────────┘ │ • Social (Twitter, LinkedIn) │ │ │
│ │ └──────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
└────────────────────────────┬─────────────────────────────────────────────┘
│
┌────────────────────────────┼─────────────────────────────────────────────┐
│ PERSISTENCE LAYER │
│ ▼ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ Neo4j │ │
│ │ + pgvector │ │ Event Store │ │ Knowledge Graph │ │
│ │ • Embeddings │ │ • Pub/Sub │ │ • Entities │ │
│ │ • Documents │ │ • Cache │ │ • Relations │ │
│ │ • Metadata │ │ • Sessions │ │ • Inference │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
Directory Structure
WidgeTDC/
├── apps/
│ ├── backend/ # Express.js backend
│ │ ├── src/
│ │ │ ├── mcp/ # MCP protocol implementation
│ │ │ ├── cognitive/ # AI agents & RAG
│ │ │ ├── memory/ # Memory systems
│ │ │ ├── ingestion/ # Data ingestion
│ │ │ └── api/ # REST endpoints
│ │ └── package.json
│ │
│ └── matrix-frontend/ # React 19 frontend ✅ BUILD FIXED
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── widgets/ # Widget implementations
│ │ └── mcp/ # MCP client
│ ├── App.css # ✅ REBUILT - All keyframes fixed
│ ├── App.tsx
│ └── package.json
│
├── packages/
│ └── shared/ # Shared types & utilities
│ ├── types/
│ └── utils/
│
├── scripts/ # ✅ UPDATED - PowerShell scripts
│ ├── update_agent.ps1 # ✅ NEW - Agent updater
│ ├── update_agent_github.ps1 # ✅ NEW - GitHub workflow updater
│ ├── update_agent_readme.ps1 # ✅ NEW - README updater
│ ├── build-shared.ps1
│ ├── install-deepseek.ps1
│ ├── setup-enterprise.ps1
│ └── test_rag_flow.ps1
│
├── database/
│ ├── migrations/ # SQL migrations
│ └── schema.sql
│
├── .github/
│ └── workflows/ # GitHub Actions
│ ├── agent-*.yml # Agent workflows
│ └── ci.yml
│
├── docs/ # Documentation
├── tests/ # Test suites
└── dist/ # ✅ Build output (created successfully)
Technology Stack
Frontend
- Framework: React 19 (latest)
- Build Tool: Vite 6.4.1
- Styling: Tailwind CSS v4 (PostCSS)
- State Management: Zustand
- Real-time: WebSocket client
- Testing: Vitest + React Testing Library
Backend
- Runtime: Node.js 20+
- Framework: Express.js
- Language: TypeScript (strict mode)
- Real-time: WebSocket (
wslibrary) - MCP: Custom implementation
Database
- Primary: PostgreSQL 15+ with pgvector
- Cache/Events: Redis 7+
- Graph: Neo4j 5+ (optional)
- Development: SQLite (fallback)
DevOps
- Containerization: Docker + Docker Compose
- CI/CD: GitHub Actions
- Deployment: Railway (configured)
- Monitoring: PM2 ecosystem
Key Components
1. MCP (Model Context Protocol)
- Location:
apps/backend/src/mcp/ - Purpose: Standardized AI agent communication
- Features:
- Tool registration and execution
- Resource management
- Event broadcasting
- WebSocket transport
2. UnifiedGraphRAG
- Location:
apps/backend/src/cognitive/UnifiedGraphRAG.ts - Purpose: Multi-hop semantic retrieval
- Features:
- Entity extraction
- Relation mapping
- Context-aware search
- Hybrid retrieval (vector + graph)
3. Autonomous Task Engine
- Location:
apps/backend/src/cognitive/AutonomousAgent.ts - Purpose: Self-directed task execution
- Features:
- Goal decomposition
- Learning from outcomes
- Dynamic replanning
- Memory integration
4. Widget System
- Location:
apps/matrix-frontend/src/widgets/ - Purpose: Modular UI components
- Features:
- Dynamic loading
- MCP integration
- Drag-and-drop layout
- Settings persistence
Build & Deployment Status
✅ Frontend Build
npm run build --prefix apps/matrix-frontend
# Status: ✅ SUCCESS (6.40s)
# Output: dist/ directory created
# Issues: None
✅ CSS Architecture
- File:
apps/matrix-frontend/App.css - Status: Completely rebuilt
- Keyframes: All properly structured
- Tailwind: v4 compatible (no @apply)
- Animations: All functional
✅ PowerShell Scripts
- Location:
scripts/ - Status: PSScriptAnalyzer compliant
- Issues: None
Configuration Files
Build Configuration
vite.config.ts- Vite build settingstsconfig.json- TypeScript compiler optionstailwind.config.js- Tailwind CSS v4 configpostcss.config.js- PostCSS plugins
Runtime Configuration
ecosystem.config.js- PM2 process managementdocker-compose.yml- Container orchestrationrailway.json- Railway deployment config
Development Workflow
Local Development
# Install dependencies
npm install
# Start backend
npm run dev:backend
# Start frontend (separate terminal)
npm run dev:frontend
# Run tests
npm test
Production Build
# Build frontend
npm run build --prefix apps/matrix-frontend
# Build backend
npm run build --prefix apps/backend
# Start production server
npm start
Security & Compliance
Authentication
- JWT-based authentication
- Session management via Redis
- CORS configured for production
Data Protection
- Encryption at rest (PostgreSQL)
- TLS for all network traffic
- Environment variable isolation
Compliance
- GDPR-ready data handling
- Audit logging
- Data retention policies
Monitoring & Observability
Logging
- Structured JSON logs
- Log levels: ERROR, WARN, INFO, DEBUG
- Persistent storage in
logs/
Metrics
- PM2 monitoring
- Database connection pooling
- WebSocket connection tracking
Health Checks
/healthendpoint- Database connectivity
- Redis availability
- Memory usage
Known Issues & Limitations
✅ Resolved
CSS build errors in App.cssFIXED 2025-11-26PowerShell script warningsFIXED 2025-11-26Tailwind @apply compatibilityFIXED 2025-11-26
Active
- None currently blocking
Future Enhancements
Planned Features
- Multi-tenant Support: Workspace isolation
- Advanced Analytics: Usage dashboards
- Plugin System: Third-party widget support
- Mobile App: React Native client
Infrastructure
- Kubernetes: Container orchestration
- Prometheus: Metrics collection
- Grafana: Visualization
- ELK Stack: Log aggregation
Contributing
See CONTRIBUTING.md for development guidelines.
License
See LICENSE for details.
Architecture maintained by: WidgeTDC Development Team
Last reviewed: 2025-11-26
Next review: 2025-12-26