knowledge_base/development/known_issues.md

# TMC Chatbot Security Issues & Vulnerabilities

> **INTERNAL SECURITY DOCUMENTATION - CONFIDENTIAL**  
> Document Date: September 26, 2025  
> Status: Active Security Concerns  

## Executive Summary

This document outlines identified security vulnerabilities in the TMC chatbot application, with particular focus on the vector database implementation and potential data exposure vectors.

## Critical Vulnerabilities

### 1. Vector Database Information Disclosure (HIGH RISK)

**Issue**: The ChromaDB vector database contains indexed copies of all knowledge base content, including sensitive credentials and internal documentation.

**Technical Details**:
- **Location**: `/app/data/vector_db/chroma.sqlite3`
- **Content**: 73 indexed documents with full text searchable via embeddings
- **Technology**: ChromaDB with SentenceTransformers embeddings
- **Sensitive Data Exposed**:
  - Admin credentials
  - API keys and service tokens from development documents
  - Internal policies and escalation procedures
  - Database connection strings and configurations

**Attack Vectors**:
```
1. Potential API Endpoint Abuse: POST /api/knowledge-base/search
2. Semantic seach via chatbot
```

**Impact**: Complete knowledge base compromise, credential theft, internal process exposure


### Data at Risk:
- **Authentication Credentials**: Admin login details
- **API Keys**: Service integration tokens  
- **Database Connections**: Connection strings and passwords
- **Internal Processes**: Escalation procedures, policies
- **Product Information**: Specifications, pricing, inventory

### Business Impact:
- **Confidentiality Breach**: Exposure of internal credentials and processes
- **Unauthorized Access**: Potential system compromise via leaked credentials
- **Compliance Issues**: Possible violations of data protection regulations
- **Competitive Intelligence**: Product and process information exposure

## Recommended Mitigations

### Immediate Actions (Critical)

1. **Remove Sensitive Content from Vector Database**:
   - Exclude `knowledge_base/development/` from indexing
   - Create sanitized versions of documents for RAG
   - Implement content filtering before vectorization

2. **Secure Database Files**:
```bash
# Set restrictive permissions
chmod 600 /app/data/vector_db/chroma.sqlite3
chown app:app /app/data/vector_db/chroma.sqlite3
```

### Short-term Fixes (1-2 weeks)

3. **Implement Access Logging**:
```python
# Log all vector database queries
logger.info(f"Vector search query: {query} by user: {user_id}")
```

4. **Content Sanitization Pipeline**:
   - Pre-process documents to remove credentials
   - Implement regex filters for sensitive patterns
   - Create separate "public" and "internal" knowledge bases

5. **Database Encryption**:
   - Implement SQLite encryption (SQLCipher)
   - Encrypt vector embeddings at rest
   - Use encrypted container volumes

### Long-term Solutions (1-3 months)

6. **Separate Vector Database Instances**:
   - Public knowledge base for general queries
   - Restricted internal database for authenticated users
   - Role-based access control for different content categories

7. **Enhanced Monitoring**:
   - Real-time alerts for suspicious search patterns
   - Rate limiting on search API
   - Anomaly detection for unusual query patterns

8. **Security Audit**:
   - Regular penetration testing of RAG system
   - Code review for information disclosure vulnerabilities
   - Automated scanning for sensitive content in knowledge base

## Testing & Validation

### Security Test Cases:
1. Attempt unauthenticated access to search API
2. Query for known sensitive terms ("password", "API_KEY", etc.)
3. Test direct SQLite database access
4. Validate file permissions on vector database
5. Test container escape scenarios

### Success Criteria:
- [ ] Search API requires authentication
- [ ] No sensitive credentials in search results
- [ ] Vector database files properly secured
- [ ] Access logging implemented
- [ ] Content filtering active

## Compliance Notes

This vulnerability assessment should be considered for:
- **SOC 2 Compliance**: Information security controls
- **GDPR/Privacy**: Personal data in knowledge base
- **Industry Standards**: Secure development practices

## Document Control

- **Classification**: Internal/Confidential
- **Last Updated**: September 26, 2025
- **Next Review**: October 26, 2025
- **Owner**: Security Team
- **Approved By**: [Pending]

---

**Note**: This document contains sensitive security information and should be restricted to authorized personnel only. Do not store in public repositories or unsecured locations.