feat: Add comprehensive observability and telemetry logging via JSONL and rotating file handlers

chore: remove tracked generated files and update gitignore

Update chat backend to include agentic features