docs/WORKLOG.md

Worklog

2026-01-17 – Work Package C

• Summary

  • Productionised the backend for deployment on Hugging Face Spaces (Docker) and added a minimal Streamlit frontend suitable for Streamlit Community Cloud.
  • Introduced optional API key protection, rate limiting, and in-memory caching.
  • Added in-memory metrics with a /metrics endpoint and an asyncio-based benchmark script.

• Key Files Changed

  • Backend runtime and deployment:
    • backend/Dockerfile
    • backend/app/core/runtime.py
  • Security, CORS, rate limiting, and caching:
    • backend/app/core/security.py
    • backend/app/core/rate_limit.py
    • backend/app/core/cache.py
  • Metrics:
    • backend/app/core/metrics.py
    • backend/app/routers/metrics.py
  • Routers and configuration:
    • backend/app/main.py
    • backend/app/routers/chat.py
    • backend/app/routers/search.py
    • backend/app/routers/ingest.py
    • backend/app/core/config.py
  • Dependencies and environment:
    • backend/requirements.txt
    • backend/.env.example
  • Tooling and frontend:
    • scripts/bench_local.py
    • frontend/app.py
    • requirements.txt (root)
  • Documentation:
    • backend/README.md
    • docs/CONTEXT.md

• Major Decisions

  • Use port 7860 by default in the Docker image, while respecting the PORT environment variable for platforms like Hugging Face Spaces.
  • Keep API key protection opt-in via API_KEY with clear logging when disabled.
  • Enable rate limiting and caching by default, with simple boolean toggles (RATE_LIMIT_ENABLED, CACHE_ENABLED) for easy operational control.
  • Implement metrics as in-memory only (no external storage) and expose them via a JSON /metrics endpoint tailored for demos and lightweight monitoring.

2026-01-17 – Security + UI + Ingestion Hardening

• Summary

  • Hardened the backend for public deployment by enforcing API key protection for all non-health endpoints and (initially) for the OpenAPI/Swagger documentation, then relaxed docs to be publicly viewable while keeping all functional endpoints protected.
  • Upgraded the Streamlit frontend to a conversational chat UI using Streamlit's chat primitives.
  • Improved local document ingestion workflows with Docling-aware scripts for single files and batch folder ingestion.
  • Added a UI-based document upload dialog in the Streamlit app that ingests files via /documents/upload-text.

• Key Files Changed

  • Backend authentication and wiring:
    • backend/app/core/auth.py
    • backend/app/core/security.py
    • backend/app/main.py
  • Frontend chatbot UI and upload:
    • frontend/app.py
    • frontend/services/file_convert.py
    • frontend/services/backend_client.py
  • Local ingestion scripts:
    • scripts/docling_convert_and_upload.py
    • scripts/batch_ingest_local_folder.py
  • Documentation:
    • backend/README.md
    • docs/CONTEXT.md
    • docs/WORKLOG.md (this file)

• Major Decisions

  • In production-like environments (ENV=production or on Hugging Face Spaces), require API_KEY and fail fast at startup when it is missing; Swagger/OpenAPI remain publicly accessible but all non-health API endpoints still enforce X-API-Key.
  • Use a single require_api_key dependency (based on APIKeyHeader) to protect all routers except /health.
  • Treat Streamlit as a first-class chat client, using st.chat_message/st.chat_input with session-based history and optional streaming from /chat/stream.
  • Keep Docling as an optional dependency used in:
    • Local ingestion scripts that upload text to /documents/upload-text.
    • The frontend upload dialog for converting PDFs/Office/HTML when available, while falling back to raw .md/.txt and showing clear errors otherwise.