# GitHub Issue Drafts (Complete Set) You can copy and paste the following markdown directly into new GitHub Issues to track the architectural migration, codebase optimization, and SaaS evolution of Talos. --- ## Issue 1: Consolidate Database Repositories into `talos_db` package **Labels:** `refactor`, `architecture`, `tech-debt` ### Description Currently, database access logic is fragmented. The `services/backend` has its own `repositories/config_repository.py`, while `packages/db/src/talos_db` already contains a Supabase client and a `repositories/` directory. To maintain a clean monorepo architecture, services should not contain direct database access logic. ### Acceptance Criteria - [ ] Move `services/backend/repositories/config_repository.py` to `packages/db/src/talos_db/repositories/`. - [ ] Update all imports in `services/backend` to import the repository from `talos_db`. - [ ] Ensure `services/backend` no longer has a `repositories/` directory. --- ## Issue 2: Migrate Search Execution Logic to `talos_search` package **Labels:** `refactor`, `architecture` ### Description The `search` domain is currently split. `packages/search/src/talos_search` handles low-level Typesense client/indexer initialization, but all the actual query execution logic (`hybrid.py`, `keyword.py`, `embeddings.py`, `vector.py`) is locked inside `services/backend/search/`. To allow other services (like `talos-manager`) to execute searches without duplicating code, all search logic must be consolidated into the shared package. ### Acceptance Criteria - [ ] Move the entire `services/backend/search/` directory into `packages/search/src/talos_search/`. - [ ] Update all routing and dependency imports in `services/backend/api/routers/` to point to the new package location. - [ ] Ensure `services/backend` no longer contains a `search/` domain directory. --- ## Issue 3: Extract AI Engines into `talos_ai` package **Labels:** `refactor`, `ai` ### Description The core AI business logic (`intent_engine`, `chat_engine`, `recommendation_engine`) is tightly coupled to `services/backend/`. The `packages/ai/src/talos_ai` package currently only holds the base `llm.py` wrapper. To enable `talos-manager` or background workers to simulate chats, test intents, and evaluate recommendations, these engines must be abstracted into the shared AI package. ### Acceptance Criteria - [ ] Move `intent_engine`, `chat_engine`, and `recommendation_engine` from `services/backend/` into `packages/ai/src/talos_ai/engines/`. - [ ] Verify that all internal FastAPI routers correctly import the engines from `talos_ai`. --- ## Issue 4: Unify Configuration & Environment Loading **Labels:** `refactor`, `tech-debt` ### Description There is fragmentation in how environment variables and configuration files are loaded across the monorepo (e.g., `services/backend/core/config.py` vs `services/talos-manager/core/settings.py`). This risks staging/production environments using mismatched settings across different services. ### Acceptance Criteria - [ ] Create a unified `packages/config` module utilizing Pydantic `BaseSettings`. - [ ] Move all environment variable parsing (Supabase URLs, Typesense keys, LLM keys) to this central package. - [ ] Refactor both `backend` and `talos-manager` to import their base configurations from `packages/config`. --- ## Issue 5: Implement Provider Pattern for Search Engine (Abstract Typesense) **Labels:** `architecture`, `feature`, `search` ### Description The `SearchService` currently has hardcoded dependencies on Typesense, executing Typesense-specific syntax and relying on the Typesense client. To prepare for future SaaS scale and allow hot-swapping of the search engine (e.g., to PostgreSQL or a custom Rust engine), we must abstract this into a Provider Pattern. ### Acceptance Criteria - [ ] Define an abstract base class `BaseSearchBackend` defining required methods (`search`, `search_candidates`, `known_facet_values`). - [ ] Create `TypesenseSearchBackend` that implements this interface. - [ ] Update `SearchService` to accept `BaseSearchBackend` via dependency injection, completely decoupling it from Typesense. --- ## Issue 6: Implement Redis for Semantic Caching & Sessions **Labels:** `performance`, `architecture` ### Description Talos currently relies on in-memory singletons (like the Refinement Engine's `SessionManager`) and lacks caching for expensive LLM intent-parsing calls. This prevents horizontal scaling of the FastAPI backend. ### Acceptance Criteria - [ ] Deploy a Redis container in `docker-compose.yaml`. - [ ] Refactor `SessionManager` to serialize and store active refinement sessions in Redis using `session_id` as the key. - [ ] Implement a semantic caching layer for `POST /ai/recommend` so identical semantic queries pull the parsed `OntologyEntry` from Redis rather than hitting the LLM. --- ## Issue 7: Dynamic Header-Based Multi-Tenancy **Labels:** `architecture`, `security` ### Description Talos currently operates in a multi-instance paradigm where one deployment serves one client by loading a `COMPANY_ID` environment variable at startup. To function as a true SaaS, the API must dynamically serve multiple clients simultaneously. ### Acceptance Criteria - [ ] Add an `X-Client-ID` (or API Key) header dependency to all relevant FastAPI endpoints. - [ ] Refactor the application to fetch configuration dynamically from Supabase at runtime based on the client header (using local LRU cache/Redis to prevent database spam). - [ ] Implement scoped search so Typesense queries are strictly isolated to the requesting client's collection prefix. --- ## Issue 8: Setup Strict Linting, Formatting, and Pre-commit Hooks **Labels:** `tooling`, `dx` ### Description To ensure code quality and prevent styling debates during code reviews, we need to enforce strict formatting and linting rules across both the Python and TypeScript codebases automatically. ### Acceptance Criteria - [ ] Add `ruff` and `black` to the Python backend requirements. - [ ] Ensure `mypy` is configured for strict static type checking in Python. - [ ] Add `prettier` and integrate it with ESLint in the frontend. - [ ] Configure `pre-commit` hooks (via `.pre-commit-config.yaml`) to run all formatters and linters automatically before a commit is allowed. --- ## Issue 9: Setup CI/CD Pipeline with GitHub Actions **Labels:** `ci-cd`, `infrastructure` ### Description We currently lack an automated continuous integration pipeline. We need to ensure that tests, type checks, and linting rules are enforced on every Pull Request before code can be merged into `main`. ### Acceptance Criteria - [ ] Create a `.github/workflows/ci.yml` file. - [ ] Configure jobs to run `pytest` for the backend and `vitest` for the frontend. - [ ] Configure jobs to run ESLint, Ruff, Black, and mypy on every PR. - [ ] Block merging if any of the CI checks fail. --- ## Issue 10: Implement Monorepo Build Tooling (Turborepo) **Labels:** `tooling`, `infrastructure` ### Description As the number of shared packages (e.g., `talos_db`, `talos_search`) and apps grows, manually managing dependencies and build scripts becomes unmanageable. We need a proper monorepo build system to cache outputs and run tasks in parallel. ### Acceptance Criteria - [ ] Initialize `turborepo` (or `nx`) at the root of the project. - [ ] Configure `turbo.json` with build, lint, and test pipelines. - [ ] Ensure `npm run dev` or `turbo run dev` seamlessly starts all frontend apps and backend services with hot-reloading. --- ## Issue 11: Optimize Frontend Rendering & Code Splitting **Labels:** `performance`, `frontend` ### Description To ensure the frontend remains snappy as the application grows, we need to implement React performance best practices to avoid unnecessary re-renders and bloated bundle sizes. ### Acceptance Criteria - [ ] Wrap heavy or off-screen components in `React.lazy()` for code splitting. - [ ] Audit the workflow builder and properties sidebar to ensure `useMemo` and `useCallback` are used appropriately to prevent unnecessary re-renders on drag-and-drop actions. - [ ] Resolve any strict-mode dependency warnings. --- ## Issue 12: Implement Learning-to-Rank Telemetry Pipeline **Labels:** `ai`, `search`, `feature` ### Description Talos's search accuracy is currently deterministic based on static YAML weights. To achieve SaaS-level accuracy, we must implement an automated feedback loop (Learning-to-Rank) that analyzes user interactions to adjust product scores dynamically. ### Acceptance Criteria - [ ] Utilize the existing `/telemetry/action` endpoint to track Click-Through Rates (CTR) and conversion events per search query. - [ ] Create a background pipeline (cron job or Celery worker) that aggregates telemetry data. - [ ] Automatically penalize or boost scoring weights in the search index if a product consistently underperforms or overperforms for a specific semantic intent. --- ## Issue 13: EPIC: Evaluate Core Search Engine Alternatives **Labels:** `epic`, `architecture`, `research` ### Description *(Long-term Strategic Goal)* To truly compete with Algolia and Typesense as a core Search Infrastructure SaaS, we cannot rely on Python to execute our core inverted index and vector similarity math due to the Global Interpreter Lock (GIL). We must evaluate migrating the underlying search execution engine. ### Acceptance Criteria - [ ] **Research Spike 1:** Benchmark PostgreSQL utilizing `pgvector` and `pg_search` (Tantivy/BM25) against our current Typesense setup for latency at scale. - [ ] **Research Spike 2:** Evaluate wrapping Apache Lucene or OpenSearch with our AI routing logic. - [ ] **Research Spike 3:** Scope the engineering effort required to build a custom memory-mapped inverted index and HNSW graph in Rust. - [ ] Draft an Architectural Decision Record (ADR) detailing the chosen path forward. --- ## Issue 14: Chat History and Sharing do not support multiple bot workflows **Labels:** `bug`, `frontend`, `backend` ### Description The chat engine was recently updated to support multiple bot workflows per company: - The default workflow is accessed via `/{company_id}` (e.g., `http://localhost:5173/Newmarcorp`) - Specific bot workflows are accessed via `/{company_id}/bot/{bot_id}` (e.g., `http://localhost:5173/Newmarcorp/bot/Legacy%20Workflow%20Template/`) This architecture change has introduced two distinct regressions in areas of the codebase that still assume a single global chat workflow: #### 1. Chat History Context Loss Currently, the `chat_sessions` table does not track the `bot_id` associated with a conversation. As a result, when a user selects a past conversation from the chat history sidebar, the application has no way of knowing which bot workflow that conversation belongs to. It becomes impossible to restore the exact bot context. #### 2. Chat Sharing Links Broken The "Share Chat" functionality generates links that do not account for the `bot_id`. When a shared chat link is opened, the frontend application defaults to the standard workflow or fails to route properly. The sharing logic needs to be aware of the bot workflow. ### Proposed Changes #### Backend / Database 1. **Schema Update:** Add a `bot_id` column to the `chat_sessions` table in Supabase. 2. **Session Creation:** Update `SessionRepository.create()` to accept and persist the `bot_id` when a new chat session is started. 3. **Session Retrieval:** Ensure that `bot_id` is returned in the API responses for chat history (`GET /ai/chat/{session_id}/messages` and the frontend's history list queries). #### Frontend 1. **History Routing:** Update the chat history sidebar so that clicking a historical session routes the user to the correct URL (e.g., `/{company_id}/bot/{bot_id}/{session_id}`). 2. **Share Link Generation:** Update `ShareDialog.tsx` and the underlying routing so that shared links either explicitly include the `bot_id` in the URL structure or infer it automatically from the session data fetched from the backend.