# BeFree API Reference This document provides a complete inventory of the REST API endpoints exposed by the BeFree backend server, including authentication requirements, request payloads, and response structures. --- ## Global Authentication Rules - **Bearer Tokens**: Endpoints requiring authentication look for a JWT token in the `Authorization: Bearer ` header. - **Roles**: - `tester` (default): View dashboards, upload and review statements. - `manager`: Access dashboard metrics, review history, and trigger exports. - `admin`: Full access, including rule manipulation, audit logs, categories, database resets, and administrative commands. --- ## 1. Authentication Endpoints ### `POST /api/v1/auth/login` - **Auth Required**: No - **Request Body**: Form parameters (`username`, `password`) - **Response Schema**: `TokenResponse` (`access_token`, `token_type`, `role`, `username`) - **Description**: Verifies credentials and returns a signed JWT access token. ### `POST /api/v1/auth/logout` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: None (retrieves token from authorization header) - **Response Schema**: `{"status": "success", "message": "Logged out"}` - **Description**: Revokes the active session token in the database. ### `GET /api/v1/auth/me` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `UserProfile` (`id`, `username`, `email`, `fullName`, `role`) - **Description**: Returns the profile details of the currently authenticated user. ### `POST /api/v1/auth/seed-users` - **Auth Required**: Yes (requires `X-Seed-Secret` header matching `SEED_SECRET` env var) - **Response Schema**: `{"status": "success", "seeded_count": int}` - **Description**: Seeds initial system users (`admin`, `manager`, `tester`) into the PostgreSQL database. --- ## 2. Dashboard Endpoints ### `GET /api/v1/dashboard/metrics` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `DashboardMetrics` (`totalStatements`, `processedSuccessfully`, `failedStatements`, `averageAccuracy`, `totalFlaggedTransactions`) - **Description**: Aggregates high-level processing statistics for the main dashboard display. ### `GET /api/v1/dashboard/recent-activity` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `List[ActivityItem]` (`id`, `timestamp`, `username`, `action`, `description`) - **Description**: Lists recent database audit log activities. ### `GET /api/v1/dashboard/recent-statements` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `List[StatementSummary]` (`id`, `fileName`, `status`, `uploadedAt`, `accuracy`) - **Description**: Lists recently processed statement metadata. --- ## 3. Statement Processing Endpoints ### `POST /api/v1/statements/upload` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: Multipart form data with key `files` (array of PDF files) - **Response Schema**: `UploadInitResponse` (`statementId`, `fileName`, `status`, `ocrFallbackTriggered`) - **Description**: Ingests PDF files, initializes the transaction database, and caches the raw PDFs on disk. ### `POST /api/v1/statements/{statement_id}/detect-layout` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `HeaderDetectionResultModel` - **Description**: Analyzes pages in the statement to automatically detect header coordinates and column mappings. ### `GET /api/v1/statements/{statement_id}/layout` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `HeaderDetectionResultModel` - **Description**: Retrieves the cached or detected column layout map. ### `GET /api/v1/statements/{statement_id}/page-image/{page_num}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: PNG binary stream - **Description**: Generates and serves a rasterized PNG image of a statement page (used for visual verification in the coordinate inspector). ### `POST /api/v1/statements/{statement_id}/verify-layout` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `HeaderDetectionResultModel` - **Response Schema**: `{"success": true}` - **Description**: Saves layout coordinates adjusted manually by the user in the UI coordinate builder. ### `PATCH /api/v1/statements/{statement_id}/layout/column/{col_name}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: JSON mapping for x-coordinates (`x_min`, `x_max`) - **Response Schema**: `{"success": true}` - **Description**: Modifies coordinate ranges for a single column. ### `POST /api/v1/statements/{statement_id}/process` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": true, "message": "Extraction started"}` - **Description**: Initiates transaction cell extraction and audits in a background thread. ### `POST /api/v1/statements/{statement_id}/finalise` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": true}` - **Description**: Finalizes the statement review and generates the default spreadsheet. ### `GET /api/v1/statements/{statement_id}/processing` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `ProcessingState` - **Description**: Polls the background thread extraction progress, stages completed, and remaining duration. ### `POST /api/v1/statements/{statement_id}/processing/cancel` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `None` - **Description**: Cancels background processing for a statement and sets its status to `cancelled`. ### `POST /api/v1/statements/{statement_id}/save-progress` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"status": "success", "message": "Progress saved"}` - **Description**: Forces writing the active transaction and correction log states to the excel cached workbook. ### `GET /api/v1/statements/{statement_id}/pdf` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: PDF binary stream - **Description**: Downloads the raw uploaded PDF file. --- ## 4. Transaction Endpoints ### `GET /api/v1/statements/{statement_id}/transactions` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Query Params**: `page` (int), `pageSize` (int), `search` (str), `status` (str), `pageNum` (int), `confidence` (str: `high`|`low`) - **Response Schema**: `TransactionsPage` (`items`: `List[TransactionRowModel]`, `total`: int, `page`: int, `pageSize`: int) - **Description**: Retrieves a paginated list of transactions, supporting search queries, status filters, page limits, and date windowing configurations. ### `POST /api/v1/statements/{statement_id}/transactions` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `TransactionCreatePayload` (`pageNumber`, `date`, `description`, `debit`, `credit`, `balance`) - **Response Schema**: `TransactionRowModel` - **Description**: Inserts a manual transaction row, triggers a statement re-audit, and persists the cache. ### `GET /api/v1/statements/{statement_id}/transactions/{tx_id}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `TransactionRowModel` - **Description**: Returns a single transaction details. ### `PATCH /api/v1/statements/{statement_id}/transactions/{tx_id}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `TransactionUpdatePayload` (`date`, `description`, `debit`, `credit`, `balance`, `category`) - **Response Schema**: `TransactionRowModel` - **Description**: Updates field values, clears historical audit warnings, and recalculates running balances. ### `DELETE /api/v1/statements/{statement_id}/transactions/{tx_id}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `None` - **Description**: Deletes a transaction row and updates subsequent running balances. --- ## 5. Review & HITL Endpoints ### `GET /api/v1/statements/{statement_id}/review-session` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `ReviewSession` (`statementId`, `fileName`, `bankName`, `pdfUrl`, `uploadedAt`, `uploadedBy`) - **Description**: Prepares review parameters, returning document URLs and upload history. ### `GET /api/v1/statements/{statement_id}/review/metrics` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `ReviewMetrics` (`totalTransactions`, `needsAttention`, `lowConfidence`, `balanceIssues`, `gapsDetected`, `overallAccuracy`) - **Description**: Aggregates discrepancy indicators and arithmetic error metrics for the review interface. ### `POST /api/v1/statements/{statement_id}/auto-swap-all` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": True, "swappedCount": int}` - **Description**: Triggers a mathematical audit pass that automatically swaps debit and credit fields on matching rows to correct ledger balance progression. ### `POST /api/v1/statements/{statement_id}/transactions/{tx_id}/swap-debit-credit` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `TransactionRowModel` - **Description**: Forcefully swaps the debit and credit columns of a single transaction and updates balances. ### `POST /api/v1/statements/{statement_id}/pages/{page_num}/review` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": true, "updatedCount": int}` - **Description**: Bulk marks all transactions on a page as manually verified, clearing audit warnings. ### `POST /api/v1/statements/{statement_id}/re-audit` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `ReAuditPayload` (`pageNum`) - **Response Schema**: `{"success": true, "auditResult": {"pageNum": int, "checked": int, "issues": int}}` - **Description**: Triggers a full-statement auto-swap audit and reports remaining issues for the specified page. ### `GET /api/v1/statements/{statement_id}/flagged` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Query Params**: `severity` (str), `bank` (str), `search` (str) - **Response Schema**: `List[FlaggedTransaction]` (`id`, `severity`, `issueType`, `pageNum`, `confidence`, `amount`, `description`, `statementId`) - **Description**: Returns all extraction and mathematical balance issues that require credit officer verification. --- ## 6. Export Endpoints ### `GET /api/v1/statements/{statement_id}/export/summary` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `ExportSummary` - **Description**: Evaluates classification statistics, distinct calendar months, and counts unverified AI classifications before export. ### `POST /api/v1/statements/{statement_id}/export` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `ExportRequest` (`format`: `xlsx`|`csv`|`xlsx_audit`) - **Response Schema**: `ExportResponse` (`downloadUrl`, `fileName`) - **Description**: Triggers export packaging and returns a download link incorporating temporary JWT tokens. ### `GET /api/v1/statements/{statement_id}/download` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Query Params**: `format` (str), `token` (str) - **Response Schema**: Binary file stream - **Description**: Downloads the statement in Excel (`xlsx` / `xlsx_audit`) or `csv` format. Authenticates via standard headers or query parameter token. --- ## 7. Classification Endpoints ### `POST /api/v1/statements/{statement_id}/classify` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"statementId": statement_id}` - **Description**: Starts transaction classification pipeline in a background worker. ### `GET /api/v1/statements/{statement_id}/classify/status` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"status": str, "progress": int, "successCount": int, "failedCount": int}` - **Description**: Retrieves the status of the classification pipeline run. ### `PATCH /api/v1/statements/{statement_id}/classify/{tx_id}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: JSON payload containing `category` (str) and `third_party` (str) - **Response Schema**: `{"success": true}` - **Description**: Overrides a transaction's category classification. ### `POST /api/v1/statements/{statement_id}/classify/{tx_id}/confirm` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": true}` - **Description**: Marks a classification as verified by a broker. ### `POST /api/v1/statements/{statement_id}/classify/confirm-bulk` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: JSON list containing `tx_ids` (List[str]) - **Response Schema**: `{"success": true, "count": int}` - **Description**: Marks multiple classifications as verified. ### `DELETE /api/v1/statements/{statement_id}/classify/{tx_id}/override` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"success": true}` - **Description**: Resets a transaction classification back to its original pipeline-assigned value. ### `POST /api/v1/classify/upload-excel` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: Multipart form data with key `file` (Excel workbook) - **Response Schema**: `{"statementId": statement_id}` - **Description**: Ingests a pre-extracted transaction list from an Excel workbook and starts background classification. --- ## 8. Overwatch Observability Endpoints These read-only endpoints are exposed from the `overwatch_router` component. ### `GET /api/v1/overwatch/events` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Query Params**: `document_id` (str), `document_name` (str), `log_level` (str), `stage` (str), `from_time` (str), `to_time` (str), `page` (int), `page_size` (int) - **Response Schema**: `{"items": List[dict], "total": int, "page": int, "page_size": int}` - **Description**: Queries system observability events from the SQLite WAL store. ### `GET /api/v1/overwatch/documents` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Query Params**: `from_time` (str), `to_time` (str), `has_errors` (bool), `has_warnings` (bool), `username` (str), `page` (int), `page_size` (int) - **Response Schema**: `{"items": List[dict], "total": int, "page": int, "page_size": int}` - **Description**: Lists document-level processing health summaries. ### `GET /api/v1/overwatch/documents/{document_id}/summary` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: Detailed document health metrics, runtime latency, and stage checks. - **Description**: Returns the observability details for a single document. Returns `404` if the document is not found. --- ## 9. History Endpoints ### `GET /api/v1/history/statements` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `List[HistoryRow]` (`statementId`, `fileName`, `status`, `uploadedAt`, `accuracy`, `txCount`, `flaggedCount`) - **Description**: Lists all statements ever uploaded to the system. ### `POST /api/v1/history/statements/{statement_id}/reopen` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `{"status": "success"}` - **Description**: Reopens a finalized statement so that users can make additional edits. ### `POST /api/v1/history/statements/{statement_id}/duplicate` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `UploadInitResponse` - **Description**: Clones an existing statement into a new statement ID to allow testing alternative layout rules. ### `DELETE /api/v1/history/statements/{statement_id}` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `None` - **Description**: Deletes a statement, removes its data folder from disk, and removes associated database audit records. --- ## 10. Administration & Settings Endpoints ### `GET /api/v1/settings` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Response Schema**: `AppSettings` - **Description**: Returns the active settings (`ocrEnabled`, `confidenceThreshold`, `autoExportEnabled`, `extractLast3Months`). ### `PATCH /api/v1/settings` - **Auth Required**: Yes (`tester` | `manager` | `admin`) - **Request Body**: `AppSettings` fields (optional) - **Response Schema**: `AppSettings` - **Description**: Modifies settings fields and flushes them to `data/settings.json`. ### `GET /api/v1/admin/audit-logs` - **Auth Required**: Yes (`admin` only) - **Response Schema**: JSON list of database audit logs - **Description**: Lists system audit log table records for security monitoring. ### `GET /api/v1/admin/overwatch/insights` - **Auth Required**: Yes (`admin` only) - **Response Schema**: Overwatch analytics metrics (overall throughput, error rates, average latency per stage) - **Description**: Returns analytics data on backend pipeline performance. ### `GET /api/v1/admin/uploads/pdfs` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `List[str]` - **Description**: Lists file names stored in the filesystem uploads folder. ### `GET /api/v1/admin/uploads/pdfs-runtime` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `List[str]` - **Description**: Scans the database and reports PDF files active in memory cache. ### `GET /api/v1/admin/overwatch/logs` - **Auth Required**: Yes (`admin` only) - **Response Schema**: Text stream of raw stdout log entries - **Description**: Returns the log output stream. ### `GET /api/v1/admin/overwatch/export` - **Auth Required**: Yes (`admin` only) - **Response Schema**: CSV stream - **Description**: Exports all database audit logs and overwatch logs in a single CSV file. ### `POST /api/v1/admin/overwatch/email-report` - **Auth Required**: Yes (`admin` only) - **Request Body**: `OverwatchEmailRequest` (`email`: comma-separated recipients) - **Response Schema**: Status message (`sent` | `simulated`) - **Description**: Emails system log reports. Generates simulated emails inside `data/simulated_emails/` if no SMTP credentials are configured. ### `GET /api/v1/admin/categories` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `List[CategoryDict]` - **Description**: Lists classification categories in the DB. ### `POST /api/v1/admin/categories` - **Auth Required**: Yes (`admin` only) - **Request Body**: Category creation fields - **Response Schema**: Category dictionary - **Description**: Inserts a classification category into the database. ### `PATCH /api/v1/admin/categories/{category_id}` - **Auth Required**: Yes (`admin` only) - **Request Body**: Category fields - **Response Schema**: Category dictionary - **Description**: Updates category details. ### `DELETE /api/v1/admin/categories/{category_id}` - **Auth Required**: Yes (`admin` only) - **Response Schema**: Status message - **Description**: Deletes a classification category and cascades updates. ### `GET /api/v1/admin/rules` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `List[RuleDict]` - **Description**: Lists all active database classification rules. ### `POST /api/v1/admin/rules` - **Auth Required**: Yes (`admin` only) - **Request Body**: Rule creation fields - **Response Schema**: Rule dictionary - **Description**: Creates a new classification rule. ### `PATCH /api/v1/admin/rules/{rule_id}` - **Auth Required**: Yes (`admin` only) - **Request Body**: Rule fields - **Response Schema**: Rule dictionary - **Description**: Modifies classification rule fields. ### `DELETE /api/v1/admin/rules/{rule_id}` - **Auth Required**: Yes (`admin` only) - **Response Schema**: Status message - **Description**: Deletes a classification rule in the database. ### `GET /api/v1/admin/llm-status` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `{"api_configured": bool, "latency_ms": float}` - **Description**: Tests connectivity to the Google Gemini API. ### `POST /api/v1/admin/reset-pipeline` - **Auth Required**: Yes (`admin` only) - **Response Schema**: `{"status": "reset"}` - **Description**: Clear in-memory caches, database tables, rules, and resets the server state. --- ## 11. Health Endpoints ### `GET /health` - **Auth Required**: No - **Response Schema**: `{"status": "ok", "version": "1.0.0", "database": "connected"}` - **Description**: Endpoint used for cloud container health checks. ### `GET /` - **Auth Required**: No - **Response Schema**: Redirect - **Description**: Redirects root visitors to `/docs` (FastAPI Swagger interface) or returns simple welcome messages depending on deployment environment settings.