bank-statement-api / docs /API_REFERENCE.md
3v324v23's picture
Merge Conflict solved | Build error solved
205f25f
|
Raw History Blame Contribute Delete
20.8 kB
# BeFree API Reference
This document provides a complete inventory of the REST API endpoints exposed by the BeFree backend server, including authentication requirements, request payloads, and response structures.
---
## Global Authentication Rules
- **Bearer Tokens**: Endpoints requiring authentication look for a JWT token in the `Authorization: Bearer <token>` header.
- **Roles**:
- `tester` (default): View dashboards, upload and review statements.
- `manager`: Access dashboard metrics, review history, and trigger exports.
- `admin`: Full access, including rule manipulation, audit logs, categories, database resets, and administrative commands.
---
## 1. Authentication Endpoints
### `POST /api/v1/auth/login`
- **Auth Required**: No
- **Request Body**: Form parameters (`username`, `password`)
- **Response Schema**: `TokenResponse` (`access_token`, `token_type`, `role`, `username`)
- **Description**: Verifies credentials and returns a signed JWT access token.
### `POST /api/v1/auth/logout`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: None (retrieves token from authorization header)
- **Response Schema**: `{"status": "success", "message": "Logged out"}`
- **Description**: Revokes the active session token in the database.
### `GET /api/v1/auth/me`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `UserProfile` (`id`, `username`, `email`, `fullName`, `role`)
- **Description**: Returns the profile details of the currently authenticated user.
### `POST /api/v1/auth/seed-users`
- **Auth Required**: Yes (requires `X-Seed-Secret` header matching `SEED_SECRET` env var)
- **Response Schema**: `{"status": "success", "seeded_count": int}`
- **Description**: Seeds initial system users (`admin`, `manager`, `tester`) into the PostgreSQL database.
---
## 2. Dashboard Endpoints
### `GET /api/v1/dashboard/metrics`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `DashboardMetrics` (`totalStatements`, `processedSuccessfully`, `failedStatements`, `averageAccuracy`, `totalFlaggedTransactions`)
- **Description**: Aggregates high-level processing statistics for the main dashboard display.
### `GET /api/v1/dashboard/recent-activity`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `List[ActivityItem]` (`id`, `timestamp`, `username`, `action`, `description`)
- **Description**: Lists recent database audit log activities.
### `GET /api/v1/dashboard/recent-statements`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `List[StatementSummary]` (`id`, `fileName`, `status`, `uploadedAt`, `accuracy`)
- **Description**: Lists recently processed statement metadata.
---
## 3. Statement Processing Endpoints
### `POST /api/v1/statements/upload`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: Multipart form data with key `files` (array of PDF files)
- **Response Schema**: `UploadInitResponse` (`statementId`, `fileName`, `status`, `ocrFallbackTriggered`)
- **Description**: Ingests PDF files, initializes the transaction database, and caches the raw PDFs on disk.
### `POST /api/v1/statements/{statement_id}/detect-layout`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `HeaderDetectionResultModel`
- **Description**: Analyzes pages in the statement to automatically detect header coordinates and column mappings.
### `GET /api/v1/statements/{statement_id}/layout`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `HeaderDetectionResultModel`
- **Description**: Retrieves the cached or detected column layout map.
### `GET /api/v1/statements/{statement_id}/page-image/{page_num}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: PNG binary stream
- **Description**: Generates and serves a rasterized PNG image of a statement page (used for visual verification in the coordinate inspector).
### `POST /api/v1/statements/{statement_id}/verify-layout`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `HeaderDetectionResultModel`
- **Response Schema**: `{"success": true}`
- **Description**: Saves layout coordinates adjusted manually by the user in the UI coordinate builder.
### `PATCH /api/v1/statements/{statement_id}/layout/column/{col_name}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: JSON mapping for x-coordinates (`x_min`, `x_max`)
- **Response Schema**: `{"success": true}`
- **Description**: Modifies coordinate ranges for a single column.
### `POST /api/v1/statements/{statement_id}/process`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": true, "message": "Extraction started"}`
- **Description**: Initiates transaction cell extraction and audits in a background thread.
### `POST /api/v1/statements/{statement_id}/finalise`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": true}`
- **Description**: Finalizes the statement review and generates the default spreadsheet.
### `GET /api/v1/statements/{statement_id}/processing`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `ProcessingState`
- **Description**: Polls the background thread extraction progress, stages completed, and remaining duration.
### `POST /api/v1/statements/{statement_id}/processing/cancel`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `None`
- **Description**: Cancels background processing for a statement and sets its status to `cancelled`.
### `POST /api/v1/statements/{statement_id}/save-progress`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"status": "success", "message": "Progress saved"}`
- **Description**: Forces writing the active transaction and correction log states to the excel cached workbook.
### `GET /api/v1/statements/{statement_id}/pdf`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: PDF binary stream
- **Description**: Downloads the raw uploaded PDF file.
---
## 4. Transaction Endpoints
### `GET /api/v1/statements/{statement_id}/transactions`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Query Params**: `page` (int), `pageSize` (int), `search` (str), `status` (str), `pageNum` (int), `confidence` (str: `high`|`low`)
- **Response Schema**: `TransactionsPage` (`items`: `List[TransactionRowModel]`, `total`: int, `page`: int, `pageSize`: int)
- **Description**: Retrieves a paginated list of transactions, supporting search queries, status filters, page limits, and date windowing configurations.
### `POST /api/v1/statements/{statement_id}/transactions`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `TransactionCreatePayload` (`pageNumber`, `date`, `description`, `debit`, `credit`, `balance`)
- **Response Schema**: `TransactionRowModel`
- **Description**: Inserts a manual transaction row, triggers a statement re-audit, and persists the cache.
### `GET /api/v1/statements/{statement_id}/transactions/{tx_id}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `TransactionRowModel`
- **Description**: Returns a single transaction details.
### `PATCH /api/v1/statements/{statement_id}/transactions/{tx_id}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `TransactionUpdatePayload` (`date`, `description`, `debit`, `credit`, `balance`, `category`)
- **Response Schema**: `TransactionRowModel`
- **Description**: Updates field values, clears historical audit warnings, and recalculates running balances.
### `DELETE /api/v1/statements/{statement_id}/transactions/{tx_id}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `None`
- **Description**: Deletes a transaction row and updates subsequent running balances.
---
## 5. Review & HITL Endpoints
### `GET /api/v1/statements/{statement_id}/review-session`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `ReviewSession` (`statementId`, `fileName`, `bankName`, `pdfUrl`, `uploadedAt`, `uploadedBy`)
- **Description**: Prepares review parameters, returning document URLs and upload history.
### `GET /api/v1/statements/{statement_id}/review/metrics`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `ReviewMetrics` (`totalTransactions`, `needsAttention`, `lowConfidence`, `balanceIssues`, `gapsDetected`, `overallAccuracy`)
- **Description**: Aggregates discrepancy indicators and arithmetic error metrics for the review interface.
### `POST /api/v1/statements/{statement_id}/auto-swap-all`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": True, "swappedCount": int}`
- **Description**: Triggers a mathematical audit pass that automatically swaps debit and credit fields on matching rows to correct ledger balance progression.
### `POST /api/v1/statements/{statement_id}/transactions/{tx_id}/swap-debit-credit`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `TransactionRowModel`
- **Description**: Forcefully swaps the debit and credit columns of a single transaction and updates balances.
### `POST /api/v1/statements/{statement_id}/pages/{page_num}/review`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": true, "updatedCount": int}`
- **Description**: Bulk marks all transactions on a page as manually verified, clearing audit warnings.
### `POST /api/v1/statements/{statement_id}/re-audit`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `ReAuditPayload` (`pageNum`)
- **Response Schema**: `{"success": true, "auditResult": {"pageNum": int, "checked": int, "issues": int}}`
- **Description**: Triggers a full-statement auto-swap audit and reports remaining issues for the specified page.
### `GET /api/v1/statements/{statement_id}/flagged`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Query Params**: `severity` (str), `bank` (str), `search` (str)
- **Response Schema**: `List[FlaggedTransaction]` (`id`, `severity`, `issueType`, `pageNum`, `confidence`, `amount`, `description`, `statementId`)
- **Description**: Returns all extraction and mathematical balance issues that require credit officer verification.
---
## 6. Export Endpoints
### `GET /api/v1/statements/{statement_id}/export/summary`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `ExportSummary`
- **Description**: Evaluates classification statistics, distinct calendar months, and counts unverified AI classifications before export.
### `POST /api/v1/statements/{statement_id}/export`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `ExportRequest` (`format`: `xlsx`|`csv`|`xlsx_audit`)
- **Response Schema**: `ExportResponse` (`downloadUrl`, `fileName`)
- **Description**: Triggers export packaging and returns a download link incorporating temporary JWT tokens.
### `GET /api/v1/statements/{statement_id}/download`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Query Params**: `format` (str), `token` (str)
- **Response Schema**: Binary file stream
- **Description**: Downloads the statement in Excel (`xlsx` / `xlsx_audit`) or `csv` format. Authenticates via standard headers or query parameter token.
---
## 7. Classification Endpoints
### `POST /api/v1/statements/{statement_id}/classify`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"statementId": statement_id}`
- **Description**: Starts transaction classification pipeline in a background worker.
### `GET /api/v1/statements/{statement_id}/classify/status`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"status": str, "progress": int, "successCount": int, "failedCount": int}`
- **Description**: Retrieves the status of the classification pipeline run.
### `PATCH /api/v1/statements/{statement_id}/classify/{tx_id}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: JSON payload containing `category` (str) and `third_party` (str)
- **Response Schema**: `{"success": true}`
- **Description**: Overrides a transaction's category classification.
### `POST /api/v1/statements/{statement_id}/classify/{tx_id}/confirm`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": true}`
- **Description**: Marks a classification as verified by a broker.
### `POST /api/v1/statements/{statement_id}/classify/confirm-bulk`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: JSON list containing `tx_ids` (List[str])
- **Response Schema**: `{"success": true, "count": int}`
- **Description**: Marks multiple classifications as verified.
### `DELETE /api/v1/statements/{statement_id}/classify/{tx_id}/override`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"success": true}`
- **Description**: Resets a transaction classification back to its original pipeline-assigned value.
### `POST /api/v1/classify/upload-excel`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: Multipart form data with key `file` (Excel workbook)
- **Response Schema**: `{"statementId": statement_id}`
- **Description**: Ingests a pre-extracted transaction list from an Excel workbook and starts background classification.
---
## 8. Overwatch Observability Endpoints
These read-only endpoints are exposed from the `overwatch_router` component.
### `GET /api/v1/overwatch/events`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Query Params**: `document_id` (str), `document_name` (str), `log_level` (str), `stage` (str), `from_time` (str), `to_time` (str), `page` (int), `page_size` (int)
- **Response Schema**: `{"items": List[dict], "total": int, "page": int, "page_size": int}`
- **Description**: Queries system observability events from the SQLite WAL store.
### `GET /api/v1/overwatch/documents`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Query Params**: `from_time` (str), `to_time` (str), `has_errors` (bool), `has_warnings` (bool), `username` (str), `page` (int), `page_size` (int)
- **Response Schema**: `{"items": List[dict], "total": int, "page": int, "page_size": int}`
- **Description**: Lists document-level processing health summaries.
### `GET /api/v1/overwatch/documents/{document_id}/summary`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: Detailed document health metrics, runtime latency, and stage checks.
- **Description**: Returns the observability details for a single document. Returns `404` if the document is not found.
---
## 9. History Endpoints
### `GET /api/v1/history/statements`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `List[HistoryRow]` (`statementId`, `fileName`, `status`, `uploadedAt`, `accuracy`, `txCount`, `flaggedCount`)
- **Description**: Lists all statements ever uploaded to the system.
### `POST /api/v1/history/statements/{statement_id}/reopen`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `{"status": "success"}`
- **Description**: Reopens a finalized statement so that users can make additional edits.
### `POST /api/v1/history/statements/{statement_id}/duplicate`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `UploadInitResponse`
- **Description**: Clones an existing statement into a new statement ID to allow testing alternative layout rules.
### `DELETE /api/v1/history/statements/{statement_id}`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `None`
- **Description**: Deletes a statement, removes its data folder from disk, and removes associated database audit records.
---
## 10. Administration & Settings Endpoints
### `GET /api/v1/settings`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Response Schema**: `AppSettings`
- **Description**: Returns the active settings (`ocrEnabled`, `confidenceThreshold`, `autoExportEnabled`, `extractLast3Months`).
### `PATCH /api/v1/settings`
- **Auth Required**: Yes (`tester` | `manager` | `admin`)
- **Request Body**: `AppSettings` fields (optional)
- **Response Schema**: `AppSettings`
- **Description**: Modifies settings fields and flushes them to `data/settings.json`.
### `GET /api/v1/admin/audit-logs`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: JSON list of database audit logs
- **Description**: Lists system audit log table records for security monitoring.
### `GET /api/v1/admin/overwatch/insights`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: Overwatch analytics metrics (overall throughput, error rates, average latency per stage)
- **Description**: Returns analytics data on backend pipeline performance.
### `GET /api/v1/admin/uploads/pdfs`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `List[str]`
- **Description**: Lists file names stored in the filesystem uploads folder.
### `GET /api/v1/admin/uploads/pdfs-runtime`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `List[str]`
- **Description**: Scans the database and reports PDF files active in memory cache.
### `GET /api/v1/admin/overwatch/logs`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: Text stream of raw stdout log entries
- **Description**: Returns the log output stream.
### `GET /api/v1/admin/overwatch/export`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: CSV stream
- **Description**: Exports all database audit logs and overwatch logs in a single CSV file.
### `POST /api/v1/admin/overwatch/email-report`
- **Auth Required**: Yes (`admin` only)
- **Request Body**: `OverwatchEmailRequest` (`email`: comma-separated recipients)
- **Response Schema**: Status message (`sent` | `simulated`)
- **Description**: Emails system log reports. Generates simulated emails inside `data/simulated_emails/` if no SMTP credentials are configured.
### `GET /api/v1/admin/categories`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `List[CategoryDict]`
- **Description**: Lists classification categories in the DB.
### `POST /api/v1/admin/categories`
- **Auth Required**: Yes (`admin` only)
- **Request Body**: Category creation fields
- **Response Schema**: Category dictionary
- **Description**: Inserts a classification category into the database.
### `PATCH /api/v1/admin/categories/{category_id}`
- **Auth Required**: Yes (`admin` only)
- **Request Body**: Category fields
- **Response Schema**: Category dictionary
- **Description**: Updates category details.
### `DELETE /api/v1/admin/categories/{category_id}`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: Status message
- **Description**: Deletes a classification category and cascades updates.
### `GET /api/v1/admin/rules`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `List[RuleDict]`
- **Description**: Lists all active database classification rules.
### `POST /api/v1/admin/rules`
- **Auth Required**: Yes (`admin` only)
- **Request Body**: Rule creation fields
- **Response Schema**: Rule dictionary
- **Description**: Creates a new classification rule.
### `PATCH /api/v1/admin/rules/{rule_id}`
- **Auth Required**: Yes (`admin` only)
- **Request Body**: Rule fields
- **Response Schema**: Rule dictionary
- **Description**: Modifies classification rule fields.
### `DELETE /api/v1/admin/rules/{rule_id}`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: Status message
- **Description**: Deletes a classification rule in the database.
### `GET /api/v1/admin/llm-status`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `{"api_configured": bool, "latency_ms": float}`
- **Description**: Tests connectivity to the Google Gemini API.
### `POST /api/v1/admin/reset-pipeline`
- **Auth Required**: Yes (`admin` only)
- **Response Schema**: `{"status": "reset"}`
- **Description**: Clear in-memory caches, database tables, rules, and resets the server state.
---
## 11. Health Endpoints
### `GET /health`
- **Auth Required**: No
- **Response Schema**: `{"status": "ok", "version": "1.0.0", "database": "connected"}`
- **Description**: Endpoint used for cloud container health checks.
### `GET /`
- **Auth Required**: No
- **Response Schema**: Redirect
- **Description**: Redirects root visitors to `/docs` (FastAPI Swagger interface) or returns simple welcome messages depending on deployment environment settings.