Spaces:
Sleeping
Sleeping
File size: 6,890 Bytes
fbb917f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 | # BeFree Developer Contribution Guide
This guide is designed to onboard new developers and interns onto the BeFree Bank Statement Processor project, detailing environment setups, codebase architecture tours, feature development patterns, and type checking.
---
## 1. Local Developer Setup
Follow these steps to configure your environment and run BeFree locally.
### 1.1 Prerequisites
- **Python 3.11+** installed.
- **Node.js 18+** and **npm** installed.
- **Git** installed.
- **Sibling Repositories**: The internal editable packages (`bank-extractor` and `bank-hitl`) must be cloned in directories alongside this repository.
### 1.2 Copy Environment Configuration
```bash
cp .env.example .env
```
Open `.env` and fill in:
- `GOOGLE_API_KEY`: Get a Gemini developer key from Google AI Studio.
- `DATABASE_URL`: Setup a free tier PostgreSQL database on NeonDB and input the URL (ensure `?sslmode=require` is appended).
### 1.3 Setup Python Virtual Environment
```bash
# Create virtual environment
python3 -m venv venv
# Activate virtual environment
source venv/bin/activate
# Install requirements (which installs local packages in editable mode)
pip install -r requirements.txt
```
### 1.4 Database Migrations & Seeding
```bash
# Run schema migrations
python auth/migrations.py
# Seed default employee users
python auth/seed_users.py
```
### 1.5 Run Backend
```bash
python api_server.py
# The backend will start on http://localhost:8000
# OpenAPI interactive Swagger docs are available at http://localhost:8000/docs
```
### 1.6 Run Frontend
```bash
cd befree-frontend
npm install
npm run dev
# The frontend will start on http://localhost:3000
```
### 1.7 Run Both Simultaneously
Alternatively, launch both backend and frontend servers with one command:
```bash
./run.sh
```
---
## 2. Project Layout Tour
- [api_server.py](file:///home/luv/Desktop/new/api_server.py) β The API server entry point that instantiates the FastAPI app, configures CORS/middlewares, and mounts the feature-domain sub-routers.
- [models/](file:///home/luv/Desktop/new/models) β Pydantic request/response payload schemas grouped by domain (e.g., `statements.py`, `transactions.py`).
- [core/](file:///home/luv/Desktop/new/core) β Core business logic, pipeline initialization, statement IO, and global HTTP middlewares.
- [routers/](file:///home/luv/Desktop/new/routers) β Modular FastAPI APIRouter endpoints split by feature domain (e.g., `upload.py`, `dashboard.py`).
- [auth/](file:///home/luv/Desktop/new/auth) β Database schemas and services for user session validation, JWT logic, and classification rules storage.
- [overwatch/](file:///home/luv/Desktop/new/overwatch) β Observable stage log database store, event definitions, and context managers.
- [shared/](file:///home/luv/Desktop/new/shared) β Shared Pydantic data schemas and constants.
- [bank-classifier/](file:///home/luv/Desktop/new/bank-classifier) β Sequential 3-stage classification engine logic, Gemini connections, and rule promotion learners.
- [befree-frontend/](file:///home/luv/Desktop/new/befree-frontend) β Next.js 15 frontend application.
- `befree-frontend/app/` β Routing structure (pages).
- `befree-frontend/features/` β Feature UI components (e.g. `overwatch`, `dashboard`, `coordinate-editor`).
- `befree-frontend/services/api-client.ts` β Central Axios API caller class.
- [scripts/](file:///home/luv/Desktop/new/scripts) β Diagnostic, data migration, and comparative verification utility scripts.
- [data/](file:///home/luv/Desktop/new/data) β Local runtime file folders (uploads, output, logs) ignored by git.
---
## 3. Adding a New API Endpoint
Backend routes are modularized inside the `routers/` directory using FastAPI `APIRouter` objects. To declare a new route:
1. Locate the appropriate router file in `routers/` (e.g., `routers/review.py` or `routers/transactions.py`).
2. Add a route using the `@router` decorator, specifying request/response validation models from `models/` and authentication dependencies:
```python
@router.post("/new-feature", response_model=MyResponseModel)
async def handle_new_feature(
payload: MyRequestPayload,
current_user: User = Depends(get_current_user) # Enforces JWT validation
):
# Ensure roles are verified if admin-only:
# if current_user.role != "admin":
# raise HTTPException(status_code=403, detail="Admin only")
# Business logic here
return {"status": "success"}
```
3. If creating a new router file, define the `router = APIRouter(...)` and mount it in `api_server.py` using `app.include_router(new_router)`.
---
## 4. Adding a New Classification Rule
New classification rules can be added in two ways:
- **Admin Interface**: Log in as an admin role, navigate to the Rules settings dashboard, and add keywords. This calls `POST /api/v1/admin/rules`.
- **Database Rules Seeding**: To add standard rules into the system database:
1. Open `bank-classifier/bank_classifier/rules/default_rules.yaml`.
2. Add your category keyword configuration.
3. Execute `python scripts/migrate_rules_to_db.py` to idempotently upsert the definitions.
---
## 5. Adding a New Frontend Feature
The frontend organizes code by features inside `befree-frontend/features/`.
1. **Component Design**: Place your components inside `befree-frontend/features/my-feature/`.
2. **API Client Integration**: Declare backend fetch calls in `befree-frontend/services/api-client.ts` as client methods.
3. **Zustand Store / Custom Hooks**: Create local stores or query hooks inside `befree-frontend/hooks/` to retrieve data.
4. **App View**: Include the feature component in a route inside `befree-frontend/app/`.
---
## 6. Running the Classification Pipeline Standalone
You can invoke the classification pipeline directly from a Python script without booting the FastAPI web server:
```python
from dotenv import load_dotenv
load_dotenv()
from bank_classifier.pipeline import ClassificationPipeline
from shared.models import NormTransaction
# Initialize pipeline
pipeline = ClassificationPipeline()
# Prepare mock transaction
tx = NormTransaction(
page_num=1,
row_num=1,
date_raw="24/05/2026",
desc_raw="Netflix subscription",
debit_raw="18.99",
amount=-18.99
)
# Run classification batch
results = pipeline.classify_batch([tx])
print(results[0].category) # Output: Recreation & Entertainment
```
---
## 7. Linting and Type Checking
- **Frontend (TypeScript)**: Run static TypeScript compiler checks before submitting code:
```bash
cd befree-frontend
npm run type-check
```
- **Backend (Python)**: The project currently does not package standard linters. It is highly recommended to install `ruff` for code linting and formatting:
```bash
pip install ruff
# Lint code
ruff check .
# Format code
ruff format .
```
|