kamau1 commited on
Commit
1dedcdf
·
1 Parent(s): 4e97905

Feature: Implement Tende Pay CSV export for ticket expenses with formatter, endpoints, and tests

Browse files
docs/features/tende_pay_csv_export.md ADDED
@@ -0,0 +1,199 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Tende Pay CSV Export Feature
2
+
3
+ ## Overview
4
+
5
+ The ticket expense CSV export has been updated to generate files in **Tende Pay bulk payment format**, compatible with the Tende Pay financial management system.
6
+
7
+ ## What Changed
8
+
9
+ ### Before
10
+ - Custom CSV format with columns: `technician_name`, `phone_number`, `account_name`, `total_amount`, `expense_count`, `date`, `tickets_summary`, `categories_summary`, `expense_ids`
11
+ - Phone numbers included `+` prefix
12
+ - Separate summary fields for tickets and categories
13
+
14
+ ### After
15
+ - **Tende Pay standard format** with columns:
16
+ - `NAME` - Technician or vendor name
17
+ - `ID NUMBER` - User's ID number (from users.id_number)
18
+ - `PHONE NUMBER` - Normalized phone (254XXXXXXXXX, no +)
19
+ - `AMOUNT` - Total payment amount
20
+ - `PAYMENT MODE` - MPESA, BANK, PAYBILL, or BUYGOODS
21
+ - `BANK (Optional)` - Bank name for bank transfers
22
+ - `BANK ACCOUNT NO (Optional)` - Bank account number
23
+ - `PAYBILL BUSINESS NO (Optional)` - Paybill business number
24
+ - `PAYBILL ACCOUNT NO (Optional)` - Paybill account number
25
+ - `BUY GOODS TILL NO (Optional)` - Till number for buy goods
26
+ - `BILL PAYMENT BILLER CODE (Optional)` - Not currently used
27
+ - `BILL PAYMENT ACCOUNT NO (Optional)` - Not currently used
28
+ - `NARRATION (OPTIONAL)` - Single narration field with expense details
29
+
30
+ ## Payment Method Mapping
31
+
32
+ | Our System | Tende Pay Mode | Notes |
33
+ |------------|----------------|-------|
34
+ | `send_money` | `MPESA` | M-Pesa send money |
35
+ | `bank_transfer` | `BANK` | Bank account transfer |
36
+ | `paybill` | `PAYBILL` | M-Pesa paybill |
37
+ | `till_number` | `BUYGOODS` | M-Pesa till number |
38
+ | `pochi_la_biashara` | `MPESA` | Treated as regular M-Pesa |
39
+ | `cash` | ❌ Not supported | Skipped with warning |
40
+
41
+ ## Phone Number Normalization
42
+
43
+ Phone numbers are automatically normalized to Tende Pay format:
44
+ - `+254712345678` → `254712345678` (remove +)
45
+ - `0712345678` → `254712345678` (convert 07 to 254)
46
+ - Spaces and dashes are removed
47
+ - Invalid formats result in warnings and row skipping
48
+
49
+ ## Narration Format
50
+
51
+ Single narration field combining all expense details:
52
+
53
+ ```
54
+ Expenses 2024-12-05: Ticket #ABC123 (Transport 500, Materials 1000) | Ticket #XYZ789 (Meals 300) - 3 items, 1800 KES
55
+ ```
56
+
57
+ - Maximum 200 characters (truncated with "..." if longer)
58
+ - Includes date, ticket references, categories, item count, and total
59
+
60
+ ## Data Validation
61
+
62
+ Before export, each expense group is validated:
63
+ - ✅ Has payment_method
64
+ - ✅ Has payment_details
65
+ - ✅ Payment method is supported (not cash)
66
+ - ⚠️ Has ID number (warns if missing, but doesn't block)
67
+ - ✅ Phone number is valid format (for MPESA)
68
+ - ✅ Bank details complete (for BANK)
69
+ - ✅ Paybill details complete (for PAYBILL)
70
+ - ✅ Till number present (for BUYGOODS)
71
+
72
+ ## Warnings
73
+
74
+ Expenses are skipped with warnings if:
75
+ - Payment method is `cash` (not supported by Tende Pay)
76
+ - Payment details are missing or incomplete
77
+ - Phone number format is invalid
78
+ - User has no ID number (warning only, still exported with empty ID field)
79
+
80
+ Warnings are appended as comments at the end of the CSV file:
81
+ ```csv
82
+ # WARNINGS:
83
+ # User John Doe has no ID number - exported with empty ID field
84
+ # Skipped 2 expense(s) for Jane Smith: No payment details found
85
+ ```
86
+
87
+ ## API Endpoints
88
+
89
+ ### 1. Export by Date Range
90
+ ```
91
+ POST /api/v1/ticket-expenses/export-for-payment
92
+ ?from_date=2024-12-01
93
+ &to_date=2024-12-08
94
+ &project_id=uuid (optional)
95
+ &ticket_id=uuid (optional)
96
+ &user_id=uuid (optional)
97
+ ```
98
+
99
+ ### 2. Bulk Export by IDs
100
+ ```
101
+ POST /api/v1/ticket-expenses/bulk-export
102
+ ?expense_ids=uuid1
103
+ &expense_ids=uuid2
104
+ &expense_ids=uuid3
105
+ ```
106
+
107
+ Both endpoints:
108
+ - Return Tende Pay formatted CSV
109
+ - Mark expenses as paid (irreversible)
110
+ - Set payment_reference to `CSV_EXPORT_{timestamp}_{user_id}`
111
+ - Include warning count in response headers
112
+
113
+ ## File Naming
114
+
115
+ - Date range export: `tende_pay_expenses_2024-12-01_2024-12-08_20241210_143022.csv`
116
+ - Bulk export: `tende_pay_bulk_20241210_143022.csv`
117
+
118
+ ## Implementation Details
119
+
120
+ ### New Service: `TendePayFormatter`
121
+
122
+ Location: `src/app/services/tende_pay_formatter.py`
123
+
124
+ Key methods:
125
+ - `normalize_phone()` - Phone number normalization
126
+ - `validate_payment_details()` - Payment details validation
127
+ - `build_narration()` - Single narration field builder
128
+ - `format_payment_row()` - Format agent payment row
129
+ - `format_vendor_payment_row()` - Format vendor payment row
130
+
131
+ ### Updated Services
132
+
133
+ **`TicketExpenseService.export_for_payment()`**
134
+ - Now uses `TendePayFormatter` for row generation
135
+ - Enhanced validation and error handling
136
+ - Better warning messages
137
+
138
+ ### Updated API Endpoints
139
+
140
+ **`/export-for-payment`** and **`/bulk-export`**
141
+ - Updated CSV column headers to match Tende Pay format
142
+ - Updated filename prefix to `tende_pay_`
143
+
144
+ ## Testing
145
+
146
+ Unit tests: `tests/unit/test_tende_pay_formatter.py`
147
+
148
+ Test coverage:
149
+ - ✅ Phone normalization (all formats)
150
+ - ✅ Payment mode mapping
151
+ - ✅ Payment details validation (all methods)
152
+ - ✅ Narration building and truncation
153
+ - ✅ Agent payment row formatting
154
+ - ✅ Vendor payment row formatting
155
+
156
+ ## Migration Notes
157
+
158
+ ### Breaking Changes
159
+ - CSV format completely changed
160
+ - Old format is no longer supported
161
+ - Any integrations expecting old format will break
162
+
163
+ ### Data Requirements
164
+ - Users should have `id_number` populated (optional but recommended)
165
+ - All expenses should have `payment_method` and `payment_details`
166
+ - Phone numbers should be in valid Kenyan format
167
+
168
+ ### Backward Compatibility
169
+ - None - this is a breaking change
170
+ - Old CSV format is not available
171
+ - Decision: Clean break is better than maintaining two formats
172
+
173
+ ## Example CSV Output
174
+
175
+ ```csv
176
+ NAME,ID NUMBER,PHONE NUMBER,AMOUNT,PAYMENT MODE,BANK (Optional),BANK ACCOUNT NO (Optional),PAYBILL BUSINESS NO (Optional),PAYBILL ACCOUNT NO (Optional),BUY GOODS TILL NO (Optional),BILL PAYMENT BILLER CODE (Optional),BILL PAYMENT ACCOUNT NO (Optional),NARRATION (OPTIONAL)
177
+ John Doe,12345678,254712345678,1500.00,MPESA,,,,,,,,Expenses 2024-12-05: Ticket #ABC123 (Transport 500, Materials 1000) - 2 items, 1500 KES
178
+ Jane Smith,,254722000001,2000.00,BANK,Equity Bank,0123456789,,,,,,"Expenses 2024-12-05: Ticket #XYZ789 (Materials 2000) - 1 items, 2000 KES"
179
+ VENDOR: ABC Hardware,,,5000.00,BUYGOODS,,,,,123456,,,Vendor payment 2024-12-06: Ticket #VENDOR001 - Materials 5000 KES
180
+
181
+ # WARNINGS:
182
+ # User Jane Smith has no ID number - exported with empty ID field
183
+ ```
184
+
185
+ ## Future Enhancements
186
+
187
+ Potential improvements:
188
+ 1. Add `BILL_PAYMENT` support if needed (GOTV, KPLC, etc.)
189
+ 2. Add export format parameter (`?format=tende`) for backward compatibility
190
+ 3. Add dry-run mode to preview export without marking as paid
191
+ 4. Add batch size limits for large exports
192
+ 5. Add export history tracking
193
+ 6. Add Tende Pay API integration for automatic upload
194
+
195
+ ## References
196
+
197
+ - Tende Pay template: `tests/fixtures/tende_bulk_payment_template.csv`
198
+ - Database schema: `supabase/migrations/20251120123500_init_schema.sql`
199
+ - Payment methods: `src/app/schemas/ticket_expense.py`
src/app/api/v1/ticket_expenses.py CHANGED
@@ -1055,7 +1055,9 @@ def bulk_export_expenses(
1055
  key = (expense.incurred_by_user_id, expense.expense_date)
1056
  grouped[key].append(expense)
1057
 
1058
- # Build CSV rows
 
 
1059
  csv_rows = []
1060
  warnings = []
1061
  payment_reference = f"CSV_EXPORT_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}_{current_user.id}"
@@ -1065,26 +1067,41 @@ def bulk_export_expenses(
1065
 
1066
  # Handle vendor payments
1067
  if first_expense.payment_recipient_type == "vendor":
1068
- row = TicketExpenseService._build_vendor_payment_row(first_expense, warnings)
1069
- if row:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1070
  csv_rows.append(row)
 
 
 
 
1071
  continue
1072
 
1073
  # Handle agent payments
1074
  user = first_expense.incurred_by_user
1075
  expense_date = first_expense.expense_date
1076
 
1077
- # Get payment details
1078
- phone_number = None
1079
- account_name = None
1080
-
1081
- if first_expense.payment_details:
1082
- phone_number = first_expense.payment_details.get("phone_number") or \
1083
- first_expense.payment_details.get("account_number")
1084
- account_name = first_expense.payment_details.get("recipient_name") or \
1085
- first_expense.payment_details.get("account_name")
1086
 
1087
- if not phone_number or not account_name:
 
1088
  financial_account = db.query(UserFinancialAccount).filter(
1089
  UserFinancialAccount.user_id == user.id,
1090
  UserFinancialAccount.is_primary == True,
@@ -1094,41 +1111,53 @@ def bulk_export_expenses(
1094
 
1095
  if financial_account:
1096
  if financial_account.payout_method == "mobile_money":
1097
- phone_number = financial_account.mobile_money_phone
1098
- account_name = financial_account.mobile_money_account_name or user.name
 
 
 
1099
  elif financial_account.payout_method == "bank_transfer":
1100
- phone_number = financial_account.bank_account_number
1101
- account_name = financial_account.bank_account_name
 
 
 
 
1102
 
1103
- if not phone_number or not account_name:
 
1104
  warnings.append(f"Skipped {len(group_expenses)} expense(s) for {user.name}: No payment details")
1105
  continue
1106
 
1107
- # Calculate totals
1108
- total_amount = sum(e.total_cost for e in group_expenses)
1109
- expense_count = len(group_expenses)
1110
- expense_ids_str = [str(e.id) for e in group_expenses]
 
 
 
 
 
1111
 
1112
- # Build summaries
1113
  try:
1114
- tickets_summary = TicketExpenseService._build_tickets_summary(group_expenses)
1115
- categories_summary = TicketExpenseService._build_categories_summary(group_expenses)
 
 
 
 
 
 
 
 
 
 
 
 
1116
  except Exception as e:
1117
- logger.error(f"Failed to build summaries for user {user.name}: {str(e)}")
1118
- tickets_summary = f"{len(group_expenses)} expenses"
1119
- categories_summary = f"Total: {float(total_amount)} KES"
1120
-
1121
- csv_rows.append({
1122
- "technician_name": user.name,
1123
- "phone_number": phone_number,
1124
- "account_name": account_name,
1125
- "total_amount": float(total_amount),
1126
- "expense_count": expense_count,
1127
- "date": expense_date.isoformat(),
1128
- "tickets_summary": tickets_summary,
1129
- "categories_summary": categories_summary,
1130
- "expense_ids": ",".join(expense_ids_str)
1131
- })
1132
 
1133
  # Mark as paid
1134
  try:
@@ -1152,12 +1181,24 @@ def bulk_export_expenses(
1152
  detail="Failed to mark expenses as paid"
1153
  )
1154
 
1155
- # Generate CSV
1156
  output = io.StringIO()
1157
  if csv_rows:
 
1158
  fieldnames = [
1159
- "technician_name", "phone_number", "account_name", "total_amount",
1160
- "expense_count", "date", "tickets_summary", "categories_summary", "expense_ids"
 
 
 
 
 
 
 
 
 
 
 
1161
  ]
1162
  writer = csv.DictWriter(output, fieldnames=fieldnames)
1163
  writer.writeheader()
@@ -1169,7 +1210,7 @@ def bulk_export_expenses(
1169
  output.write(f"# {warning}\n")
1170
 
1171
  output.seek(0)
1172
- filename = f"expense_payments_bulk_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.csv"
1173
 
1174
  # Send notification to initiator about export completion
1175
  if csv_rows:
@@ -1278,19 +1319,24 @@ def export_expenses_for_payment(
1278
  user_id=user_id
1279
  )
1280
 
1281
- # Generate CSV
1282
  output = io.StringIO()
1283
  if csv_rows:
 
1284
  fieldnames = [
1285
- "technician_name",
1286
- "phone_number",
1287
- "account_name",
1288
- "total_amount",
1289
- "expense_count",
1290
- "date",
1291
- "tickets_summary",
1292
- "categories_summary",
1293
- "expense_ids"
 
 
 
 
1294
  ]
1295
 
1296
  writer = csv.DictWriter(output, fieldnames=fieldnames)
@@ -1305,7 +1351,7 @@ def export_expenses_for_payment(
1305
 
1306
  # Prepare response
1307
  output.seek(0)
1308
- filename = f"expense_payments_{from_date.isoformat()}_{to_date.isoformat()}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.csv"
1309
 
1310
  return StreamingResponse(
1311
  iter([output.getvalue()]),
 
1055
  key = (expense.incurred_by_user_id, expense.expense_date)
1056
  grouped[key].append(expense)
1057
 
1058
+ # Build Tende Pay CSV rows
1059
+ from app.services.tende_pay_formatter import TendePayFormatter
1060
+
1061
  csv_rows = []
1062
  warnings = []
1063
  payment_reference = f"CSV_EXPORT_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}_{current_user.id}"
 
1067
 
1068
  # Handle vendor payments
1069
  if first_expense.payment_recipient_type == "vendor":
1070
+ try:
1071
+ # Validate payment details
1072
+ is_valid, error_msg = TendePayFormatter.validate_payment_details(
1073
+ payment_method=first_expense.payment_method,
1074
+ payment_details=first_expense.payment_details,
1075
+ user_name=f"Vendor (Expense {first_expense.id})"
1076
+ )
1077
+
1078
+ if not is_valid:
1079
+ warnings.append(error_msg)
1080
+ continue
1081
+
1082
+ # Format vendor row
1083
+ row = TendePayFormatter.format_vendor_payment_row(
1084
+ expense=first_expense,
1085
+ payment_method=first_expense.payment_method,
1086
+ payment_details=first_expense.payment_details
1087
+ )
1088
  csv_rows.append(row)
1089
+
1090
+ except Exception as e:
1091
+ logger.error(f"Failed to format vendor payment row: {str(e)}")
1092
+ warnings.append(f"Skipped vendor expense {first_expense.id}: {str(e)}")
1093
  continue
1094
 
1095
  # Handle agent payments
1096
  user = first_expense.incurred_by_user
1097
  expense_date = first_expense.expense_date
1098
 
1099
+ # Get payment method and details from first expense
1100
+ payment_method = first_expense.payment_method
1101
+ payment_details = first_expense.payment_details
 
 
 
 
 
 
1102
 
1103
+ # If no payment details on expense, try to get from financial account
1104
+ if not payment_method or not payment_details:
1105
  financial_account = db.query(UserFinancialAccount).filter(
1106
  UserFinancialAccount.user_id == user.id,
1107
  UserFinancialAccount.is_primary == True,
 
1111
 
1112
  if financial_account:
1113
  if financial_account.payout_method == "mobile_money":
1114
+ payment_method = "send_money"
1115
+ payment_details = {
1116
+ "phone_number": financial_account.mobile_money_phone,
1117
+ "recipient_name": financial_account.mobile_money_account_name or user.name
1118
+ }
1119
  elif financial_account.payout_method == "bank_transfer":
1120
+ payment_method = "bank_transfer"
1121
+ payment_details = {
1122
+ "bank_name": financial_account.bank_name,
1123
+ "account_number": financial_account.bank_account_number,
1124
+ "account_name": financial_account.bank_account_name
1125
+ }
1126
 
1127
+ # Validate payment details
1128
+ if not payment_method or not payment_details:
1129
  warnings.append(f"Skipped {len(group_expenses)} expense(s) for {user.name}: No payment details")
1130
  continue
1131
 
1132
+ is_valid, error_msg = TendePayFormatter.validate_payment_details(
1133
+ payment_method=payment_method,
1134
+ payment_details=payment_details,
1135
+ user_name=user.name
1136
+ )
1137
+
1138
+ if not is_valid:
1139
+ warnings.append(f"Skipped {len(group_expenses)} expense(s) for {user.name}: {error_msg}")
1140
+ continue
1141
 
1142
+ # Format agent payment row
1143
  try:
1144
+ row = TendePayFormatter.format_payment_row(
1145
+ expenses=group_expenses,
1146
+ user=user,
1147
+ expense_date=expense_date,
1148
+ payment_method=payment_method,
1149
+ payment_details=payment_details,
1150
+ user_id_number=user.id_number
1151
+ )
1152
+ csv_rows.append(row)
1153
+
1154
+ # Warn if user has no ID number
1155
+ if not user.id_number:
1156
+ warnings.append(f"User {user.name} has no ID number - exported with empty ID field")
1157
+
1158
  except Exception as e:
1159
+ logger.error(f"Failed to format payment row for {user.name}: {str(e)}")
1160
+ warnings.append(f"Skipped {len(group_expenses)} expense(s) for {user.name}: {str(e)}")
 
 
 
 
 
 
 
 
 
 
 
 
 
1161
 
1162
  # Mark as paid
1163
  try:
 
1181
  detail="Failed to mark expenses as paid"
1182
  )
1183
 
1184
+ # Generate Tende Pay CSV
1185
  output = io.StringIO()
1186
  if csv_rows:
1187
+ # Tende Pay column order (must match template exactly)
1188
  fieldnames = [
1189
+ "NAME",
1190
+ "ID NUMBER",
1191
+ "PHONE NUMBER",
1192
+ "AMOUNT",
1193
+ "PAYMENT MODE",
1194
+ "BANK (Optional)",
1195
+ "BANK ACCOUNT NO (Optional)",
1196
+ "PAYBILL BUSINESS NO (Optional)",
1197
+ "PAYBILL ACCOUNT NO (Optional)",
1198
+ "BUY GOODS TILL NO (Optional)",
1199
+ "BILL PAYMENT BILLER CODE (Optional)",
1200
+ "BILL PAYMENT ACCOUNT NO (Optional)",
1201
+ "NARRATION (OPTIONAL)"
1202
  ]
1203
  writer = csv.DictWriter(output, fieldnames=fieldnames)
1204
  writer.writeheader()
 
1210
  output.write(f"# {warning}\n")
1211
 
1212
  output.seek(0)
1213
+ filename = f"tende_pay_bulk_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.csv"
1214
 
1215
  # Send notification to initiator about export completion
1216
  if csv_rows:
 
1319
  user_id=user_id
1320
  )
1321
 
1322
+ # Generate Tende Pay CSV
1323
  output = io.StringIO()
1324
  if csv_rows:
1325
+ # Tende Pay column order (must match template exactly)
1326
  fieldnames = [
1327
+ "NAME",
1328
+ "ID NUMBER",
1329
+ "PHONE NUMBER",
1330
+ "AMOUNT",
1331
+ "PAYMENT MODE",
1332
+ "BANK (Optional)",
1333
+ "BANK ACCOUNT NO (Optional)",
1334
+ "PAYBILL BUSINESS NO (Optional)",
1335
+ "PAYBILL ACCOUNT NO (Optional)",
1336
+ "BUY GOODS TILL NO (Optional)",
1337
+ "BILL PAYMENT BILLER CODE (Optional)",
1338
+ "BILL PAYMENT ACCOUNT NO (Optional)",
1339
+ "NARRATION (OPTIONAL)"
1340
  ]
1341
 
1342
  writer = csv.DictWriter(output, fieldnames=fieldnames)
 
1351
 
1352
  # Prepare response
1353
  output.seek(0)
1354
+ filename = f"tende_pay_expenses_{from_date.isoformat()}_{to_date.isoformat()}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.csv"
1355
 
1356
  return StreamingResponse(
1357
  iter([output.getvalue()]),
src/app/services/tende_pay_formatter.py ADDED
@@ -0,0 +1,331 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Tende Pay CSV Formatter
3
+
4
+ Converts expense data to Tende Pay bulk payment CSV format.
5
+ Handles payment method mapping, phone normalization, and narration building.
6
+ """
7
+
8
+ from typing import List, Dict, Optional, Tuple
9
+ from decimal import Decimal
10
+ from datetime import date
11
+ import logging
12
+
13
+ logger = logging.getLogger(__name__)
14
+
15
+
16
+ class TendePayFormatter:
17
+ """
18
+ Formatter for Tende Pay bulk payment CSV exports.
19
+
20
+ Tende Pay supports these payment modes:
21
+ - MPESA: Mobile money (M-Pesa send money)
22
+ - BANK: Bank transfer
23
+ - PAYBILL: M-Pesa paybill
24
+ - BUYGOODS: M-Pesa till number (buy goods)
25
+ - BILL_PAYMENT: Bill payment (GOTV, KPLC, etc.) - not currently used
26
+ """
27
+
28
+ # Payment method mapping: our system -> Tende Pay
29
+ PAYMENT_MODE_MAP = {
30
+ "send_money": "MPESA",
31
+ "bank_transfer": "BANK",
32
+ "paybill": "PAYBILL",
33
+ "till_number": "BUYGOODS",
34
+ "pochi_la_biashara": "MPESA", # Treat as regular M-Pesa
35
+ "cash": None, # Not supported by Tende Pay
36
+ }
37
+
38
+ # Maximum narration length (conservative limit)
39
+ MAX_NARRATION_LENGTH = 200
40
+
41
+ @staticmethod
42
+ def normalize_phone(phone: Optional[str]) -> str:
43
+ """
44
+ Normalize phone number to Tende Pay format (254XXXXXXXXX without +).
45
+
46
+ Args:
47
+ phone: Phone number in various formats (+254..., 254..., 07...)
48
+
49
+ Returns:
50
+ Normalized phone number (254XXXXXXXXX) or empty string
51
+ """
52
+ if not phone:
53
+ return ""
54
+
55
+ # Remove +, spaces, dashes
56
+ normalized = phone.replace("+", "").replace(" ", "").replace("-", "")
57
+
58
+ # Convert 07... to 2547...
59
+ if normalized.startswith("0"):
60
+ normalized = "254" + normalized[1:]
61
+
62
+ # Validate format
63
+ if not normalized.startswith("254") or len(normalized) != 12:
64
+ logger.warning(f"Invalid phone number format: {phone} -> {normalized}")
65
+ return ""
66
+
67
+ return normalized
68
+
69
+ @staticmethod
70
+ def build_narration(
71
+ expenses: List,
72
+ expense_date: date,
73
+ max_length: int = MAX_NARRATION_LENGTH
74
+ ) -> str:
75
+ """
76
+ Build single narration field from multiple expenses.
77
+
78
+ Format: "Expenses 2024-12-05: Ticket #ABC (Transport 500, Materials 1000) | Ticket #XYZ (Meals 300) - 3 items, 1800 KES"
79
+
80
+ Args:
81
+ expenses: List of TicketExpense objects
82
+ expense_date: Date of expenses
83
+ max_length: Maximum narration length
84
+
85
+ Returns:
86
+ Formatted narration string
87
+ """
88
+ from collections import defaultdict
89
+
90
+ # Group by ticket
91
+ by_ticket = defaultdict(list)
92
+ total_amount = Decimal(0)
93
+
94
+ for expense in expenses:
95
+ if expense.ticket:
96
+ ticket_ref = expense.ticket.ticket_name or str(expense.ticket.id)[:8]
97
+ else:
98
+ ticket_ref = "Unknown"
99
+ by_ticket[ticket_ref].append(expense)
100
+ total_amount += expense.total_cost
101
+
102
+ # Build ticket summaries
103
+ ticket_parts = []
104
+ for ticket_ref, ticket_expenses in by_ticket.items():
105
+ expense_details = [
106
+ f"{e.category.title()} {float(e.total_cost)}"
107
+ for e in ticket_expenses
108
+ ]
109
+ ticket_parts.append(f"Ticket #{ticket_ref} ({', '.join(expense_details)})")
110
+
111
+ # Build full narration
112
+ narration = (
113
+ f"Expenses {expense_date.isoformat()}: "
114
+ f"{' | '.join(ticket_parts)} - "
115
+ f"{len(expenses)} items, {float(total_amount)} KES"
116
+ )
117
+
118
+ # Truncate if too long
119
+ if len(narration) > max_length:
120
+ truncated = narration[:max_length - 3] + "..."
121
+ logger.warning(f"Narration truncated from {len(narration)} to {max_length} chars")
122
+ return truncated
123
+
124
+ return narration
125
+
126
+ @staticmethod
127
+ def validate_payment_details(
128
+ payment_method: str,
129
+ payment_details: Optional[Dict],
130
+ user_name: str
131
+ ) -> Tuple[bool, Optional[str]]:
132
+ """
133
+ Validate payment details are complete for the payment method.
134
+
135
+ Args:
136
+ payment_method: Payment method (send_money, bank_transfer, etc.)
137
+ payment_details: Payment details dict
138
+ user_name: User name for error messages
139
+
140
+ Returns:
141
+ Tuple of (is_valid, error_message)
142
+ """
143
+ if not payment_details:
144
+ return False, f"No payment details for {user_name}"
145
+
146
+ if payment_method == "send_money":
147
+ phone = payment_details.get("phone_number")
148
+ if not phone:
149
+ return False, f"Missing phone number for {user_name}"
150
+ if not TendePayFormatter.normalize_phone(phone):
151
+ return False, f"Invalid phone number format for {user_name}: {phone}"
152
+ return True, None
153
+
154
+ elif payment_method == "bank_transfer":
155
+ bank_name = payment_details.get("bank_name")
156
+ account_number = payment_details.get("account_number")
157
+ if not bank_name or not account_number:
158
+ return False, f"Missing bank details for {user_name}"
159
+ return True, None
160
+
161
+ elif payment_method == "paybill":
162
+ business_number = payment_details.get("business_number")
163
+ account_number = payment_details.get("account_number")
164
+ if not business_number or not account_number:
165
+ return False, f"Missing paybill details for {user_name}"
166
+ return True, None
167
+
168
+ elif payment_method == "till_number":
169
+ till_number = payment_details.get("till_number")
170
+ if not till_number:
171
+ return False, f"Missing till number for {user_name}"
172
+ return True, None
173
+
174
+ elif payment_method == "pochi_la_biashara":
175
+ phone = payment_details.get("phone_number")
176
+ if not phone:
177
+ return False, f"Missing phone number for {user_name}"
178
+ if not TendePayFormatter.normalize_phone(phone):
179
+ return False, f"Invalid phone number format for {user_name}: {phone}"
180
+ return True, None
181
+
182
+ elif payment_method == "cash":
183
+ return False, f"Cash payments not supported by Tende Pay for {user_name}"
184
+
185
+ else:
186
+ return False, f"Unknown payment method '{payment_method}' for {user_name}"
187
+
188
+ @staticmethod
189
+ def format_payment_row(
190
+ expenses: List,
191
+ user,
192
+ expense_date: date,
193
+ payment_method: str,
194
+ payment_details: Dict,
195
+ user_id_number: Optional[str] = None
196
+ ) -> Dict[str, str]:
197
+ """
198
+ Format a group of expenses into a Tende Pay CSV row.
199
+
200
+ Args:
201
+ expenses: List of TicketExpense objects (grouped by user+date)
202
+ user: User object (incurred_by_user)
203
+ expense_date: Date of expenses
204
+ payment_method: Payment method (send_money, bank_transfer, etc.)
205
+ payment_details: Payment details dict
206
+ user_id_number: User's ID number (optional)
207
+
208
+ Returns:
209
+ Dict with Tende Pay CSV columns
210
+ """
211
+ # Calculate totals
212
+ total_amount = sum(e.total_cost for e in expenses)
213
+
214
+ # Build narration
215
+ narration = TendePayFormatter.build_narration(expenses, expense_date)
216
+
217
+ # Map payment mode
218
+ payment_mode = TendePayFormatter.PAYMENT_MODE_MAP.get(payment_method)
219
+ if not payment_mode:
220
+ raise ValueError(f"Unsupported payment method: {payment_method}")
221
+
222
+ # Base row structure (all Tende Pay columns)
223
+ row = {
224
+ "NAME": user.name,
225
+ "ID NUMBER": user_id_number or "",
226
+ "PHONE NUMBER": "",
227
+ "AMOUNT": float(total_amount),
228
+ "PAYMENT MODE": payment_mode,
229
+ "BANK (Optional)": "",
230
+ "BANK ACCOUNT NO (Optional)": "",
231
+ "PAYBILL BUSINESS NO (Optional)": "",
232
+ "PAYBILL ACCOUNT NO (Optional)": "",
233
+ "BUY GOODS TILL NO (Optional)": "",
234
+ "BILL PAYMENT BILLER CODE (Optional)": "",
235
+ "BILL PAYMENT ACCOUNT NO (Optional)": "",
236
+ "NARRATION (OPTIONAL)": narration
237
+ }
238
+
239
+ # Fill in method-specific fields
240
+ if payment_mode == "MPESA":
241
+ phone = payment_details.get("phone_number")
242
+ row["PHONE NUMBER"] = TendePayFormatter.normalize_phone(phone)
243
+
244
+ elif payment_mode == "BANK":
245
+ row["BANK (Optional)"] = payment_details.get("bank_name", "")
246
+ row["BANK ACCOUNT NO (Optional)"] = payment_details.get("account_number", "")
247
+
248
+ elif payment_mode == "PAYBILL":
249
+ row["PAYBILL BUSINESS NO (Optional)"] = payment_details.get("business_number", "")
250
+ row["PAYBILL ACCOUNT NO (Optional)"] = payment_details.get("account_number", "")
251
+
252
+ elif payment_mode == "BUYGOODS":
253
+ row["BUY GOODS TILL NO (Optional)"] = payment_details.get("till_number", "")
254
+
255
+ return row
256
+
257
+ @staticmethod
258
+ def format_vendor_payment_row(
259
+ expense,
260
+ payment_method: str,
261
+ payment_details: Dict
262
+ ) -> Dict[str, str]:
263
+ """
264
+ Format a vendor expense into a Tende Pay CSV row.
265
+
266
+ Args:
267
+ expense: TicketExpense object
268
+ payment_method: Payment method
269
+ payment_details: Payment details dict
270
+
271
+ Returns:
272
+ Dict with Tende Pay CSV columns
273
+ """
274
+ # Map payment mode
275
+ payment_mode = TendePayFormatter.PAYMENT_MODE_MAP.get(payment_method)
276
+ if not payment_mode:
277
+ raise ValueError(f"Unsupported payment method: {payment_method}")
278
+
279
+ # Get vendor name
280
+ vendor_name = (
281
+ payment_details.get("business_name") or
282
+ payment_details.get("recipient_name") or
283
+ payment_details.get("account_name") or
284
+ "Unknown Vendor"
285
+ )
286
+
287
+ # Build simple narration for single expense
288
+ if expense.ticket:
289
+ ticket_ref = expense.ticket.ticket_name or str(expense.ticket.id)[:8]
290
+ else:
291
+ ticket_ref = "Unknown"
292
+
293
+ narration = (
294
+ f"Vendor payment {expense.expense_date.isoformat()}: "
295
+ f"Ticket #{ticket_ref} - {expense.category.title()} {float(expense.total_cost)} KES"
296
+ )
297
+
298
+ # Base row structure
299
+ row = {
300
+ "NAME": f"VENDOR: {vendor_name}",
301
+ "ID NUMBER": "", # Vendors don't have ID numbers
302
+ "PHONE NUMBER": "",
303
+ "AMOUNT": float(expense.total_cost),
304
+ "PAYMENT MODE": payment_mode,
305
+ "BANK (Optional)": "",
306
+ "BANK ACCOUNT NO (Optional)": "",
307
+ "PAYBILL BUSINESS NO (Optional)": "",
308
+ "PAYBILL ACCOUNT NO (Optional)": "",
309
+ "BUY GOODS TILL NO (Optional)": "",
310
+ "BILL PAYMENT BILLER CODE (Optional)": "",
311
+ "BILL PAYMENT ACCOUNT NO (Optional)": "",
312
+ "NARRATION (OPTIONAL)": narration
313
+ }
314
+
315
+ # Fill in method-specific fields
316
+ if payment_mode == "MPESA":
317
+ phone = payment_details.get("phone_number")
318
+ row["PHONE NUMBER"] = TendePayFormatter.normalize_phone(phone)
319
+
320
+ elif payment_mode == "BANK":
321
+ row["BANK (Optional)"] = payment_details.get("bank_name", "")
322
+ row["BANK ACCOUNT NO (Optional)"] = payment_details.get("account_number", "")
323
+
324
+ elif payment_mode == "PAYBILL":
325
+ row["PAYBILL BUSINESS NO (Optional)"] = payment_details.get("business_number", "")
326
+ row["PAYBILL ACCOUNT NO (Optional)"] = payment_details.get("account_number", "")
327
+
328
+ elif payment_mode == "BUYGOODS":
329
+ row["BUY GOODS TILL NO (Optional)"] = payment_details.get("till_number", "")
330
+
331
+ return row
src/app/services/ticket_expense_service.py CHANGED
@@ -1028,7 +1028,7 @@ class TicketExpenseService:
1028
  user_id: Optional[UUID] = None
1029
  ) -> Tuple[List[dict], List[str]]:
1030
  """
1031
- Export approved unpaid expenses grouped by user+date for payment processing.
1032
  Marks all exported expenses as paid.
1033
 
1034
  Args:
@@ -1042,7 +1042,7 @@ class TicketExpenseService:
1042
 
1043
  Returns:
1044
  Tuple of (csv_rows, warnings)
1045
- - csv_rows: List of dicts with payment data
1046
  - warnings: List of warning messages
1047
 
1048
  Raises:
@@ -1050,6 +1050,7 @@ class TicketExpenseService:
1050
  """
1051
  from app.models.user_financial_account import UserFinancialAccount
1052
  from app.models.ticket import Ticket
 
1053
  from collections import defaultdict
1054
 
1055
  # Authorization: Only PM, dispatcher, or platform admin
@@ -1099,7 +1100,7 @@ class TicketExpenseService:
1099
  key = (expense.incurred_by_user_id, expense.expense_date)
1100
  grouped[key].append(expense)
1101
 
1102
- # Build CSV rows
1103
  csv_rows = []
1104
  warnings = []
1105
  payment_reference = f"CSV_EXPORT_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}_{current_user.id}"
@@ -1113,30 +1114,41 @@ class TicketExpenseService:
1113
 
1114
  # Handle vendor payments separately
1115
  if first_expense.payment_recipient_type == "vendor":
1116
- row = TicketExpenseService._build_vendor_payment_row(
1117
- expense=first_expense,
1118
- warnings=warnings
1119
- )
1120
- if row:
 
 
 
 
 
 
 
 
 
 
 
 
 
1121
  csv_rows.append(row)
 
 
 
 
1122
  continue
1123
 
1124
  # Handle agent payments
1125
  user = first_expense.incurred_by_user
1126
  expense_date = first_expense.expense_date
1127
 
1128
- # Get payment details from first expense (should be same for all in group)
1129
- phone_number = None
1130
- account_name = None
1131
-
1132
- if first_expense.payment_details:
1133
- phone_number = first_expense.payment_details.get("phone_number") or \
1134
- first_expense.payment_details.get("account_number")
1135
- account_name = first_expense.payment_details.get("recipient_name") or \
1136
- first_expense.payment_details.get("account_name")
1137
 
1138
- # If no payment details, try to get from financial account
1139
- if not phone_number or not account_name:
1140
  financial_account = db.query(UserFinancialAccount).filter(
1141
  UserFinancialAccount.user_id == user.id,
1142
  UserFinancialAccount.is_primary == True,
@@ -1146,43 +1158,58 @@ class TicketExpenseService:
1146
 
1147
  if financial_account:
1148
  if financial_account.payout_method == "mobile_money":
1149
- phone_number = financial_account.mobile_money_phone
1150
- account_name = financial_account.mobile_money_account_name or user.name
 
 
 
1151
  elif financial_account.payout_method == "bank_transfer":
1152
- phone_number = financial_account.bank_account_number
1153
- account_name = financial_account.bank_account_name
 
 
 
 
1154
 
1155
- # Skip if still no payment details
1156
- if not phone_number or not account_name:
1157
  warnings.append(
1158
  f"Skipped {len(group_expenses)} expense(s) for {user.name} on {expense_date}: "
1159
  f"No payment details found"
1160
  )
1161
  continue
1162
 
1163
- # Calculate totals
1164
- total_amount = sum(e.total_cost for e in group_expenses)
1165
- expense_count = len(group_expenses)
1166
- expense_ids = [str(e.id) for e in group_expenses]
1167
-
1168
- # Build tickets summary
1169
- tickets_summary = TicketExpenseService._build_tickets_summary(group_expenses)
1170
 
1171
- # Build categories summary
1172
- categories_summary = TicketExpenseService._build_categories_summary(group_expenses)
 
 
 
1173
 
1174
- # Build CSV row
1175
- csv_rows.append({
1176
- "technician_name": user.name,
1177
- "phone_number": phone_number,
1178
- "account_name": account_name,
1179
- "total_amount": float(total_amount),
1180
- "expense_count": expense_count,
1181
- "date": expense_date.isoformat(),
1182
- "tickets_summary": tickets_summary,
1183
- "categories_summary": categories_summary,
1184
- "expense_ids": ",".join(expense_ids)
1185
- })
 
 
 
 
 
 
 
1186
 
1187
  # Mark all expenses as paid (in transaction)
1188
  try:
@@ -1196,7 +1223,7 @@ class TicketExpenseService:
1196
 
1197
  logger.info(
1198
  f"Exported {len(expenses)} expenses for payment by user {current_user.id}. "
1199
- f"Generated {len(csv_rows)} payment rows. Reference: {payment_reference}"
1200
  )
1201
  except Exception as e:
1202
  db.rollback()
@@ -1207,100 +1234,7 @@ class TicketExpenseService:
1207
  )
1208
 
1209
  return csv_rows, warnings
1210
-
1211
- @staticmethod
1212
- def _build_vendor_payment_row(expense: TicketExpense, warnings: List[str]) -> Optional[dict]:
1213
- """Build CSV row for vendor payment"""
1214
- if not expense.payment_details:
1215
- warnings.append(
1216
- f"Skipped vendor expense {expense.id}: No payment details"
1217
- )
1218
- return None
1219
-
1220
- phone_number = expense.payment_details.get("phone_number") or \
1221
- expense.payment_details.get("till_number") or \
1222
- expense.payment_details.get("account_number")
1223
- account_name = expense.payment_details.get("recipient_name") or \
1224
- expense.payment_details.get("business_name") or \
1225
- expense.payment_details.get("account_name")
1226
-
1227
- if not phone_number or not account_name:
1228
- warnings.append(
1229
- f"Skipped vendor expense {expense.id}: Incomplete payment details"
1230
- )
1231
- return None
1232
-
1233
- # Build ticket summary for single expense
1234
- if expense.ticket:
1235
- ticket_ref = expense.ticket.ticket_name or str(expense.ticket.id)[:8]
1236
- else:
1237
- ticket_ref = "Unknown"
1238
- tickets_summary = f"Ticket {ticket_ref} (1 expense: {expense.category.title()} {float(expense.total_cost)} KES)"
1239
-
1240
- return {
1241
- "technician_name": f"VENDOR: {account_name}",
1242
- "phone_number": phone_number,
1243
- "account_name": account_name,
1244
- "total_amount": float(expense.total_cost),
1245
- "expense_count": 1,
1246
- "date": expense.expense_date.isoformat(),
1247
- "tickets_summary": tickets_summary,
1248
- "categories_summary": f"{expense.category.title()}: {float(expense.total_cost)} (1x)",
1249
- "expense_ids": str(expense.id)
1250
- }
1251
-
1252
- @staticmethod
1253
- def _build_tickets_summary(expenses: List[TicketExpense]) -> str:
1254
- """Build detailed tickets summary for CSV"""
1255
- from collections import defaultdict
1256
-
1257
- # Group by ticket
1258
- by_ticket = defaultdict(list)
1259
- for expense in expenses:
1260
- # Use ticket ID or name as reference
1261
- if expense.ticket:
1262
- ticket_ref = expense.ticket.ticket_name or str(expense.ticket.id)[:8]
1263
- else:
1264
- ticket_ref = "Unknown"
1265
- by_ticket[ticket_ref].append(expense)
1266
-
1267
- # Build summary for each ticket
1268
- ticket_summaries = []
1269
- for ticket_ref, ticket_expenses in by_ticket.items():
1270
- expense_details = []
1271
- ticket_total = Decimal(0)
1272
-
1273
- for expense in ticket_expenses:
1274
- expense_details.append(
1275
- f"{expense.category.title()} {float(expense.total_cost)}"
1276
- )
1277
- ticket_total += expense.total_cost
1278
-
1279
- summary = f"Ticket {ticket_ref} ({len(ticket_expenses)} expenses: {', '.join(expense_details)} = {float(ticket_total)} KES)"
1280
- ticket_summaries.append(summary)
1281
-
1282
- return " | ".join(ticket_summaries)
1283
-
1284
- @staticmethod
1285
- def _build_categories_summary(expenses: List[TicketExpense]) -> str:
1286
- """Build categories summary for CSV"""
1287
- from collections import defaultdict
1288
-
1289
- # Group by category
1290
- by_category = defaultdict(list)
1291
- for expense in expenses:
1292
- by_category[expense.category].append(expense)
1293
-
1294
- # Build summary for each category
1295
- category_summaries = []
1296
- for category, cat_expenses in by_category.items():
1297
- total = sum(e.total_cost for e in cat_expenses)
1298
- count = len(cat_expenses)
1299
- category_summaries.append(
1300
- f"{category.title()}: {float(total)} ({count}x)"
1301
- )
1302
-
1303
- return " | ".join(category_summaries)
1304
 
1305
 
1306
  @staticmethod
 
1028
  user_id: Optional[UUID] = None
1029
  ) -> Tuple[List[dict], List[str]]:
1030
  """
1031
+ Export approved unpaid expenses in Tende Pay CSV format.
1032
  Marks all exported expenses as paid.
1033
 
1034
  Args:
 
1042
 
1043
  Returns:
1044
  Tuple of (csv_rows, warnings)
1045
+ - csv_rows: List of dicts with Tende Pay CSV columns
1046
  - warnings: List of warning messages
1047
 
1048
  Raises:
 
1050
  """
1051
  from app.models.user_financial_account import UserFinancialAccount
1052
  from app.models.ticket import Ticket
1053
+ from app.services.tende_pay_formatter import TendePayFormatter
1054
  from collections import defaultdict
1055
 
1056
  # Authorization: Only PM, dispatcher, or platform admin
 
1100
  key = (expense.incurred_by_user_id, expense.expense_date)
1101
  grouped[key].append(expense)
1102
 
1103
+ # Build Tende Pay CSV rows
1104
  csv_rows = []
1105
  warnings = []
1106
  payment_reference = f"CSV_EXPORT_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}_{current_user.id}"
 
1114
 
1115
  # Handle vendor payments separately
1116
  if first_expense.payment_recipient_type == "vendor":
1117
+ try:
1118
+ # Validate payment details
1119
+ is_valid, error_msg = TendePayFormatter.validate_payment_details(
1120
+ payment_method=first_expense.payment_method,
1121
+ payment_details=first_expense.payment_details,
1122
+ user_name=f"Vendor (Expense {first_expense.id})"
1123
+ )
1124
+
1125
+ if not is_valid:
1126
+ warnings.append(error_msg)
1127
+ continue
1128
+
1129
+ # Format vendor row
1130
+ row = TendePayFormatter.format_vendor_payment_row(
1131
+ expense=first_expense,
1132
+ payment_method=first_expense.payment_method,
1133
+ payment_details=first_expense.payment_details
1134
+ )
1135
  csv_rows.append(row)
1136
+
1137
+ except Exception as e:
1138
+ logger.error(f"Failed to format vendor payment row: {str(e)}")
1139
+ warnings.append(f"Skipped vendor expense {first_expense.id}: {str(e)}")
1140
  continue
1141
 
1142
  # Handle agent payments
1143
  user = first_expense.incurred_by_user
1144
  expense_date = first_expense.expense_date
1145
 
1146
+ # Get payment method and details from first expense
1147
+ payment_method = first_expense.payment_method
1148
+ payment_details = first_expense.payment_details
 
 
 
 
 
 
1149
 
1150
+ # If no payment details on expense, try to get from financial account
1151
+ if not payment_method or not payment_details:
1152
  financial_account = db.query(UserFinancialAccount).filter(
1153
  UserFinancialAccount.user_id == user.id,
1154
  UserFinancialAccount.is_primary == True,
 
1158
 
1159
  if financial_account:
1160
  if financial_account.payout_method == "mobile_money":
1161
+ payment_method = "send_money"
1162
+ payment_details = {
1163
+ "phone_number": financial_account.mobile_money_phone,
1164
+ "recipient_name": financial_account.mobile_money_account_name or user.name
1165
+ }
1166
  elif financial_account.payout_method == "bank_transfer":
1167
+ payment_method = "bank_transfer"
1168
+ payment_details = {
1169
+ "bank_name": financial_account.bank_name,
1170
+ "account_number": financial_account.bank_account_number,
1171
+ "account_name": financial_account.bank_account_name
1172
+ }
1173
 
1174
+ # Validate payment details
1175
+ if not payment_method or not payment_details:
1176
  warnings.append(
1177
  f"Skipped {len(group_expenses)} expense(s) for {user.name} on {expense_date}: "
1178
  f"No payment details found"
1179
  )
1180
  continue
1181
 
1182
+ is_valid, error_msg = TendePayFormatter.validate_payment_details(
1183
+ payment_method=payment_method,
1184
+ payment_details=payment_details,
1185
+ user_name=user.name
1186
+ )
 
 
1187
 
1188
+ if not is_valid:
1189
+ warnings.append(
1190
+ f"Skipped {len(group_expenses)} expense(s) for {user.name} on {expense_date}: {error_msg}"
1191
+ )
1192
+ continue
1193
 
1194
+ # Format agent payment row
1195
+ try:
1196
+ row = TendePayFormatter.format_payment_row(
1197
+ expenses=group_expenses,
1198
+ user=user,
1199
+ expense_date=expense_date,
1200
+ payment_method=payment_method,
1201
+ payment_details=payment_details,
1202
+ user_id_number=user.id_number
1203
+ )
1204
+ csv_rows.append(row)
1205
+
1206
+ # Warn if user has no ID number
1207
+ if not user.id_number:
1208
+ warnings.append(f"User {user.name} has no ID number - exported with empty ID field")
1209
+
1210
+ except Exception as e:
1211
+ logger.error(f"Failed to format payment row for {user.name}: {str(e)}")
1212
+ warnings.append(f"Skipped {len(group_expenses)} expense(s) for {user.name}: {str(e)}")
1213
 
1214
  # Mark all expenses as paid (in transaction)
1215
  try:
 
1223
 
1224
  logger.info(
1225
  f"Exported {len(expenses)} expenses for payment by user {current_user.id}. "
1226
+ f"Generated {len(csv_rows)} Tende Pay payment rows. Reference: {payment_reference}"
1227
  )
1228
  except Exception as e:
1229
  db.rollback()
 
1234
  )
1235
 
1236
  return csv_rows, warnings
1237
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1238
 
1239
 
1240
  @staticmethod
tests/fixtures/tende_bulk_payment_template.csv ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ NAME,ID NUMBER,PHONE NUMBER,AMOUNT,PAYMENT MODE,BANK (Optional),BANK ACCOUNT NO (Optional),PAYBILL BUSINESS NO (Optional),PAYBILL ACCOUNT NO (Optional),BUY GOODS TILL NO (Optional),BILL PAYMENT BILLER CODE (Optional),BILL PAYMENT ACCOUNT NO (Optional),NARRATION (OPTIONAL)
2
+ John Doe,12345678,254722000000,15,BANK,KCB BANK,123456789,,,,,,
3
+ Jane Doe,12345678,254722000001,15,MPESA,,,,,,,,
4
+ John Smith,12345678,254722000002,15,PAYBILL,,,888888,123456789,,,,
5
+ Jane Smith,12345678,254722000003,15,BUYGOODS,,,,,123456789,,,
6
+ Barbra Smith,12345678,254722000004,15,BILL_PAYMENT,,,,,,GOTV,123456789,
7
+ ,,,,,,,,,,,,
8
+ ,,,,,,,,,,,,
9
+ ,,,,,,,,,,,,
10
+ ,,,,,,,,,,,,
tests/unit/test_tende_pay_formatter.py ADDED
@@ -0,0 +1,310 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Unit tests for Tende Pay CSV Formatter
3
+ """
4
+
5
+ import pytest
6
+ from datetime import date
7
+ from decimal import Decimal
8
+ from app.services.tende_pay_formatter import TendePayFormatter
9
+
10
+
11
+ class TestPhoneNormalization:
12
+ """Test phone number normalization"""
13
+
14
+ def test_normalize_with_plus(self):
15
+ """Should remove + prefix"""
16
+ assert TendePayFormatter.normalize_phone("+254712345678") == "254712345678"
17
+
18
+ def test_normalize_without_plus(self):
19
+ """Should keep 254 format as is"""
20
+ assert TendePayFormatter.normalize_phone("254712345678") == "254712345678"
21
+
22
+ def test_normalize_with_zero_prefix(self):
23
+ """Should convert 07... to 2547..."""
24
+ assert TendePayFormatter.normalize_phone("0712345678") == "254712345678"
25
+
26
+ def test_normalize_empty(self):
27
+ """Should return empty string for None"""
28
+ assert TendePayFormatter.normalize_phone(None) == ""
29
+ assert TendePayFormatter.normalize_phone("") == ""
30
+
31
+ def test_normalize_with_spaces(self):
32
+ """Should remove spaces"""
33
+ assert TendePayFormatter.normalize_phone("+254 712 345 678") == "254712345678"
34
+
35
+ def test_normalize_invalid_format(self):
36
+ """Should return empty for invalid format"""
37
+ result = TendePayFormatter.normalize_phone("123456")
38
+ assert result == ""
39
+
40
+
41
+ class TestPaymentModeMapping:
42
+ """Test payment method to Tende Pay mode mapping"""
43
+
44
+ def test_send_money_maps_to_mpesa(self):
45
+ """send_money should map to MPESA"""
46
+ assert TendePayFormatter.PAYMENT_MODE_MAP["send_money"] == "MPESA"
47
+
48
+ def test_bank_transfer_maps_to_bank(self):
49
+ """bank_transfer should map to BANK"""
50
+ assert TendePayFormatter.PAYMENT_MODE_MAP["bank_transfer"] == "BANK"
51
+
52
+ def test_paybill_maps_to_paybill(self):
53
+ """paybill should map to PAYBILL"""
54
+ assert TendePayFormatter.PAYMENT_MODE_MAP["paybill"] == "PAYBILL"
55
+
56
+ def test_till_number_maps_to_buygoods(self):
57
+ """till_number should map to BUYGOODS"""
58
+ assert TendePayFormatter.PAYMENT_MODE_MAP["till_number"] == "BUYGOODS"
59
+
60
+ def test_pochi_maps_to_mpesa(self):
61
+ """pochi_la_biashara should map to MPESA"""
62
+ assert TendePayFormatter.PAYMENT_MODE_MAP["pochi_la_biashara"] == "MPESA"
63
+
64
+ def test_cash_not_supported(self):
65
+ """cash should map to None (not supported)"""
66
+ assert TendePayFormatter.PAYMENT_MODE_MAP["cash"] is None
67
+
68
+
69
+ class TestPaymentDetailsValidation:
70
+ """Test payment details validation"""
71
+
72
+ def test_validate_send_money_valid(self):
73
+ """Valid send_money details should pass"""
74
+ is_valid, error = TendePayFormatter.validate_payment_details(
75
+ payment_method="send_money",
76
+ payment_details={"phone_number": "+254712345678", "recipient_name": "John Doe"},
77
+ user_name="John Doe"
78
+ )
79
+ assert is_valid is True
80
+ assert error is None
81
+
82
+ def test_validate_send_money_missing_phone(self):
83
+ """send_money without phone should fail"""
84
+ is_valid, error = TendePayFormatter.validate_payment_details(
85
+ payment_method="send_money",
86
+ payment_details={"recipient_name": "John Doe"},
87
+ user_name="John Doe"
88
+ )
89
+ assert is_valid is False
90
+ assert "phone number" in error.lower()
91
+
92
+ def test_validate_bank_transfer_valid(self):
93
+ """Valid bank_transfer details should pass"""
94
+ is_valid, error = TendePayFormatter.validate_payment_details(
95
+ payment_method="bank_transfer",
96
+ payment_details={
97
+ "bank_name": "KCB Bank",
98
+ "account_number": "1234567890",
99
+ "account_name": "John Doe"
100
+ },
101
+ user_name="John Doe"
102
+ )
103
+ assert is_valid is True
104
+ assert error is None
105
+
106
+ def test_validate_bank_transfer_missing_details(self):
107
+ """bank_transfer without bank name should fail"""
108
+ is_valid, error = TendePayFormatter.validate_payment_details(
109
+ payment_method="bank_transfer",
110
+ payment_details={"account_number": "1234567890"},
111
+ user_name="John Doe"
112
+ )
113
+ assert is_valid is False
114
+ assert "bank details" in error.lower()
115
+
116
+ def test_validate_paybill_valid(self):
117
+ """Valid paybill details should pass"""
118
+ is_valid, error = TendePayFormatter.validate_payment_details(
119
+ payment_method="paybill",
120
+ payment_details={
121
+ "business_number": "123456",
122
+ "account_number": "789",
123
+ "business_name": "ABC Ltd"
124
+ },
125
+ user_name="ABC Ltd"
126
+ )
127
+ assert is_valid is True
128
+ assert error is None
129
+
130
+ def test_validate_till_number_valid(self):
131
+ """Valid till_number details should pass"""
132
+ is_valid, error = TendePayFormatter.validate_payment_details(
133
+ payment_method="till_number",
134
+ payment_details={"till_number": "123456", "business_name": "ABC Hardware"},
135
+ user_name="ABC Hardware"
136
+ )
137
+ assert is_valid is True
138
+ assert error is None
139
+
140
+ def test_validate_cash_not_supported(self):
141
+ """cash payment should fail"""
142
+ is_valid, error = TendePayFormatter.validate_payment_details(
143
+ payment_method="cash",
144
+ payment_details={"recipient_name": "John Doe"},
145
+ user_name="John Doe"
146
+ )
147
+ assert is_valid is False
148
+ assert "not supported" in error.lower()
149
+
150
+ def test_validate_no_payment_details(self):
151
+ """No payment details should fail"""
152
+ is_valid, error = TendePayFormatter.validate_payment_details(
153
+ payment_method="send_money",
154
+ payment_details=None,
155
+ user_name="John Doe"
156
+ )
157
+ assert is_valid is False
158
+ assert "no payment details" in error.lower()
159
+
160
+
161
+ class TestNarrationBuilding:
162
+ """Test narration field building"""
163
+
164
+ def test_build_narration_single_ticket(self):
165
+ """Should build narration for single ticket"""
166
+ # Mock expense objects
167
+ class MockTicket:
168
+ ticket_name = "ABC123"
169
+
170
+ class MockExpense:
171
+ def __init__(self, category, cost):
172
+ self.category = category
173
+ self.total_cost = Decimal(cost)
174
+ self.ticket = MockTicket()
175
+
176
+ expenses = [
177
+ MockExpense("transport", "500"),
178
+ MockExpense("materials", "1000")
179
+ ]
180
+
181
+ narration = TendePayFormatter.build_narration(
182
+ expenses=expenses,
183
+ expense_date=date(2024, 12, 5)
184
+ )
185
+
186
+ assert "2024-12-05" in narration
187
+ assert "ABC123" in narration
188
+ assert "Transport 500" in narration
189
+ assert "Materials 1000" in narration
190
+ assert "2 items" in narration
191
+ assert "1500" in narration
192
+
193
+ def test_build_narration_truncates_long_text(self):
194
+ """Should truncate narration if too long"""
195
+ class MockTicket:
196
+ ticket_name = "A" * 100
197
+
198
+ class MockExpense:
199
+ def __init__(self):
200
+ self.category = "transport"
201
+ self.total_cost = Decimal("500")
202
+ self.ticket = MockTicket()
203
+
204
+ expenses = [MockExpense() for _ in range(20)]
205
+
206
+ narration = TendePayFormatter.build_narration(
207
+ expenses=expenses,
208
+ expense_date=date(2024, 12, 5),
209
+ max_length=100
210
+ )
211
+
212
+ assert len(narration) <= 100
213
+ assert narration.endswith("...")
214
+
215
+
216
+ class TestFormatPaymentRow:
217
+ """Test formatting payment rows"""
218
+
219
+ def test_format_mpesa_payment(self):
220
+ """Should format MPESA payment row correctly"""
221
+ class MockUser:
222
+ name = "John Doe"
223
+ id_number = "12345678"
224
+
225
+ class MockTicket:
226
+ ticket_name = "ABC123"
227
+
228
+ class MockExpense:
229
+ category = "transport"
230
+ total_cost = Decimal("500")
231
+ ticket = MockTicket()
232
+
233
+ row = TendePayFormatter.format_payment_row(
234
+ expenses=[MockExpense()],
235
+ user=MockUser(),
236
+ expense_date=date(2024, 12, 5),
237
+ payment_method="send_money",
238
+ payment_details={"phone_number": "+254712345678", "recipient_name": "John Doe"},
239
+ user_id_number="12345678"
240
+ )
241
+
242
+ assert row["NAME"] == "John Doe"
243
+ assert row["ID NUMBER"] == "12345678"
244
+ assert row["PHONE NUMBER"] == "254712345678" # No + prefix
245
+ assert row["AMOUNT"] == 500.0
246
+ assert row["PAYMENT MODE"] == "MPESA"
247
+ assert "2024-12-05" in row["NARRATION (OPTIONAL)"]
248
+
249
+ def test_format_bank_payment(self):
250
+ """Should format BANK payment row correctly"""
251
+ class MockUser:
252
+ name = "Jane Smith"
253
+ id_number = None # No ID number
254
+
255
+ class MockTicket:
256
+ ticket_name = "XYZ789"
257
+
258
+ class MockExpense:
259
+ category = "materials"
260
+ total_cost = Decimal("2000")
261
+ ticket = MockTicket()
262
+
263
+ row = TendePayFormatter.format_payment_row(
264
+ expenses=[MockExpense()],
265
+ user=MockUser(),
266
+ expense_date=date(2024, 12, 6),
267
+ payment_method="bank_transfer",
268
+ payment_details={
269
+ "bank_name": "Equity Bank",
270
+ "account_number": "0123456789",
271
+ "account_name": "Jane Smith"
272
+ },
273
+ user_id_number=None
274
+ )
275
+
276
+ assert row["NAME"] == "Jane Smith"
277
+ assert row["ID NUMBER"] == "" # Empty for missing ID
278
+ assert row["PHONE NUMBER"] == "" # Not needed for bank
279
+ assert row["AMOUNT"] == 2000.0
280
+ assert row["PAYMENT MODE"] == "BANK"
281
+ assert row["BANK (Optional)"] == "Equity Bank"
282
+ assert row["BANK ACCOUNT NO (Optional)"] == "0123456789"
283
+
284
+
285
+ class TestFormatVendorPaymentRow:
286
+ """Test formatting vendor payment rows"""
287
+
288
+ def test_format_vendor_till_payment(self):
289
+ """Should format vendor till number payment"""
290
+ class MockTicket:
291
+ ticket_name = "VENDOR001"
292
+ id = "abc-123"
293
+
294
+ class MockExpense:
295
+ category = "materials"
296
+ total_cost = Decimal("5000")
297
+ expense_date = date(2024, 12, 7)
298
+ ticket = MockTicket()
299
+
300
+ row = TendePayFormatter.format_vendor_payment_row(
301
+ expense=MockExpense(),
302
+ payment_method="till_number",
303
+ payment_details={"till_number": "123456", "business_name": "ABC Hardware"}
304
+ )
305
+
306
+ assert row["NAME"] == "VENDOR: ABC Hardware"
307
+ assert row["ID NUMBER"] == "" # Vendors don't have ID
308
+ assert row["PAYMENT MODE"] == "BUYGOODS"
309
+ assert row["BUY GOODS TILL NO (Optional)"] == "123456"
310
+ assert row["AMOUNT"] == 5000.0