viraj.kothari commited on
Commit
58b74a0
Β·
1 Parent(s): 5b651ee

fix: rename agent folders to remove spaces

Browse files
eod_review_agent/README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # End-of-Day Review Agent
2
+
3
+ Fires every day at 6:30 PM. Pulls everything from your day β€”
4
+ meetings, emails sent/received, tasks done vs pending, tomorrow's
5
+ calendar β€” and generates a structured EOD review + shutdown checklist.
6
+
7
+ Delivers via Email + WhatsApp + Notion.
8
+
9
+ ## File structure
10
+
11
+ ```
12
+ eod_review_agent/
13
+ β”œβ”€β”€ eod_review.py ← Main entry point
14
+ β”œβ”€β”€ data_fetcher.py ← Pulls all 4 data sources
15
+ β”œβ”€β”€ review_llm.py ← Groq call + system prompt
16
+ β”œβ”€β”€ delivery.py ← Email / WhatsApp / Notion / file
17
+ β”œβ”€β”€ tasks.txt ← Edit daily if not using Notion tasks
18
+ β”œβ”€β”€ requirements.txt
19
+ └── .env.example
20
+ ```
21
+
22
+ ## Quick start
23
+
24
+ ```bash
25
+ # 1. Install
26
+ pip install -r requirements.txt
27
+
28
+ # 2. Configure
29
+ cp .env.example .env
30
+ # Add GROQ_API_KEY at minimum
31
+
32
+ # 3. Copy Google credentials from Daily Planner Agent
33
+ cp ../daily_planner_agent/credentials.json .
34
+ cp ../daily_planner_agent/token.json .
35
+
36
+ # 4. Add today's tasks to tasks.txt (or connect Notion)
37
+ nano tasks.txt
38
+
39
+ # 5. Test immediately
40
+ python eod_review.py
41
+
42
+ # 6. Run on schedule
43
+ python eod_review.py --schedule
44
+ ```
45
+
46
+ ## Tasks: two options
47
+
48
+ **Option A β€” Simple (tasks.txt):**
49
+ Edit `tasks.txt` each morning, mark `[done]` as you complete things.
50
+ The agent reads it at 6:30 PM.
51
+
52
+ **Option B β€” Notion database:**
53
+ Set `NOTION_TASKS_DB_ID` in `.env`. Your Notion database needs:
54
+ - A `Name` (title) property
55
+ - A `Status` (checkbox or select) property
56
+ - A `Due` or `Created` date property
57
+
58
+ ## Sample output
59
+
60
+ ```
61
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
62
+ END OF DAY β€” Wednesday, 14 May 2026
63
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
64
+
65
+ TODAY AT A GLANCE
66
+ πŸ—“ Meetings : 3 meetings, ~2 hrs
67
+ πŸ“§ Emails : 8 received, 3 sent
68
+ βœ… Tasks Done: 2 of 5
69
+
70
+ WHAT YOU ACCOMPLISHED
71
+ β€’ Completed the TechVentix demo β€” Rajesh confirmed he wants to proceed to pilot
72
+ β€’ Merged PR #47 β€” sprint blocker cleared, team can proceed
73
+ β€’ Replied to all starred emails (3/3)
74
+
75
+ WHAT'S STILL OPEN
76
+ ⬜ Follow up on invoice INV-2031 β€” send a firm payment reminder first thing tomorrow
77
+ ⬜ Update project roadmap β€” can be done in your 13:00 focus block tomorrow
78
+ ⬜ Weekly status report β€” delegate the data pull to Amit, you write the summary
79
+
80
+ EMAIL FOLLOW-UPS NEEDED
81
+ - Rajesh confirmed pilot interest β€” send onboarding doc and next steps tonight or first thing
82
+ - Accounts team re: INV-2031 β€” needs a direct call, not just email
83
+
84
+ TOMORROW PREVIEW
85
+ 10:00–11:30 Sprint planning β€” bring updated velocity chart
86
+ 14:00–14:30 1:1 with manager β€” prepare 3 updates: TechVentix pilot, PR status, Q2 targets
87
+
88
+ REFLECTION
89
+ Strong client outcome today, but 2 hrs in meetings left the afternoon fragmented.
90
+ Tomorrow's sprint planning is your biggest commitment β€” prep it tonight in 20 min.
91
+
92
+ SHUT DOWN RITUAL βœ“
93
+ β–‘ Pending tasks moved to tomorrow's list
94
+ β–‘ Urgent emails flagged for morning
95
+ β–‘ Calendar checked for 8 AM conflicts
96
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
97
+ ```
98
+
99
+ ## Run permanently in background
100
+
101
+ ```bash
102
+ nohup python eod_review.py --schedule > eod_review.log 2>&1 &
103
+ ```
104
+
105
+ Or via cron (more reliable):
106
+ ```bash
107
+ crontab -e
108
+ # Add:
109
+ 30 18 * * 1-6 cd /path/to/eod_review_agent && python eod_review.py >> eod_review.log 2>&1
110
+ ```
eod_review_agent/data_fetcher.py ADDED
@@ -0,0 +1,402 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ data_fetcher.py
3
+ ================
4
+ Fetches all data needed for the EOD review:
5
+ 1. Meetings had today (Google Calendar)
6
+ 2. Emails received today (Gmail)
7
+ 3. Emails sent today (Gmail sent)
8
+ 4. Tasks completed vs pending (Notion API or local tasks.txt)
9
+ 5. Tomorrow's calendar preview (Google Calendar)
10
+ """
11
+
12
+ import os
13
+ import json
14
+ import requests
15
+ from datetime import datetime, timedelta, timezone
16
+ from google.oauth2.credentials import Credentials
17
+ from google_auth_oauthlib.flow import InstalledAppFlow
18
+ from google.auth.transport.requests import Request
19
+ from googleapiclient.discovery import build
20
+
21
+ SCOPES = [
22
+ "https://www.googleapis.com/auth/calendar.readonly",
23
+ "https://www.googleapis.com/auth/gmail.readonly",
24
+ ]
25
+ CREDENTIALS_FILE = "credentials.json"
26
+ TOKEN_FILE = "token.json"
27
+
28
+
29
+ # ─── AUTH ─────────────────────────────────────────────────────────────────────
30
+
31
+ def get_google_service(api_name: str, api_version: str):
32
+ creds = None
33
+ if os.path.exists(TOKEN_FILE):
34
+ creds = Credentials.from_authorized_user_file(TOKEN_FILE, SCOPES)
35
+ if not creds or not creds.valid:
36
+ if creds and creds.expired and creds.refresh_token:
37
+ creds.refresh(Request())
38
+ else:
39
+ if not os.path.exists(CREDENTIALS_FILE):
40
+ raise FileNotFoundError(
41
+ "Missing credentials.json β€” copy from Daily Planner Agent folder."
42
+ )
43
+ flow = InstalledAppFlow.from_client_secrets_file(CREDENTIALS_FILE, SCOPES)
44
+ creds = flow.run_local_server(port=0)
45
+ with open(TOKEN_FILE, "w") as f:
46
+ f.write(creds.to_json())
47
+ return build(api_name, api_version, credentials=creds)
48
+
49
+
50
+ # ─── MAIN FETCH ───────────────────────────────────────────────────────────────
51
+
52
+ def fetch_all_data() -> dict:
53
+ """Fetch all data sources and return as a single dict."""
54
+ return {
55
+ "date_today": datetime.now().strftime("%A, %d %B %Y"),
56
+ "meetings_today": get_meetings_today(),
57
+ "emails_received": get_emails_received_today(),
58
+ "emails_sent": get_emails_sent_today(),
59
+ "tasks": get_tasks(),
60
+ "tomorrow_meetings": get_tomorrow_meetings(),
61
+ }
62
+
63
+
64
+ # ─── 1. MEETINGS TODAY ────────────────────────────────────────────────────────
65
+
66
+ def get_meetings_today() -> list[dict]:
67
+ """Fetch all calendar events that occurred today."""
68
+ try:
69
+ service = get_google_service("calendar", "v3")
70
+ now = datetime.now(timezone.utc)
71
+ start = now.replace(hour=0, minute=0, second=0, microsecond=0)
72
+ # Only fetch meetings that have already started (up to now)
73
+ end = now
74
+
75
+ result = service.events().list(
76
+ calendarId="primary",
77
+ timeMin=start.isoformat(),
78
+ timeMax=end.isoformat(),
79
+ singleEvents=True,
80
+ orderBy="startTime",
81
+ maxResults=20,
82
+ ).execute()
83
+
84
+ meetings = []
85
+ for e in result.get("items", []):
86
+ start_raw = e.get("start", {})
87
+ end_raw = e.get("end", {})
88
+ start_str = start_raw.get("dateTime", start_raw.get("date", ""))
89
+ end_str = end_raw.get("dateTime", end_raw.get("date", ""))
90
+
91
+ def fmt(s):
92
+ try:
93
+ return datetime.fromisoformat(s).strftime("%H:%M")
94
+ except Exception:
95
+ return s
96
+
97
+ attendees = [
98
+ a.get("email", "") for a in e.get("attendees", [])
99
+ if not a.get("self", False)
100
+ ]
101
+
102
+ meetings.append({
103
+ "title": e.get("summary", "Untitled"),
104
+ "start": fmt(start_str),
105
+ "end": fmt(end_str),
106
+ "attendees": attendees,
107
+ "duration": _duration_mins(start_str, end_str),
108
+ })
109
+
110
+ return meetings or _demo_meetings_today()
111
+
112
+ except FileNotFoundError:
113
+ return _demo_meetings_today()
114
+ except Exception as e:
115
+ print(f" Calendar error: {e}")
116
+ return _demo_meetings_today()
117
+
118
+
119
+ # ─── 2. EMAILS RECEIVED TODAY ─────────────────────────────────────────────────
120
+
121
+ def get_emails_received_today(max_results: int = 20) -> list[dict]:
122
+ """Fetch emails received today (inbox)."""
123
+ try:
124
+ service = get_google_service("gmail", "v1")
125
+ today_str = datetime.now().strftime("%Y/%m/%d")
126
+ query = f"after:{today_str} -category:promotions -category:social in:inbox"
127
+
128
+ result = service.users().messages().list(
129
+ userId="me", q=query, maxResults=max_results
130
+ ).execute()
131
+
132
+ return _parse_messages(service, result.get("messages", []))
133
+
134
+ except Exception as e:
135
+ print(f" Gmail received error: {e}")
136
+ return _demo_emails_received()
137
+
138
+
139
+ # ─── 3. EMAILS SENT TODAY ─────────────────────────────────────────────────────
140
+
141
+ def get_emails_sent_today(max_results: int = 15) -> list[dict]:
142
+ """Fetch emails you sent today."""
143
+ try:
144
+ service = get_google_service("gmail", "v1")
145
+ today_str = datetime.now().strftime("%Y/%m/%d")
146
+ query = f"in:sent after:{today_str}"
147
+
148
+ result = service.users().messages().list(
149
+ userId="me", q=query, maxResults=max_results
150
+ ).execute()
151
+
152
+ return _parse_messages(service, result.get("messages", []))
153
+
154
+ except Exception as e:
155
+ print(f" Gmail sent error: {e}")
156
+ return _demo_emails_sent()
157
+
158
+
159
+ def _parse_messages(service, messages: list) -> list[dict]:
160
+ emails = []
161
+ for msg in messages:
162
+ detail = service.users().messages().get(
163
+ userId="me",
164
+ id=msg["id"],
165
+ format="metadata",
166
+ metadataHeaders=["Subject", "From", "To", "Date"]
167
+ ).execute()
168
+
169
+ headers = {
170
+ h["name"]: h["value"]
171
+ for h in detail.get("payload", {}).get("headers", [])
172
+ }
173
+ snippet = detail.get("snippet", "").replace("'", "'")[:150]
174
+
175
+ emails.append({
176
+ "subject": headers.get("Subject", "(no subject)"),
177
+ "from": headers.get("From", ""),
178
+ "to": headers.get("To", ""),
179
+ "snippet": snippet,
180
+ "starred": "STARRED" in detail.get("labelIds", []),
181
+ })
182
+ return emails
183
+
184
+
185
+ # ─── 4. TASKS ─────────────────────────────────────────────────────────────────
186
+
187
+ def get_tasks() -> list[dict]:
188
+ """
189
+ Fetch tasks from Notion database or fall back to local tasks.txt.
190
+ Set NOTION_TASKS_DB_ID in .env for Notion integration.
191
+ Otherwise, create a tasks.txt file with lines like:
192
+ [done] Write project proposal
193
+ [pending] Review PR #47
194
+ [pending] Call Rajesh
195
+ """
196
+ notion_db = os.getenv("NOTION_TASKS_DB_ID")
197
+ if notion_db:
198
+ return _get_notion_tasks(notion_db)
199
+ else:
200
+ return _get_local_tasks()
201
+
202
+
203
+ def _get_notion_tasks(db_id: str) -> list[dict]:
204
+ """Fetch today's tasks from a Notion database."""
205
+ token = os.getenv("NOTION_TOKEN")
206
+ if not token:
207
+ return []
208
+
209
+ try:
210
+ today = datetime.now().strftime("%Y-%m-%d")
211
+ headers = {
212
+ "Authorization": f"Bearer {token}",
213
+ "Content-Type": "application/json",
214
+ "Notion-Version": "2022-06-28",
215
+ }
216
+
217
+ # Query tasks created or due today
218
+ payload = {
219
+ "filter": {
220
+ "or": [
221
+ {"property": "Due", "date": {"equals": today}},
222
+ {"property": "Created", "date": {"equals": today}},
223
+ ]
224
+ }
225
+ }
226
+
227
+ resp = requests.post(
228
+ f"https://api.notion.com/v1/databases/{db_id}/query",
229
+ headers=headers,
230
+ json=payload,
231
+ timeout=10,
232
+ )
233
+ resp.raise_for_status()
234
+ pages = resp.json().get("results", [])
235
+
236
+ tasks = []
237
+ for page in pages:
238
+ props = page.get("properties", {})
239
+ title_prop = props.get("Name", props.get("Title", {}))
240
+ title_items = title_prop.get("title", [])
241
+ title = title_items[0]["plain_text"] if title_items else "Untitled"
242
+
243
+ # Try to get status
244
+ status_prop = props.get("Status", props.get("Checkbox", {}))
245
+ done = False
246
+ if status_prop.get("type") == "checkbox":
247
+ done = status_prop.get("checkbox", False)
248
+ elif status_prop.get("type") == "status":
249
+ done = status_prop.get("status", {}).get("name", "").lower() in ("done", "complete", "completed")
250
+
251
+ tasks.append({"title": title, "done": done})
252
+
253
+ return tasks or _demo_tasks()
254
+
255
+ except Exception as e:
256
+ print(f" Notion tasks error: {e}")
257
+ return _demo_tasks()
258
+
259
+
260
+ def _get_local_tasks() -> list[dict]:
261
+ """
262
+ Read from local tasks.txt.
263
+ Format each line as:
264
+ [done] Task name
265
+ [pending] Task name
266
+ """
267
+ tasks_file = os.getenv("TASKS_FILE", "tasks.txt")
268
+ tasks = []
269
+
270
+ if not os.path.exists(tasks_file):
271
+ return _demo_tasks()
272
+
273
+ with open(tasks_file, "r", encoding="utf-8") as f:
274
+ for line in f:
275
+ line = line.strip()
276
+ if not line:
277
+ continue
278
+ if line.lower().startswith("[done]"):
279
+ tasks.append({"title": line[6:].strip(), "done": True})
280
+ elif line.lower().startswith("[pending]"):
281
+ tasks.append({"title": line[9:].strip(), "done": False})
282
+ else:
283
+ tasks.append({"title": line, "done": False})
284
+
285
+ return tasks or _demo_tasks()
286
+
287
+
288
+ # ─── 5. TOMORROW'S MEETINGS ───────────────────────────────────────────────────
289
+
290
+ def get_tomorrow_meetings() -> list[dict]:
291
+ """Fetch tomorrow's calendar events."""
292
+ try:
293
+ service = get_google_service("calendar", "v3")
294
+ now = datetime.now(timezone.utc)
295
+ tomorrow_start = (now + timedelta(days=1)).replace(
296
+ hour=0, minute=0, second=0, microsecond=0
297
+ )
298
+ tomorrow_end = tomorrow_start.replace(hour=23, minute=59, second=59)
299
+
300
+ result = service.events().list(
301
+ calendarId="primary",
302
+ timeMin=tomorrow_start.isoformat(),
303
+ timeMax=tomorrow_end.isoformat(),
304
+ singleEvents=True,
305
+ orderBy="startTime",
306
+ maxResults=10,
307
+ ).execute()
308
+
309
+ meetings = []
310
+ for e in result.get("items", []):
311
+ start_raw = e.get("start", {})
312
+ end_raw = e.get("end", {})
313
+ start_str = start_raw.get("dateTime", start_raw.get("date", ""))
314
+ end_str = end_raw.get("dateTime", end_raw.get("date", ""))
315
+
316
+ def fmt(s):
317
+ try:
318
+ return datetime.fromisoformat(s).strftime("%H:%M")
319
+ except Exception:
320
+ return s
321
+
322
+ attendees = [
323
+ a.get("email", "") for a in e.get("attendees", [])
324
+ if not a.get("self", False)
325
+ ]
326
+
327
+ meetings.append({
328
+ "title": e.get("summary", "Untitled"),
329
+ "start": fmt(start_str),
330
+ "end": fmt(end_str),
331
+ "attendees": attendees,
332
+ })
333
+
334
+ return meetings or _demo_tomorrow_meetings()
335
+
336
+ except FileNotFoundError:
337
+ return _demo_tomorrow_meetings()
338
+ except Exception as e:
339
+ print(f" Tomorrow calendar error: {e}")
340
+ return _demo_tomorrow_meetings()
341
+
342
+
343
+ # ─── UTILITIES ────────────────────────────────────────────────────────────────
344
+
345
+ def _duration_mins(start_str: str, end_str: str) -> int:
346
+ try:
347
+ start = datetime.fromisoformat(start_str)
348
+ end = datetime.fromisoformat(end_str)
349
+ return int((end - start).total_seconds() / 60)
350
+ except Exception:
351
+ return 0
352
+
353
+
354
+ # ─── DEMO DATA ────────────────────────────────────────────────────────────────
355
+
356
+ def _demo_meetings_today() -> list[dict]:
357
+ print(" Using demo meeting data.")
358
+ return [
359
+ {"title": "Team standup", "start": "09:00", "end": "09:30",
360
+ "attendees": ["team@company.com"], "duration": 30},
361
+ {"title": "Client demo β€” TechVentix", "start": "11:00", "end": "12:00",
362
+ "attendees": ["rajesh@techventix.com"], "duration": 60},
363
+ {"title": "HR review call", "start": "15:00", "end": "15:30",
364
+ "attendees": ["hr@company.com"], "duration": 30},
365
+ ]
366
+
367
+ def _demo_emails_received() -> list[dict]:
368
+ print(" Using demo email data.")
369
+ return [
370
+ {"subject": "Re: Project proposal", "from": "Rajesh <rajesh@client.com>",
371
+ "snippet": "Looks good, let's proceed with the pilot.", "starred": True},
372
+ {"subject": "PR #47 merged", "from": "GitHub <noreply@github.com>",
373
+ "snippet": "Amit merged pull request #47 into main.", "starred": False},
374
+ {"subject": "Invoice paid", "from": "Razorpay <noreply@razorpay.com>",
375
+ "snippet": "Payment of β‚Ή45,000 received from TechVentix.", "starred": False},
376
+ ]
377
+
378
+ def _demo_emails_sent() -> list[dict]:
379
+ return [
380
+ {"subject": "Demo follow-up", "to": "rajesh@techventix.com",
381
+ "snippet": "Thanks for your time today. Sharing the pilot details.", "starred": False},
382
+ {"subject": "PR review comments", "to": "amit@team.com",
383
+ "snippet": "Left comments on lines 45-67. Please fix before re-review.", "starred": False},
384
+ ]
385
+
386
+ def _demo_tasks() -> list[dict]:
387
+ print(" Using demo task data.")
388
+ return [
389
+ {"title": "Reply to Rajesh demo follow-up", "done": True},
390
+ {"title": "Review PR #47", "done": True},
391
+ {"title": "Follow up on invoice INV-2031", "done": False},
392
+ {"title": "Update project roadmap doc", "done": False},
393
+ {"title": "Send weekly report to manager", "done": False},
394
+ ]
395
+
396
+ def _demo_tomorrow_meetings() -> list[dict]:
397
+ return [
398
+ {"title": "Sprint planning", "start": "10:00", "end": "11:30",
399
+ "attendees": ["team@company.com"]},
400
+ {"title": "1:1 with manager", "start": "14:00", "end": "14:30",
401
+ "attendees": ["manager@company.com"]},
402
+ ]
eod_review_agent/delivery.py ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ delivery.py
3
+ ============
4
+ Delivers the EOD review via Email, WhatsApp, and Notion.
5
+ Also saves a local copy to eod_reviews/ folder.
6
+ """
7
+
8
+ import os
9
+ import smtplib
10
+ import requests
11
+ from email.mime.text import MIMEText
12
+ from email.mime.multipart import MIMEMultipart
13
+ from datetime import datetime
14
+ from dotenv import load_dotenv
15
+
16
+ load_dotenv()
17
+
18
+
19
+ def deliver_review(review: str, data: dict):
20
+ """Deliver EOD review to all configured channels."""
21
+ today = data.get("date_today", datetime.now().strftime("%A, %d %B %Y"))
22
+ subject = f"End-of-Day Review β€” {today}"
23
+
24
+ # Always print to console
25
+ print("\n" + "="*54)
26
+ print(review)
27
+ print("="*54 + "\n")
28
+
29
+ # Always save locally
30
+ _save_to_file(review)
31
+ print(" βœ“ Saved to eod_reviews/ folder")
32
+
33
+ # Email
34
+ if os.getenv("EMAIL_ENABLED", "false").lower() == "true":
35
+ try:
36
+ _send_email(subject, review)
37
+ print(" βœ“ Email sent")
38
+ except Exception as e:
39
+ print(f" βœ— Email failed: {e}")
40
+
41
+ # WhatsApp
42
+ if os.getenv("WHATSAPP_ENABLED", "false").lower() == "true":
43
+ try:
44
+ _send_whatsapp(review)
45
+ print(" βœ“ WhatsApp sent")
46
+ except Exception as e:
47
+ print(f" βœ— WhatsApp failed: {e}")
48
+
49
+ # Notion
50
+ if os.getenv("NOTION_ENABLED", "false").lower() == "true":
51
+ try:
52
+ _save_to_notion(review, subject)
53
+ print(" βœ“ Saved to Notion")
54
+ except Exception as e:
55
+ print(f" βœ— Notion failed: {e}")
56
+
57
+
58
+ # ─── EMAIL ────────────────────────────────────────────────────────────────────
59
+
60
+ def _send_email(subject: str, body: str):
61
+ sender = os.getenv("EMAIL_SENDER")
62
+ password = os.getenv("EMAIL_PASSWORD")
63
+ recipient = os.getenv("EMAIL_RECIPIENT", sender)
64
+
65
+ if not sender or not password:
66
+ raise ValueError("EMAIL_SENDER and EMAIL_PASSWORD required in .env")
67
+
68
+ msg = MIMEMultipart("alternative")
69
+ msg["Subject"] = subject
70
+ msg["From"] = sender
71
+ msg["To"] = recipient
72
+
73
+ msg.attach(MIMEText(body, "plain"))
74
+ msg.attach(MIMEText(_to_html(body), "html"))
75
+
76
+ with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
77
+ server.login(sender, password)
78
+ server.sendmail(sender, recipient, msg.as_string())
79
+
80
+
81
+ def _to_html(text: str) -> str:
82
+ EMOJI_COLORS = {
83
+ "βœ…": "#27ae60", "⬜": "#e67e22", "⭐": "#f1c40f",
84
+ "πŸ—“": "#3498db", "πŸ“§": "#9b59b6", "β–‘": "#95a5a6",
85
+ }
86
+ lines = text.split("\n")
87
+ html = []
88
+ for line in lines:
89
+ if line.startswith("━"):
90
+ html.append("<hr style='border:1px solid #ddd;margin:12px 0'>")
91
+ elif line.startswith("END OF DAY"):
92
+ html.append(f"<h2 style='color:#2c3e50;font-family:sans-serif'>{line}</h2>")
93
+ elif line.isupper() and len(line) > 3 and not line.startswith("β–‘"):
94
+ html.append(f"<h3 style='color:#555;font-family:sans-serif;margin-top:16px'>{line}</h3>")
95
+ elif line.strip():
96
+ html.append(f"<p style='font-family:sans-serif;color:#333;margin:4px 0;line-height:1.5'>{line}</p>")
97
+ return f"""
98
+ <div style='max-width:620px;margin:0 auto;padding:24px;background:#fff'>
99
+ {''.join(html)}
100
+ <p style='color:#bbb;font-size:11px;margin-top:24px'>
101
+ Sent by your End-of-Day Review Agent at {datetime.now().strftime('%H:%M')}
102
+ </p>
103
+ </div>
104
+ """
105
+
106
+
107
+ # ─── WHATSAPP ─────────────────────────────────────────────────────────────────
108
+
109
+ def _send_whatsapp(message: str):
110
+ try:
111
+ from twilio.rest import Client
112
+ except ImportError:
113
+ raise ImportError("Run: pip install twilio")
114
+
115
+ sid = os.getenv("TWILIO_ACCOUNT_SID")
116
+ auth = os.getenv("TWILIO_AUTH_TOKEN")
117
+ from_num = os.getenv("TWILIO_FROM")
118
+ to_num = os.getenv("TWILIO_TO")
119
+
120
+ if not all([sid, auth, from_num, to_num]):
121
+ raise ValueError("Twilio credentials missing in .env")
122
+
123
+ client = Client(sid, auth)
124
+ # Split into 1600-char chunks (WhatsApp limit)
125
+ for chunk in [message[i:i+1600] for i in range(0, len(message), 1600)]:
126
+ client.messages.create(body=chunk, from_=from_num, to=to_num)
127
+
128
+
129
+ # ─── NOTION ───────────────────────────────────────────────────────────────────
130
+
131
+ def _save_to_notion(review: str, title: str):
132
+ token = os.getenv("NOTION_TOKEN")
133
+ db_id = os.getenv("NOTION_DATABASE_ID")
134
+
135
+ if not token or not db_id:
136
+ raise ValueError("NOTION_TOKEN and NOTION_DATABASE_ID required in .env")
137
+
138
+ headers = {
139
+ "Authorization": f"Bearer {token}",
140
+ "Content-Type": "application/json",
141
+ "Notion-Version": "2022-06-28",
142
+ }
143
+
144
+ blocks = []
145
+ for para in review.split("\n"):
146
+ if not para.strip():
147
+ continue
148
+ if para.startswith("━"):
149
+ blocks.append({"object": "block", "type": "divider", "divider": {}})
150
+ else:
151
+ blocks.append({
152
+ "object": "block",
153
+ "type": "paragraph",
154
+ "paragraph": {
155
+ "rich_text": [{"type": "text", "text": {"content": para[:2000]}}]
156
+ }
157
+ })
158
+
159
+ payload = {
160
+ "parent": {"database_id": db_id},
161
+ "properties": {
162
+ "Name": {"title": [{"text": {"content": title}}]},
163
+ "Date": {"date": {"start": datetime.now().strftime("%Y-%m-%d")}},
164
+ },
165
+ "children": blocks[:100],
166
+ }
167
+
168
+ resp = requests.post(
169
+ "https://api.notion.com/v1/pages",
170
+ headers=headers,
171
+ json=payload,
172
+ timeout=15,
173
+ )
174
+ resp.raise_for_status()
175
+
176
+
177
+ # ─── FILE SAVE ────────────────────────────────────────────────────────────────
178
+
179
+ def _save_to_file(review: str):
180
+ os.makedirs("eod_reviews", exist_ok=True)
181
+ filename = f"eod_reviews/{datetime.now().strftime('%Y-%m-%d')}.txt"
182
+ with open(filename, "w", encoding="utf-8") as f:
183
+ f.write(review)
eod_review_agent/eod_review.py ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ End-of-Day Review Agent
3
+ ========================
4
+ Fires every day at 6:30 PM.
5
+ Pulls data from 4 sources:
6
+ - Google Calendar (meetings had today)
7
+ - Gmail (emails sent & received today)
8
+ - Tasks (Notion or local file)
9
+ - Google Calendar again (tomorrow's preview)
10
+ Then generates a structured EOD review + tomorrow prep via Groq.
11
+ Delivers via Email + WhatsApp + Notion.
12
+
13
+ Usage:
14
+ python eod_review.py # Run once immediately
15
+ python eod_review.py --schedule # Run on schedule at 6:30 PM daily
16
+ """
17
+
18
+ import os
19
+ import argparse
20
+ import schedule
21
+ import time
22
+ from datetime import datetime
23
+ import os as _os
24
+ from dotenv import load_dotenv
25
+
26
+ from data_fetcher import fetch_all_data
27
+ from review_llm import generate_eod_review
28
+ from delivery import deliver_review
29
+
30
+ load_dotenv(_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "..", ".env"))
31
+
32
+
33
+ def run_agent():
34
+ print(f"\n{'='*54}")
35
+ print(f"End-of-Day Review Agent β€” {datetime.now().strftime('%A %d %B %Y, %H:%M')}")
36
+ print("="*54)
37
+
38
+ print("\n[1/3] Fetching today's data...")
39
+ data = fetch_all_data()
40
+ print(f" Meetings today : {len(data['meetings_today'])}")
41
+ print(f" Emails received: {len(data['emails_received'])}")
42
+ print(f" Emails sent : {len(data['emails_sent'])}")
43
+ print(f" Tasks : {len(data['tasks'])}")
44
+ print(f" Tomorrow events: {len(data['tomorrow_meetings'])}")
45
+
46
+ print("\n[2/3] Generating review with Groq...")
47
+ review = generate_eod_review(data)
48
+ print(" Review generated.")
49
+
50
+ print("\n[3/3] Delivering review...")
51
+ deliver_review(review, data)
52
+
53
+ print("\nDone! End-of-day review delivered.\n")
54
+ return review
55
+
56
+
57
+ if __name__ == "__main__":
58
+ parser = argparse.ArgumentParser(description="End-of-Day Review Agent")
59
+ parser.add_argument("--schedule", action="store_true",
60
+ help="Run on schedule at 6:30 PM daily")
61
+ parser.add_argument("--time", default="18:30",
62
+ help="Schedule time HH:MM (default: 18:30)")
63
+ args = parser.parse_args()
64
+
65
+ if args.schedule:
66
+ print(f"Scheduler started. Will run daily at {args.time}.")
67
+ print("Press Ctrl+C to stop.\n")
68
+ schedule.every().day.at(args.time).do(run_agent)
69
+ # Run once immediately on start
70
+ run_agent()
71
+ while True:
72
+ schedule.run_pending()
73
+ time.sleep(60)
74
+ else:
75
+ review = run_agent()
76
+ print("\n" + "="*54)
77
+ print("YOUR END-OF-DAY REVIEW:")
78
+ print("="*54)
79
+ print(review)
eod_review_agent/eod_reviews/2026-05-14.txt ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2
+ END OF DAY β€” Thursday, 14 May 2026
3
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
4
+
5
+ TODAY AT A GLANCE
6
+ πŸ—“ Meetings : 3 meetings, total 2 hrs
7
+ πŸ“§ Emails : 3 received, 2 sent
8
+ βœ… Tasks Done: 2 of 11 total
9
+
10
+ WHAT YOU ACCOMPLISHED
11
+ * Successfully completed the client demo with TechVentix and sent a follow-up email with pilot details.
12
+ * Approved PR #47 and left review comments on lines 45-67 for Amit to address.
13
+ * Responded to Rajesh's email regarding the project proposal, moving the pilot forward.
14
+
15
+ WHAT'S STILL OPEN
16
+ * Follow up on overdue invoice INV-2031: Send a reminder email to the client with a clear deadline for payment.
17
+ * Update project roadmap document: Allocate 30 minutes tomorrow to review and update the document.
18
+ * Send weekly status report to manager: Prepare a draft tonight and review it before sending it to the manager tomorrow.
19
+
20
+ EMAIL FOLLOW-UPS NEEDED
21
+ * Rajesh's response to the demo follow-up email: Wait for his confirmation on the pilot details.
22
+ * Amit's response to PR review comments: Follow up on the changes made to the code.
23
+
24
+ TOMORROW PREVIEW
25
+ * 10:00–11:30 Sprint planning: Review the team's tasks and objectives for the upcoming sprint.
26
+ * 14:00–14:30 1:1 with manager: Prepare an update on your tasks and discuss any challenges or concerns.
27
+
28
+ REFLECTION
29
+ You had a productive day with effective meetings and task management, but there are still 9 pending tasks that need attention. Consider prioritizing tasks based on urgency and importance to manage your workload better.
30
+
31
+ SHUT DOWN RITUAL βœ“
32
+ β–‘ Pending tasks moved to tomorrow's list
33
+ β–‘ Urgent emails flagged for morning
34
+ β–‘ Calendar checked for 8 AM conflicts
35
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
eod_review_agent/requirements.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ groq>=0.9.0
2
+ google-auth>=2.27.0
3
+ google-auth-oauthlib>=1.2.0
4
+ google-auth-httplib2>=0.2.0
5
+ google-api-python-client>=2.120.0
6
+ python-dotenv>=1.0.0
7
+ schedule>=1.2.0
8
+ requests>=2.31.0
9
+ twilio>=9.0.0
eod_review_agent/review_llm.py ADDED
@@ -0,0 +1,146 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ review_llm.py
3
+ ==============
4
+ Sends all day's data to Groq and returns a structured EOD review.
5
+ """
6
+
7
+ import os
8
+ from groq import Groq
9
+
10
+ SYSTEM_PROMPT = """You are a sharp personal productivity coach wrapping up the day for a busy IT professional in Ahmedabad, India.
11
+
12
+ You receive a full summary of their day: meetings had, emails received and sent, tasks done vs pending, and tomorrow's calendar.
13
+
14
+ Generate a concise, honest end-of-day review in EXACTLY this format:
15
+
16
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
17
+ END OF DAY β€” [Day, Date]
18
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
19
+
20
+ TODAY AT A GLANCE
21
+ πŸ—“ Meetings : [N meetings, total X hrs]
22
+ πŸ“§ Emails : [N received, N sent]
23
+ βœ… Tasks Done: [N of N total]
24
+
25
+ WHAT YOU ACCOMPLISHED
26
+ [2-3 bullet points β€” specific wins from meetings, emails, and tasks completed]
27
+
28
+ WHAT'S STILL OPEN
29
+ [Bullet each pending task with a one-line suggestion: carry forward / delegate / drop]
30
+
31
+ EMAIL FOLLOW-UPS NEEDED
32
+ [List 1-3 emails that likely need a reply or action tomorrow. If none: "Inbox actioned."]
33
+
34
+ TOMORROW PREVIEW
35
+ [List each meeting with time and one-line prep note]
36
+ [If no meetings: "No meetings tomorrow β€” protect it for deep work."]
37
+
38
+ REFLECTION
39
+ [One honest, specific observation about today β€” productivity pattern, energy use, or a lesson]
40
+
41
+ SHUT DOWN RITUAL βœ“
42
+ β–‘ Pending tasks moved to tomorrow's list
43
+ β–‘ Urgent emails flagged for morning
44
+ β–‘ Calendar checked for 8 AM conflicts
45
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
46
+
47
+ Rules:
48
+ - Be specific β€” reference actual meeting names, email subjects, task titles from the data.
49
+ - "What's still open" must have concrete next-action suggestions, not vague "follow up."
50
+ - Reflection must be honest and personal, not generic motivational fluff.
51
+ - Total length: under 400 words.
52
+ - Tone: calm, grounded, like a trusted coach reviewing the day with you.
53
+ """
54
+
55
+
56
+ def generate_eod_review(data: dict) -> str:
57
+ """
58
+ Generate end-of-day review from the day's collected data.
59
+
60
+ Args:
61
+ data: Dict from data_fetcher.fetch_all_data()
62
+
63
+ Returns:
64
+ Formatted EOD review string
65
+ """
66
+ api_key = os.getenv("GROQ_API_KEY")
67
+ if not api_key:
68
+ raise ValueError("GROQ_API_KEY not found in .env")
69
+
70
+ client = Groq(api_key=api_key)
71
+ context = _build_context(data)
72
+
73
+ response = client.chat.completions.create(
74
+ model="llama-3.3-70b-versatile",
75
+ max_tokens=1200,
76
+ messages=[
77
+ {"role": "system", "content": SYSTEM_PROMPT},
78
+ {"role": "user", "content": context},
79
+ ]
80
+ )
81
+
82
+ return response.choices[0].message.content
83
+
84
+
85
+ def _build_context(data: dict) -> str:
86
+ lines = [f"DATE: {data['date_today']}", ""]
87
+
88
+ # Meetings today
89
+ meetings = data.get("meetings_today", [])
90
+ total_mins = sum(m.get("duration", 0) for m in meetings)
91
+ lines.append(f"MEETINGS TODAY ({len(meetings)}, ~{total_mins} min total):")
92
+ if meetings:
93
+ for m in meetings:
94
+ att = ", ".join(m.get("attendees", [])) or "internal"
95
+ lines.append(f" - {m['start']}–{m['end']} {m['title']} [{att}]")
96
+ else:
97
+ lines.append(" No meetings today.")
98
+ lines.append("")
99
+
100
+ # Emails received
101
+ received = data.get("emails_received", [])
102
+ lines.append(f"EMAILS RECEIVED TODAY ({len(received)}):")
103
+ for e in received[:10]:
104
+ flag = "⭐ " if e.get("starred") else " "
105
+ lines.append(f" {flag}From: {e['from']} | {e['subject']} | {e['snippet'][:100]}")
106
+ if not received:
107
+ lines.append(" None.")
108
+ lines.append("")
109
+
110
+ # Emails sent
111
+ sent = data.get("emails_sent", [])
112
+ lines.append(f"EMAILS SENT TODAY ({len(sent)}):")
113
+ for e in sent[:8]:
114
+ lines.append(f" - To: {e.get('to','')} | {e['subject']} | {e['snippet'][:100]}")
115
+ if not sent:
116
+ lines.append(" None sent today.")
117
+ lines.append("")
118
+
119
+ # Tasks
120
+ tasks = data.get("tasks", [])
121
+ done = [t for t in tasks if t.get("done")]
122
+ pending = [t for t in tasks if not t.get("done")]
123
+ lines.append(f"TASKS ({len(done)} done, {len(pending)} pending):")
124
+ if done:
125
+ lines.append(" Completed:")
126
+ for t in done:
127
+ lines.append(f" βœ… {t['title']}")
128
+ if pending:
129
+ lines.append(" Still pending:")
130
+ for t in pending:
131
+ lines.append(f" ⬜ {t['title']}")
132
+ if not tasks:
133
+ lines.append(" No tasks data available.")
134
+ lines.append("")
135
+
136
+ # Tomorrow
137
+ tomorrow = data.get("tomorrow_meetings", [])
138
+ lines.append(f"TOMORROW'S CALENDAR ({len(tomorrow)} meetings):")
139
+ if tomorrow:
140
+ for m in tomorrow:
141
+ att = ", ".join(m.get("attendees", [])) or "internal"
142
+ lines.append(f" - {m['start']}–{m['end']} {m['title']} [{att}]")
143
+ else:
144
+ lines.append(" No meetings scheduled tomorrow.")
145
+
146
+ return "\n".join(lines)
eod_review_agent/tasks.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # tasks.txt β€” Edit this file daily with your tasks
2
+ # Format:
3
+ # [done] Task you completed today
4
+ # [pending] Task still to do
5
+ #
6
+ # The EOD Review Agent reads this file at 6:30 PM.
7
+
8
+ [done] Reply to Rajesh demo follow-up email
9
+ [done] Review and approve PR #47
10
+ [pending] Follow up on overdue invoice INV-2031
11
+ [pending] Update project roadmap document
12
+ [pending] Send weekly status report to manager
knowledge_agent/cli.py ADDED
@@ -0,0 +1,224 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ cli.py β€” Command-line interface for the Knowledge Agent
3
+ =======================================================
4
+ Usage examples:
5
+ python cli.py index # index ./documents/
6
+ python cli.py index --dir ~/my_docs # index a custom folder
7
+ python cli.py index --force # force re-index everything
8
+ python cli.py ask "What is our pricing?" # one-shot Q&A
9
+ python cli.py chat # interactive chat loop
10
+ python cli.py list # list indexed documents
11
+ python cli.py stats # show DB stats
12
+ python cli.py delete report.pdf # remove a document
13
+ """
14
+
15
+ import argparse
16
+ import sys
17
+ import os
18
+ from typing import List, Dict
19
+
20
+ # ── Pretty printing helpers ───────────────────────────────────────────────────
21
+ try:
22
+ from rich.console import Console
23
+ from rich.markdown import Markdown
24
+ from rich.table import Table
25
+ from rich.panel import Panel
26
+ from rich import print as rprint
27
+ RICH = True
28
+ console = Console()
29
+ except ImportError:
30
+ RICH = False
31
+ console = None
32
+
33
+ from knowledge_api import (
34
+ query_knowledge,
35
+ index_docs,
36
+ index_single,
37
+ get_knowledge_stats,
38
+ list_indexed_docs,
39
+ delete_doc,
40
+ )
41
+
42
+
43
+ def print_answer(result: Dict):
44
+ answer = result.get("answer", "")
45
+ sources = result.get("sources", [])
46
+ chunks = result.get("chunks_used", 0)
47
+
48
+ if RICH:
49
+ console.print(Panel(
50
+ Markdown(answer),
51
+ title="[bold cyan]Answer[/bold cyan]",
52
+ border_style="cyan",
53
+ ))
54
+ if sources:
55
+ console.print(f"[dim]Sources ({chunks} chunks): {', '.join(sources)}[/dim]")
56
+ else:
57
+ print("\n" + "="*60)
58
+ print("ANSWER:")
59
+ print(answer)
60
+ if sources:
61
+ print(f"\nSources ({chunks} chunks): {', '.join(sources)}")
62
+ print("="*60 + "\n")
63
+
64
+
65
+ def cmd_index(args):
66
+ docs_dir = args.dir or "./documents"
67
+ print(f"[Knowledge Agent] Indexing documents in: {docs_dir}")
68
+ index_docs(docs_dir=docs_dir, force=args.force)
69
+
70
+
71
+ def cmd_ask(args):
72
+ question = " ".join(args.question)
73
+ if RICH:
74
+ console.print(f"\n[bold]Question:[/bold] {question}")
75
+ else:
76
+ print(f"\nQuestion: {question}")
77
+
78
+ result = query_knowledge(question, top_k=args.top_k)
79
+ print_answer(result)
80
+
81
+
82
+ def cmd_chat(args):
83
+ """Interactive REPL β€” keeps asking questions until the user exits."""
84
+ if RICH:
85
+ console.print(Panel(
86
+ "[bold cyan]Knowledge Agent β€” Interactive Chat[/bold cyan]\n"
87
+ "Ask anything about your documents.\n"
88
+ "Type [yellow]exit[/yellow] or [yellow]quit[/yellow] to stop.",
89
+ border_style="cyan",
90
+ ))
91
+ else:
92
+ print("\n" + "="*60)
93
+ print(" Knowledge Agent β€” Interactive Chat")
94
+ print(" Type 'exit' to quit.")
95
+ print("="*60 + "\n")
96
+
97
+ while True:
98
+ try:
99
+ if RICH:
100
+ question = console.input("[bold green]You:[/bold green] ").strip()
101
+ else:
102
+ question = input("You: ").strip()
103
+ except (KeyboardInterrupt, EOFError):
104
+ print("\nGoodbye!")
105
+ break
106
+
107
+ if not question:
108
+ continue
109
+ if question.lower() in ("exit", "quit", "q", "bye"):
110
+ print("Goodbye!")
111
+ break
112
+
113
+ # Special commands in chat mode
114
+ if question.lower() == "/stats":
115
+ cmd_stats(args)
116
+ continue
117
+ if question.lower() == "/list":
118
+ cmd_list(args)
119
+ continue
120
+ if question.lower().startswith("/index "):
121
+ path = question[7:].strip()
122
+ index_single(path)
123
+ continue
124
+
125
+ result = query_knowledge(question, top_k=args.top_k if hasattr(args, "top_k") else 5)
126
+ print_answer(result)
127
+
128
+
129
+ def cmd_list(args):
130
+ docs = list_indexed_docs()
131
+ if not docs:
132
+ print("No documents indexed yet.")
133
+ return
134
+
135
+ if RICH:
136
+ table = Table(title="Indexed Documents", border_style="cyan")
137
+ table.add_column("File", style="bold")
138
+ table.add_column("Status")
139
+ table.add_column("Path", style="dim")
140
+ for d in docs:
141
+ status = "[green]βœ“ exists[/green]" if d["exists"] else "[red]βœ— missing[/red]"
142
+ table.add_row(d["name"], status, d["path"])
143
+ console.print(table)
144
+ else:
145
+ print(f"\n{'File':<40} {'Status':<12} Path")
146
+ print("-" * 80)
147
+ for d in docs:
148
+ status = "βœ“ exists" if d["exists"] else "βœ— missing"
149
+ print(f"{d['name']:<40} {status:<12} {d['path']}")
150
+
151
+
152
+ def cmd_stats(args):
153
+ stats = get_knowledge_stats()
154
+ if RICH:
155
+ console.print(Panel(
156
+ f"[bold]Total chunks:[/bold] {stats['total_chunks']}\n"
157
+ f"[bold]Indexed files:[/bold] {stats['indexed_files']}\n"
158
+ f"[bold]Embed model:[/bold] {stats['embed_model']}\n"
159
+ f"[bold]ChromaDB dir:[/bold] {stats['chroma_dir']}",
160
+ title="[cyan]Knowledge Base Stats[/cyan]",
161
+ border_style="cyan",
162
+ ))
163
+ else:
164
+ print("\nKnowledge Base Stats:")
165
+ for k, v in stats.items():
166
+ print(f" {k}: {v}")
167
+
168
+
169
+ def cmd_delete(args):
170
+ filename = args.filename
171
+ confirm = input(f"Delete all chunks for '{filename}'? [y/N] ").strip().lower()
172
+ if confirm == "y":
173
+ delete_doc(filename)
174
+ print(f"Deleted: {filename}")
175
+ else:
176
+ print("Cancelled.")
177
+
178
+
179
+ # ── Argument parser ───────────────────────────────────────────────────────────
180
+
181
+ def main():
182
+ parser = argparse.ArgumentParser(
183
+ prog="knowledge",
184
+ description="Personal Knowledge Agent β€” RAG over your documents",
185
+ )
186
+ subparsers = parser.add_subparsers(dest="command", required=True)
187
+
188
+ # index
189
+ p_index = subparsers.add_parser("index", help="Index documents into ChromaDB")
190
+ p_index.add_argument("--dir", default=None, help="Documents directory (default: ./documents)")
191
+ p_index.add_argument("--force", action="store_true", help="Force re-index all files")
192
+ p_index.set_defaults(func=cmd_index)
193
+
194
+ # ask
195
+ p_ask = subparsers.add_parser("ask", help="Ask a one-shot question")
196
+ p_ask.add_argument("question", nargs="+", help="Your question (in quotes or multiple words)")
197
+ p_ask.add_argument("--top-k", dest="top_k", type=int, default=5,
198
+ help="Number of chunks to retrieve (default: 5)")
199
+ p_ask.set_defaults(func=cmd_ask)
200
+
201
+ # chat
202
+ p_chat = subparsers.add_parser("chat", help="Interactive chat loop")
203
+ p_chat.add_argument("--top-k", dest="top_k", type=int, default=5)
204
+ p_chat.set_defaults(func=cmd_chat)
205
+
206
+ # list
207
+ p_list = subparsers.add_parser("list", help="List indexed documents")
208
+ p_list.set_defaults(func=cmd_list)
209
+
210
+ # stats
211
+ p_stats = subparsers.add_parser("stats", help="Show knowledge base statistics")
212
+ p_stats.set_defaults(func=cmd_stats)
213
+
214
+ # delete
215
+ p_del = subparsers.add_parser("delete", help="Remove a document from the index")
216
+ p_del.add_argument("filename", help="Filename to delete (e.g. report.pdf)")
217
+ p_del.set_defaults(func=cmd_delete)
218
+
219
+ args = parser.parse_args()
220
+ args.func(args)
221
+
222
+
223
+ if __name__ == "__main__":
224
+ main()
knowledge_agent/indexer.py ADDED
@@ -0,0 +1,317 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ indexer.py β€” Document Indexer for Knowledge Agent
3
+ Loads PDFs, Word docs, .txt, and .md files, chunks them,
4
+ embeds with sentence-transformers, and stores in ChromaDB.
5
+ """
6
+
7
+ import os
8
+ import hashlib
9
+ import json
10
+ from pathlib import Path
11
+ from typing import List, Dict, Optional
12
+ from datetime import datetime
13
+
14
+ import chromadb
15
+ from chromadb.config import Settings
16
+ from sentence_transformers import SentenceTransformer
17
+ from langchain_text_splitters import RecursiveCharacterTextSplitter
18
+
19
+ # ── Document loaders ────────────────────────────────────────────────────────
20
+ import pdfplumber # PDF text extraction
21
+ from docx import Document as DocxDocument # Word docs
22
+ import markdown # Markdown β†’ plain text
23
+ import re
24
+
25
+ # ── Config ───────────────────────────────────────────────────────────────────
26
+ CHROMA_DIR = os.getenv("CHROMA_DIR", "./chroma_db")
27
+ DOCS_DIR = os.getenv("DOCS_DIR", "./documents")
28
+ MODEL_NAME = os.getenv("EMBED_MODEL", "all-MiniLM-L6-v2") # ~80 MB, fast
29
+ COLLECTION = "knowledge_base"
30
+ CHUNK_SIZE = 800 # characters per chunk
31
+ CHUNK_OVERLAP= 120 # overlap so context isn't lost at boundaries
32
+ META_FILE = "./chroma_db/indexed_files.json" # tracks what's been indexed
33
+
34
+ SUPPORTED_EXTENSIONS = {".pdf", ".docx", ".doc", ".txt", ".md", ".markdown"}
35
+
36
+
37
+ # ── Helpers ──────────────────────────────────────────────────────────────────
38
+
39
+ def _file_hash(path: str) -> str:
40
+ """MD5 hash of file contents β€” used to detect changes."""
41
+ h = hashlib.md5()
42
+ with open(path, "rb") as f:
43
+ for chunk in iter(lambda: f.read(8192), b""):
44
+ h.update(chunk)
45
+ return h.hexdigest()
46
+
47
+
48
+ def _load_meta() -> Dict[str, str]:
49
+ """Load the {filepath: hash} index from disk."""
50
+ if os.path.exists(META_FILE):
51
+ with open(META_FILE) as f:
52
+ return json.load(f)
53
+ return {}
54
+
55
+
56
+ def _save_meta(meta: Dict[str, str]):
57
+ os.makedirs(os.path.dirname(META_FILE), exist_ok=True)
58
+ with open(META_FILE, "w") as f:
59
+ json.dump(meta, f, indent=2)
60
+
61
+
62
+ # ── Text extraction ──────────────────────────────────────────────────────────
63
+
64
+ def extract_text_from_pdf(path: str) -> str:
65
+ """Extract all text from a PDF using pdfplumber."""
66
+ text_parts = []
67
+ try:
68
+ with pdfplumber.open(path) as pdf:
69
+ for page in pdf.pages:
70
+ page_text = page.extract_text()
71
+ if page_text:
72
+ text_parts.append(page_text)
73
+ except Exception as e:
74
+ print(f" [WARN] PDF extraction error for {path}: {e}")
75
+ return "\n\n".join(text_parts)
76
+
77
+
78
+ def extract_text_from_docx(path: str) -> str:
79
+ """Extract paragraph text from a .docx file."""
80
+ try:
81
+ doc = DocxDocument(path)
82
+ paragraphs = [p.text for p in doc.paragraphs if p.text.strip()]
83
+ return "\n\n".join(paragraphs)
84
+ except Exception as e:
85
+ print(f" [WARN] DOCX extraction error for {path}: {e}")
86
+ return ""
87
+
88
+
89
+ def extract_text_from_md(path: str) -> str:
90
+ """Convert Markdown to plain text (strips HTML tags)."""
91
+ with open(path, encoding="utf-8") as f:
92
+ raw = f.read()
93
+ html = markdown.markdown(raw)
94
+ # Strip HTML tags
95
+ plain = re.sub(r"<[^>]+>", " ", html)
96
+ return plain.strip()
97
+
98
+
99
+ def extract_text_from_txt(path: str) -> str:
100
+ with open(path, encoding="utf-8", errors="replace") as f:
101
+ return f.read()
102
+
103
+
104
+ def extract_text(path: str) -> str:
105
+ """Route to the correct extractor based on file extension."""
106
+ ext = Path(path).suffix.lower()
107
+ if ext == ".pdf":
108
+ return extract_text_from_pdf(path)
109
+ elif ext in (".docx", ".doc"):
110
+ return extract_text_from_docx(path)
111
+ elif ext in (".md", ".markdown"):
112
+ return extract_text_from_md(path)
113
+ else:
114
+ return extract_text_from_txt(path)
115
+
116
+
117
+ # ── Chunking ─────────────────────────────────────────────────────────────────
118
+
119
+ def chunk_text(text: str) -> List[str]:
120
+ """
121
+ Split text into overlapping chunks using LangChain's splitter.
122
+ Tries to split on paragraphs / sentences first to keep context intact.
123
+ """
124
+ splitter = RecursiveCharacterTextSplitter(
125
+ chunk_size=CHUNK_SIZE,
126
+ chunk_overlap=CHUNK_OVERLAP,
127
+ separators=["\n\n", "\n", ". ", " ", ""],
128
+ )
129
+ return splitter.split_text(text)
130
+
131
+
132
+ # ── Indexer class ──���──────────────────────────────────────────────────────────
133
+
134
+ class KnowledgeIndexer:
135
+ """
136
+ Manages the ChromaDB collection and sentence-transformer embeddings.
137
+ This class is the single source of truth for what's in the vector store.
138
+ """
139
+
140
+ def __init__(self):
141
+ print(f"[Indexer] Loading embedding model: {MODEL_NAME} ...")
142
+ self.embedder = SentenceTransformer(MODEL_NAME)
143
+
144
+ print(f"[Indexer] Connecting to ChromaDB at: {CHROMA_DIR}")
145
+ os.makedirs(CHROMA_DIR, exist_ok=True)
146
+ self.client = chromadb.PersistentClient(path=CHROMA_DIR)
147
+ self.collection = self.client.get_or_create_collection(
148
+ name=COLLECTION,
149
+ metadata={"hnsw:space": "cosine"}, # cosine similarity for RAG
150
+ )
151
+ print(f"[Indexer] Collection '{COLLECTION}' has "
152
+ f"{self.collection.count()} chunks.")
153
+
154
+ # ── Public API ────────────────────────────────────────────────────────────
155
+
156
+ def index_directory(self, docs_dir: str = DOCS_DIR, force: bool = False):
157
+ """
158
+ Walk docs_dir, index any new or changed files.
159
+ Set force=True to re-index everything regardless of hash.
160
+ """
161
+ docs_dir = Path(docs_dir)
162
+ if not docs_dir.exists():
163
+ print(f"[Indexer] Creating documents directory: {docs_dir}")
164
+ docs_dir.mkdir(parents=True)
165
+
166
+ meta = {} if force else _load_meta()
167
+ files = [
168
+ p for p in docs_dir.rglob("*")
169
+ if p.is_file() and p.suffix.lower() in SUPPORTED_EXTENSIONS
170
+ ]
171
+
172
+ if not files:
173
+ print(f"[Indexer] No supported documents found in {docs_dir}")
174
+ return
175
+
176
+ new_count = skipped_count = error_count = 0
177
+ for filepath in files:
178
+ path_str = str(filepath)
179
+ current_hash = _file_hash(path_str)
180
+
181
+ if not force and meta.get(path_str) == current_hash:
182
+ skipped_count += 1
183
+ continue # unchanged file
184
+
185
+ print(f" β†’ Indexing: {filepath.name}")
186
+ try:
187
+ self._index_file(path_str, filepath.name)
188
+ meta[path_str] = current_hash
189
+ new_count += 1
190
+ except Exception as e:
191
+ print(f" [ERROR] Failed to index {filepath.name}: {e}")
192
+ error_count += 1
193
+
194
+ _save_meta(meta)
195
+ print(f"\n[Indexer] Done. "
196
+ f"New/updated: {new_count} | Skipped: {skipped_count} | "
197
+ f"Errors: {error_count} | "
198
+ f"Total chunks in DB: {self.collection.count()}")
199
+
200
+ def index_single_file(self, path: str):
201
+ """Index (or re-index) a single file by path."""
202
+ filepath = Path(path)
203
+ if not filepath.exists():
204
+ raise FileNotFoundError(f"File not found: {path}")
205
+ if filepath.suffix.lower() not in SUPPORTED_EXTENSIONS:
206
+ raise ValueError(f"Unsupported file type: {filepath.suffix}")
207
+
208
+ print(f"[Indexer] Indexing file: {filepath.name}")
209
+ self._index_file(path, filepath.name)
210
+
211
+ # Update meta
212
+ meta = _load_meta()
213
+ meta[path] = _file_hash(path)
214
+ _save_meta(meta)
215
+ print(f"[Indexer] Done. Total chunks: {self.collection.count()}")
216
+
217
+ def query(self, question: str, top_k: int = 5) -> List[Dict]:
218
+ """
219
+ Embed the question and retrieve top_k most similar chunks.
220
+ Returns a list of dicts with keys: text, source, score.
221
+ """
222
+ if self.collection.count() == 0:
223
+ return []
224
+
225
+ q_embedding = self.embedder.encode(question).tolist()
226
+ results = self.collection.query(
227
+ query_embeddings=[q_embedding],
228
+ n_results=min(top_k, self.collection.count()),
229
+ include=["documents", "metadatas", "distances"],
230
+ )
231
+
232
+ chunks = []
233
+ docs = results["documents"][0]
234
+ metadatas = results["metadatas"][0]
235
+ distances = results["distances"][0]
236
+
237
+ for doc, meta, dist in zip(docs, metadatas, distances):
238
+ chunks.append({
239
+ "text": doc,
240
+ "source": meta.get("source", "unknown"),
241
+ "page": meta.get("page", ""),
242
+ "score": round(1 - dist, 4), # cosine similarity (higher = better)
243
+ })
244
+ return chunks
245
+
246
+ def list_documents(self) -> List[Dict]:
247
+ """Return unique source documents currently in the DB."""
248
+ meta = _load_meta()
249
+ docs = []
250
+ for path_str, file_hash in meta.items():
251
+ p = Path(path_str)
252
+ docs.append({
253
+ "name": p.name,
254
+ "path": path_str,
255
+ "hash": file_hash,
256
+ "exists": p.exists(),
257
+ })
258
+ return docs
259
+
260
+ def delete_document(self, filename: str):
261
+ """Remove all chunks belonging to a source file."""
262
+ self.collection.delete(where={"source": filename})
263
+ # Remove from meta
264
+ meta = _load_meta()
265
+ meta = {k: v for k, v in meta.items() if Path(k).name != filename}
266
+ _save_meta(meta)
267
+ print(f"[Indexer] Deleted all chunks for: {filename}")
268
+
269
+ def get_stats(self) -> Dict:
270
+ return {
271
+ "total_chunks": self.collection.count(),
272
+ "indexed_files": len(_load_meta()),
273
+ "chroma_dir": CHROMA_DIR,
274
+ "embed_model": MODEL_NAME,
275
+ }
276
+
277
+ # ── Private ───────────────────────────────────────────────────────────────
278
+
279
+ def _index_file(self, path: str, filename: str):
280
+ """Extract β†’ chunk β†’ embed β†’ upsert into ChromaDB."""
281
+ # 1. Extract text
282
+ text = extract_text(path)
283
+ if not text.strip():
284
+ print(f" [WARN] No text extracted from {filename}")
285
+ return
286
+
287
+ # 2. Chunk
288
+ chunks = chunk_text(text)
289
+ if not chunks:
290
+ return
291
+
292
+ # 3. Remove old chunks for this file (handles re-indexing)
293
+ try:
294
+ self.collection.delete(where={"source": filename})
295
+ except Exception:
296
+ pass # Collection might not have this source yet
297
+
298
+ # 4. Embed + upsert in batches of 100
299
+ BATCH = 100
300
+ for i in range(0, len(chunks), BATCH):
301
+ batch_chunks = chunks[i : i + BATCH]
302
+ embeddings = self.embedder.encode(batch_chunks).tolist()
303
+ ids = [f"{filename}_{i+j}" for j in range(len(batch_chunks))]
304
+ metadatas = [
305
+ {
306
+ "source": filename,
307
+ "chunk_idx": i + j,
308
+ "indexed_at": datetime.now().isoformat(),
309
+ }
310
+ for j in range(len(batch_chunks))
311
+ ]
312
+ self.collection.upsert(
313
+ ids=ids,
314
+ embeddings=embeddings,
315
+ documents=batch_chunks,
316
+ metadatas=metadatas,
317
+ )
knowledge_agent/knowledge_api.py ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ knowledge_api.py β€” Public interface for the Knowledge Agent
3
+ ============================================================
4
+ This module is the integration layer for the Master Orchestrator and any
5
+ other agent that needs to query your personal document store.
6
+
7
+ Usage from another agent:
8
+ from knowledge_agent.knowledge_api import query_knowledge, index_docs
9
+
10
+ It is intentionally stateless at the call level β€” the KnowledgeIndexer
11
+ caches the ChromaDB connection internally for performance.
12
+ """
13
+
14
+ from typing import List, Dict, Optional
15
+ from indexer import KnowledgeIndexer
16
+ from llm import generate_answer, stream_answer
17
+
18
+ # Module-level singleton β€” one DB connection shared across all calls
19
+ # (safe for single-process use; FastAPI uses one process by default)
20
+ _indexer: Optional[KnowledgeIndexer] = None
21
+
22
+
23
+ def _get_indexer() -> KnowledgeIndexer:
24
+ global _indexer
25
+ if _indexer is None:
26
+ _indexer = KnowledgeIndexer()
27
+ return _indexer
28
+
29
+
30
+ # ── Core public functions ─────────────────────────────────────────────────────
31
+
32
+ def query_knowledge(
33
+ question: str,
34
+ top_k: int = 5,
35
+ return_sources: bool = True,
36
+ ) -> Dict:
37
+ """
38
+ PRIMARY ENTRY POINT for the Master Orchestrator and other agents.
39
+
40
+ Ask a question against your indexed personal documents.
41
+
42
+ Args:
43
+ question: Natural language question.
44
+ top_k: Number of chunks to retrieve from ChromaDB.
45
+ return_sources: If True, include source filenames in the response.
46
+
47
+ Returns:
48
+ {
49
+ "answer": str, # LLM-generated, grounded answer
50
+ "sources": [str, ...], # source filenames (if return_sources=True)
51
+ "chunks_used": int, # number of relevant chunks found
52
+ "question": str, # echo back for logging
53
+ }
54
+
55
+ Example:
56
+ >>> result = query_knowledge("What is our refund policy?")
57
+ >>> print(result["answer"])
58
+ """
59
+ indexer = _get_indexer()
60
+ chunks = indexer.query(question, top_k=top_k)
61
+
62
+ if not chunks:
63
+ return {
64
+ "answer": "No documents have been indexed yet. "
65
+ "Please run `index_docs()` first.",
66
+ "sources": [],
67
+ "chunks_used": 0,
68
+ "question": question,
69
+ }
70
+
71
+ result = generate_answer(question, chunks)
72
+ result["question"] = question
73
+
74
+ if not return_sources:
75
+ result.pop("sources", None)
76
+
77
+ return result
78
+
79
+
80
+ def index_docs(docs_dir: str = "./documents", force: bool = False):
81
+ """
82
+ Index (or re-index) all supported documents in docs_dir.
83
+ Called by the CLI, the web UI, and can be called by the Orchestrator
84
+ to trigger a refresh after new documents are added.
85
+
86
+ Args:
87
+ docs_dir: Path to the folder containing your documents.
88
+ force: Re-index everything even if files haven't changed.
89
+ """
90
+ indexer = _get_indexer()
91
+ indexer.index_directory(docs_dir=docs_dir, force=force)
92
+
93
+
94
+ def index_single(path: str):
95
+ """Index a single file immediately (called by file-watcher or API upload)."""
96
+ indexer = _get_indexer()
97
+ indexer.index_single_file(path)
98
+
99
+
100
+ def get_knowledge_stats() -> Dict:
101
+ """Return stats about the current knowledge base (for health checks)."""
102
+ return _get_indexer().get_stats()
103
+
104
+
105
+ def list_indexed_docs() -> List[Dict]:
106
+ """List all documents currently indexed."""
107
+ return _get_indexer().list_documents()
108
+
109
+
110
+ def delete_doc(filename: str):
111
+ """Remove a document and all its chunks from the index."""
112
+ _get_indexer().delete_document(filename)
113
+
114
+
115
+ def stream_knowledge_answer(question: str, top_k: int = 5):
116
+ """
117
+ Streaming variant β€” yields tokens one by one.
118
+ Used by the FastAPI /stream endpoint for the web UI.
119
+ """
120
+ indexer = _get_indexer()
121
+ chunks = indexer.query(question, top_k=top_k)
122
+ if not chunks:
123
+ yield "No documents indexed yet."
124
+ return
125
+ yield from stream_answer(question, chunks)
knowledge_agent/llm.py ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ llm.py β€” Groq-powered RAG answer generation for Knowledge Agent
3
+ Takes a question + retrieved context chunks β†’ returns a grounded answer.
4
+ """
5
+
6
+ import os
7
+ from typing import List, Dict, Optional
8
+ from groq import Groq
9
+
10
+ # ── Config ────────────────────────────────────────────────────────────────────
11
+ GROQ_API_KEY = os.getenv("GROQ_API_KEY", "")
12
+ GROQ_MODEL = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")
13
+ MAX_TOKENS = int(os.getenv("LLM_MAX_TOKENS", "1500"))
14
+ TEMPERATURE = float(os.getenv("LLM_TEMPERATURE", "0.2"))
15
+
16
+ # How many context chunks to pass to the LLM
17
+ TOP_K_CHUNKS = int(os.getenv("TOP_K_CHUNKS", "5"))
18
+
19
+ # Minimum similarity score to include a chunk (0–1, cosine)
20
+ MIN_SCORE = float(os.getenv("MIN_CHUNK_SCORE", "0.30"))
21
+
22
+ # ── System prompt ─────────────────────────────────────────────────────────────
23
+ SYSTEM_PROMPT = """You are a precise, helpful personal knowledge assistant.
24
+ You answer questions STRICTLY based on the document excerpts provided below.
25
+
26
+ Rules:
27
+ - Ground every claim in the provided context. Cite source filenames inline like [source.pdf].
28
+ - If the context doesn't contain enough information, say so clearly β€” do NOT hallucinate.
29
+ - Be concise and structured. Use bullet points or numbered lists when helpful.
30
+ - If asked for a summary, provide a well-organised paragraph-style answer.
31
+ - When multiple documents cover the same topic, synthesise them coherently.
32
+ """
33
+
34
+
35
+ def _build_context_block(chunks: List[Dict]) -> str:
36
+ """Format retrieved chunks into a readable context block for the prompt."""
37
+ filtered = [c for c in chunks if c.get("score", 0) >= MIN_SCORE]
38
+ if not filtered:
39
+ return "No relevant context found in your documents."
40
+
41
+ lines = []
42
+ for i, c in enumerate(filtered, 1):
43
+ source = c.get("source", "unknown")
44
+ score = c.get("score", 0)
45
+ text = c.get("text", "").strip()
46
+ lines.append(f"[Excerpt {i} | Source: {source} | Relevance: {score:.2f}]\n{text}")
47
+
48
+ return "\n\n---\n\n".join(lines)
49
+
50
+
51
+ def _build_user_message(question: str, context: str) -> str:
52
+ return f"""Here are relevant excerpts from your personal documents:
53
+
54
+ {context}
55
+
56
+ ---
57
+
58
+ Question: {question}
59
+
60
+ Answer (cite sources inline):"""
61
+
62
+
63
+ # ── Main function ─────────────────────────────────────────────────────────────
64
+
65
+ def generate_answer(
66
+ question: str,
67
+ chunks: List[Dict],
68
+ conversation_history: Optional[List[Dict]] = None,
69
+ ) -> Dict:
70
+ """
71
+ Generate a grounded answer using Groq + retrieved chunks.
72
+
73
+ Args:
74
+ question: The user's question.
75
+ chunks: List of dicts from KnowledgeIndexer.query()
76
+ Each has keys: text, source, score.
77
+ conversation_history: Optional list of prior {role, content} turns
78
+ for multi-turn chat support.
79
+
80
+ Returns:
81
+ {
82
+ "answer": str,
83
+ "sources": [str, ...], # unique source filenames cited
84
+ "chunks_used": int,
85
+ "model": str,
86
+ }
87
+ """
88
+ if not GROQ_API_KEY:
89
+ raise EnvironmentError(
90
+ "GROQ_API_KEY is not set. Add it to your .env file."
91
+ )
92
+
93
+ client = Groq(api_key=GROQ_API_KEY)
94
+
95
+ # Build context from retrieved chunks
96
+ context = _build_context_block(chunks)
97
+ user_message = _build_user_message(question, context)
98
+
99
+ # Build message list (supports conversational follow-ups)
100
+ messages = [{"role": "system", "content": SYSTEM_PROMPT}]
101
+ if conversation_history:
102
+ messages.extend(conversation_history)
103
+ messages.append({"role": "user", "content": user_message})
104
+
105
+ # ── Call Groq ──────────────────────────────────────────────────────────────
106
+ response = client.chat.completions.create(
107
+ model=GROQ_MODEL,
108
+ messages=messages,
109
+ temperature=TEMPERATURE,
110
+ max_completion_tokens=MAX_TOKENS,
111
+ )
112
+
113
+ answer = response.choices[0].message.content.strip()
114
+
115
+ # Extract unique source filenames from chunks that met the score threshold
116
+ sources = list({
117
+ c["source"]
118
+ for c in chunks
119
+ if c.get("score", 0) >= MIN_SCORE
120
+ })
121
+
122
+ return {
123
+ "answer": answer,
124
+ "sources": sources,
125
+ "chunks_used": len([c for c in chunks if c.get("score", 0) >= MIN_SCORE]),
126
+ "model": GROQ_MODEL,
127
+ }
128
+
129
+
130
+ # ── Streaming variant (used by FastAPI for real-time web UI) ──────────────────
131
+
132
+ def stream_answer(question: str, chunks: List[Dict]):
133
+ """
134
+ Generator that yields answer tokens one by one for SSE streaming.
135
+ Usage: for token in stream_answer(q, chunks): ...
136
+ """
137
+ if not GROQ_API_KEY:
138
+ yield "ERROR: GROQ_API_KEY not set."
139
+ return
140
+
141
+ client = Groq(api_key=GROQ_API_KEY)
142
+ context = _build_context_block(chunks)
143
+ user_message = _build_user_message(question, context)
144
+
145
+ messages = [
146
+ {"role": "system", "content": SYSTEM_PROMPT},
147
+ {"role": "user", "content": user_message},
148
+ ]
149
+
150
+ stream = client.chat.completions.create(
151
+ model=GROQ_MODEL,
152
+ messages=messages,
153
+ temperature=TEMPERATURE,
154
+ max_completion_tokens=MAX_TOKENS,
155
+ stream=True,
156
+ )
157
+
158
+ for chunk in stream:
159
+ if chunk.choices[0].delta.content:
160
+ yield chunk.choices[0].delta.content
knowledge_agent/main.py ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ main.py β€” Knowledge Agent entry point
3
+ ======================================
4
+ Starts:
5
+ 1. File watcher (background thread) β€” auto-indexes new/changed documents
6
+ 2. FastAPI web server (foreground) β€” CLI and web UI
7
+
8
+ Run:
9
+ python main.py # web UI on http://localhost:8000
10
+ python main.py --port 8080 # custom port
11
+ python main.py --no-watcher # disable file watcher
12
+ python main.py --index-on-start # index all docs before starting
13
+ """
14
+
15
+ import argparse
16
+ import os
17
+ import sys
18
+ import uvicorn
19
+
20
+ import os as _os
21
+ from dotenv import load_dotenv
22
+ load_dotenv(_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "..", ".env"))
23
+
24
+ DOCS_DIR = os.getenv("DOCS_DIR", "./documents")
25
+
26
+
27
+ def main():
28
+ parser = argparse.ArgumentParser(
29
+ description="Knowledge Agent β€” Personal RAG server"
30
+ )
31
+ parser.add_argument("--host", default="0.0.0.0",
32
+ help="Bind host (default: 0.0.0.0)")
33
+ parser.add_argument("--port", type=int, default=8000,
34
+ help="Port (default: 8000)")
35
+ parser.add_argument("--no-watcher", action="store_true",
36
+ help="Disable the file-system watcher")
37
+ parser.add_argument("--index-on-start", action="store_true",
38
+ help="Re-index all documents before starting server")
39
+ parser.add_argument("--reload", action="store_true",
40
+ help="Enable uvicorn auto-reload (development)")
41
+ args = parser.parse_args()
42
+
43
+ # ── Optional: index on startup ─────────────────────────────────────────
44
+ if args.index_on_start:
45
+ print("[Main] Indexing documents on startup …")
46
+ from knowledge_api import index_docs
47
+ index_docs(docs_dir=DOCS_DIR)
48
+
49
+ # ── File watcher (background thread) ──────────────────────────────────
50
+ if not args.no_watcher:
51
+ from watcher import start_watcher_thread
52
+ wt = start_watcher_thread(docs_dir=DOCS_DIR)
53
+ print(f"[Main] File watcher started for: {DOCS_DIR}")
54
+
55
+ # ── Start FastAPI ──────────────────────────────────────────────────────
56
+ print(f"\n[Main] Starting Knowledge Agent on http://{args.host}:{args.port}")
57
+ print("[Main] Web UI: http://localhost:{port}".format(port=args.port))
58
+ print("[Main] API docs: http://localhost:{port}/docs\n".format(port=args.port))
59
+
60
+ uvicorn.run(
61
+ "web_app:app",
62
+ host=args.host,
63
+ port=args.port,
64
+ reload=args.reload,
65
+ log_level="info",
66
+ )
67
+
68
+
69
+ if __name__ == "__main__":
70
+ main()
knowledge_agent/requirements.txt ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── Knowledge Agent β€” Python Dependencies ────────────────────────────────────
2
+ # Install: pip install -r requirements.txt
3
+
4
+ # ── Core AI / LLM ─────────────────────────────────────────────────────────────
5
+ groq>=0.9.0 # Groq API client (llama-3.3-70b-versatile)
6
+ sentence-transformers>=2.7.0 # Local embeddings (all-MiniLM-L6-v2, free)
7
+
8
+ # ── Vector Store ──────────────────────────────────────────────────────────────
9
+ chromadb>=0.5.0 # Local persistent vector database
10
+
11
+ # ── Document Loading ──────────────────────────────────────────────────────────
12
+ pdfplumber>=0.11.0 # PDF text extraction
13
+ python-docx>=1.1.0 # Word document (.docx) reader
14
+ markdown>=3.6 # Markdown β†’ plain text conversion
15
+
16
+ # ── Text Splitting ────────────────────────────────────────────────────────────
17
+ langchain-text-splitters>=0.2.0 # RecursiveCharacterTextSplitter
18
+
19
+ # ── Web Server ────────────────────────────────────────────────────────────────
20
+ fastapi>=0.111.0 # REST API + SSE streaming
21
+ uvicorn[standard]>=0.30.0 # ASGI server
22
+ python-multipart>=0.0.9 # File upload support
23
+
24
+ # ── CLI ────────────────────────────────────────────────────────────────────────
25
+ rich>=13.7.0 # Pretty terminal output (optional but nice)
26
+
27
+ # ── File Watcher ──────────────────────────────────────────────────────────────
28
+ watchdog>=4.0.0 # File-system event monitoring
29
+
30
+ # ── Config ────────────────────────────────────────────────────────────────────
31
+ python-dotenv>=1.0.0 # .env file loader
32
+ pydantic>=2.0.0 # Data validation (used by FastAPI)
knowledge_agent/watcher.py ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ watcher.py β€” File-system watcher for auto re-indexing
3
+ ======================================================
4
+ Watches the documents directory for new, modified, or deleted files
5
+ and automatically updates the ChromaDB index.
6
+
7
+ Run standalone: python watcher.py
8
+ Or import: from watcher import start_watcher_thread
9
+ """
10
+
11
+ import os
12
+ import time
13
+ import threading
14
+ from pathlib import Path
15
+
16
+ try:
17
+ from watchdog.observers import Observer
18
+ from watchdog.events import FileSystemEventHandler, FileSystemEvent
19
+ WATCHDOG_AVAILABLE = True
20
+ except ImportError:
21
+ WATCHDOG_AVAILABLE = False
22
+ print("[Watcher] watchdog not installed β€” falling back to polling watcher.")
23
+
24
+ from knowledge_api import index_single, delete_doc
25
+
26
+ DOCS_DIR = os.getenv("DOCS_DIR", "./documents")
27
+ SUPPORTED = {".pdf", ".docx", ".doc", ".txt", ".md", ".markdown"}
28
+
29
+ # Debounce time in seconds (avoid re-indexing for rapid successive saves)
30
+ DEBOUNCE_SECONDS = 2.0
31
+
32
+
33
+ # ── Watchdog handler ──────────────────────────────────────────────────────────
34
+
35
+ class DocumentHandler(FileSystemEventHandler):
36
+ """Handles file-system events and triggers indexing."""
37
+
38
+ def __init__(self):
39
+ self._timers: dict = {} # path β†’ Timer (debounce)
40
+
41
+ def _debounced_index(self, path: str):
42
+ """Cancel any pending timer for this path and set a new one."""
43
+ if path in self._timers:
44
+ self._timers[path].cancel()
45
+ timer = threading.Timer(DEBOUNCE_SECONDS, self._do_index, args=[path])
46
+ self._timers[path] = timer
47
+ timer.start()
48
+
49
+ @staticmethod
50
+ def _do_index(path: str):
51
+ print(f"[Watcher] Indexing: {Path(path).name}")
52
+ try:
53
+ index_single(path)
54
+ print(f"[Watcher] βœ“ Done: {Path(path).name}")
55
+ except Exception as e:
56
+ print(f"[Watcher] βœ— Error indexing {path}: {e}")
57
+
58
+ def on_created(self, event: FileSystemEvent):
59
+ if not event.is_directory and Path(event.src_path).suffix.lower() in SUPPORTED:
60
+ print(f"[Watcher] New file detected: {Path(event.src_path).name}")
61
+ self._debounced_index(event.src_path)
62
+
63
+ def on_modified(self, event: FileSystemEvent):
64
+ if not event.is_directory and Path(event.src_path).suffix.lower() in SUPPORTED:
65
+ print(f"[Watcher] File modified: {Path(event.src_path).name}")
66
+ self._debounced_index(event.src_path)
67
+
68
+ def on_deleted(self, event: FileSystemEvent):
69
+ if not event.is_directory and Path(event.src_path).suffix.lower() in SUPPORTED:
70
+ fname = Path(event.src_path).name
71
+ print(f"[Watcher] File deleted: {fname} β€” removing from index.")
72
+ try:
73
+ delete_doc(fname)
74
+ except Exception as e:
75
+ print(f"[Watcher] Error removing {fname}: {e}")
76
+
77
+ def on_moved(self, event: FileSystemEvent):
78
+ # Treat as delete old + create new
79
+ if not event.is_directory:
80
+ if Path(event.src_path).suffix.lower() in SUPPORTED:
81
+ delete_doc(Path(event.src_path).name)
82
+ if Path(event.dest_path).suffix.lower() in SUPPORTED:
83
+ self._debounced_index(event.dest_path)
84
+
85
+
86
+ # ── Polling fallback ──────────────────────────────────────────────────────────
87
+
88
+ class PollingWatcher:
89
+ """
90
+ Simple polling watcher for when watchdog isn't available.
91
+ Checks file modification times every POLL_INTERVAL seconds.
92
+ """
93
+ POLL_INTERVAL = 10 # seconds
94
+
95
+ def __init__(self, docs_dir: str):
96
+ self.docs_dir = Path(docs_dir)
97
+ self._seen: dict = {} # path β†’ mtime
98
+
99
+ def run(self):
100
+ print(f"[Watcher] Polling {self.docs_dir} every {self.POLL_INTERVAL}s …")
101
+ while True:
102
+ self._poll()
103
+ time.sleep(self.POLL_INTERVAL)
104
+
105
+ def _poll(self):
106
+ current = {}
107
+ for p in self.docs_dir.rglob("*"):
108
+ if p.is_file() and p.suffix.lower() in SUPPORTED:
109
+ current[str(p)] = p.stat().st_mtime
110
+
111
+ # New or modified
112
+ for path, mtime in current.items():
113
+ if path not in self._seen or self._seen[path] != mtime:
114
+ print(f"[Watcher] Change detected: {Path(path).name}")
115
+ try:
116
+ index_single(path)
117
+ except Exception as e:
118
+ print(f"[Watcher] Error: {e}")
119
+
120
+ # Deleted
121
+ for path in set(self._seen) - set(current):
122
+ fname = Path(path).name
123
+ print(f"[Watcher] Deleted: {fname}")
124
+ try:
125
+ delete_doc(fname)
126
+ except Exception as e:
127
+ print(f"[Watcher] Error removing {fname}: {e}")
128
+
129
+ self._seen = current
130
+
131
+
132
+ # ── Public API ────────────────���───────────────────────────────────────────────
133
+
134
+ def start_watcher(docs_dir: str = DOCS_DIR, blocking: bool = True):
135
+ """
136
+ Start the file watcher.
137
+ Set blocking=False to run in a background thread (used by web_app.py).
138
+ """
139
+ os.makedirs(docs_dir, exist_ok=True)
140
+
141
+ if WATCHDOG_AVAILABLE:
142
+ handler = DocumentHandler()
143
+ observer = Observer()
144
+ observer.schedule(handler, path=docs_dir, recursive=True)
145
+ observer.start()
146
+ print(f"[Watcher] Watching {docs_dir} with watchdog …")
147
+ try:
148
+ if blocking:
149
+ while True:
150
+ time.sleep(1)
151
+ # Non-blocking: caller is responsible for keeping process alive
152
+ except KeyboardInterrupt:
153
+ observer.stop()
154
+ print("[Watcher] Stopped.")
155
+ observer.join()
156
+ else:
157
+ pw = PollingWatcher(docs_dir)
158
+ if blocking:
159
+ pw.run()
160
+ else:
161
+ t = threading.Thread(target=pw.run, daemon=True)
162
+ t.start()
163
+ return t
164
+
165
+
166
+ def start_watcher_thread(docs_dir: str = DOCS_DIR) -> threading.Thread:
167
+ """
168
+ Start the watcher in a background daemon thread.
169
+ Returns the thread (for monitoring only β€” it's a daemon so it stops with the process).
170
+ """
171
+ t = threading.Thread(target=start_watcher, args=(docs_dir, True), daemon=True)
172
+ t.start()
173
+ return t
174
+
175
+
176
+ if __name__ == "__main__":
177
+ start_watcher(DOCS_DIR, blocking=True)
knowledge_agent/web_app.py ADDED
@@ -0,0 +1,658 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ web_app.py β€” FastAPI web UI + REST API for the Knowledge Agent
3
+ ==============================================================
4
+ Run with: uvicorn web_app:app --host 0.0.0.0 --port 8000 --reload
5
+
6
+ Endpoints:
7
+ GET / β†’ Web UI (chat interface)
8
+ POST /api/ask β†’ JSON Q&A
9
+ GET /api/stream β†’ SSE streaming answer
10
+ POST /api/index β†’ Trigger re-indexing
11
+ POST /api/upload β†’ Upload and index a new document
12
+ GET /api/docs-list β†’ List indexed documents
13
+ GET /api/stats β†’ Knowledge base stats
14
+ DELETE /api/doc/{filename} β†’ Remove a document
15
+ """
16
+
17
+ import os
18
+ import shutil
19
+ import asyncio
20
+ from pathlib import Path
21
+ from typing import Optional
22
+
23
+ from fastapi import FastAPI, UploadFile, File, HTTPException, Query
24
+ from fastapi.responses import HTMLResponse, StreamingResponse, JSONResponse
25
+ from fastapi.middleware.cors import CORSMiddleware
26
+ from pydantic import BaseModel
27
+
28
+ from knowledge_api import (
29
+ query_knowledge,
30
+ index_docs,
31
+ index_single,
32
+ get_knowledge_stats,
33
+ list_indexed_docs,
34
+ delete_doc,
35
+ stream_knowledge_answer,
36
+ )
37
+
38
+ DOCS_DIR = os.getenv("DOCS_DIR", "./documents")
39
+
40
+ # ── App setup ─────────────────────────────────────────────────────────────────
41
+ app = FastAPI(
42
+ title="Knowledge Agent",
43
+ description="Personal RAG system over your documents",
44
+ version="1.0.0",
45
+ )
46
+
47
+ app.add_middleware(
48
+ CORSMiddleware,
49
+ allow_origins=["*"],
50
+ allow_methods=["*"],
51
+ allow_headers=["*"],
52
+ )
53
+
54
+
55
+ # ── Request / Response models ─────────────────────────────────────────────────
56
+ class AskRequest(BaseModel):
57
+ question: str
58
+ top_k: int = 5
59
+
60
+ class IndexRequest(BaseModel):
61
+ docs_dir: Optional[str] = None
62
+ force: bool = False
63
+
64
+
65
+ # ── REST API endpoints ────────────────────────────────────────────────────────
66
+
67
+ @app.post("/api/ask")
68
+ async def ask(req: AskRequest):
69
+ """Ask a question and get a grounded answer from your documents."""
70
+ if not req.question.strip():
71
+ raise HTTPException(400, "Question cannot be empty.")
72
+ result = query_knowledge(req.question, top_k=req.top_k)
73
+ return result
74
+
75
+
76
+ @app.get("/api/stream")
77
+ async def stream(
78
+ question: str = Query(..., description="Your question"),
79
+ top_k: int = Query(5, description="Number of chunks to retrieve"),
80
+ ):
81
+ """
82
+ SSE endpoint β€” streams the answer token by token.
83
+ Used by the web UI for a typing-cursor effect.
84
+ """
85
+ if not question.strip():
86
+ raise HTTPException(400, "Question cannot be empty.")
87
+
88
+ def event_generator():
89
+ for token in stream_knowledge_answer(question, top_k=top_k):
90
+ # SSE format: "data: <token>\n\n"
91
+ yield f"data: {token}\n\n"
92
+ yield "data: [DONE]\n\n"
93
+
94
+ return StreamingResponse(
95
+ event_generator(),
96
+ media_type="text/event-stream",
97
+ headers={
98
+ "Cache-Control": "no-cache",
99
+ "X-Accel-Buffering": "no",
100
+ },
101
+ )
102
+
103
+
104
+ @app.post("/api/index")
105
+ async def trigger_index(req: IndexRequest):
106
+ """Trigger indexing of the documents directory."""
107
+ docs_dir = req.docs_dir or DOCS_DIR
108
+ # Run in a thread pool so we don't block the event loop
109
+ loop = asyncio.get_event_loop()
110
+ await loop.run_in_executor(
111
+ None, lambda: index_docs(docs_dir=docs_dir, force=req.force)
112
+ )
113
+ stats = get_knowledge_stats()
114
+ return {"message": "Indexing complete.", "stats": stats}
115
+
116
+
117
+ @app.post("/api/upload")
118
+ async def upload_document(file: UploadFile = File(...)):
119
+ """
120
+ Upload a document file; save it to the documents dir and index it.
121
+ Supports: .pdf, .docx, .txt, .md
122
+ """
123
+ allowed = {".pdf", ".docx", ".doc", ".txt", ".md", ".markdown"}
124
+ ext = Path(file.filename).suffix.lower()
125
+ if ext not in allowed:
126
+ raise HTTPException(400, f"Unsupported file type: {ext}. Allowed: {allowed}")
127
+
128
+ os.makedirs(DOCS_DIR, exist_ok=True)
129
+ dest = os.path.join(DOCS_DIR, file.filename)
130
+
131
+ # Save file
132
+ with open(dest, "wb") as f:
133
+ shutil.copyfileobj(file.file, f)
134
+
135
+ # Index it
136
+ loop = asyncio.get_event_loop()
137
+ await loop.run_in_executor(None, lambda: index_single(dest))
138
+
139
+ return {
140
+ "message": f"'{file.filename}' uploaded and indexed.",
141
+ "path": dest,
142
+ }
143
+
144
+
145
+ @app.get("/api/docs-list")
146
+ async def docs_list():
147
+ """List all documents currently in the knowledge base."""
148
+ return {"documents": list_indexed_docs()}
149
+
150
+
151
+ @app.get("/api/stats")
152
+ async def stats():
153
+ """Return knowledge base statistics."""
154
+ return get_knowledge_stats()
155
+
156
+
157
+ @app.delete("/api/doc/{filename}")
158
+ async def remove_doc(filename: str):
159
+ """Remove a document and all its chunks from the index."""
160
+ delete_doc(filename)
161
+ return {"message": f"Deleted '{filename}' from the knowledge base."}
162
+
163
+
164
+ # ── Web UI ────────────────────────────────────────────────────────────────────
165
+
166
+ @app.get("/", response_class=HTMLResponse)
167
+ async def ui():
168
+ """Serve the single-page chat interface."""
169
+ return HTMLResponse(content=WEB_UI_HTML)
170
+
171
+
172
+ # ── HTML for the web UI (self-contained, no build step needed) ────────────────
173
+ WEB_UI_HTML = """<!DOCTYPE html>
174
+ <html lang="en">
175
+ <head>
176
+ <meta charset="UTF-8">
177
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
178
+ <title>Knowledge Agent</title>
179
+ <link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500&family=IBM+Plex+Sans:wght@300;400;600&display=swap" rel="stylesheet">
180
+ <style>
181
+ :root {
182
+ --bg: #0d0f14;
183
+ --surface: #161921;
184
+ --border: #2a2f3d;
185
+ --accent: #5b8def;
186
+ --accent2: #7ecfb0;
187
+ --text: #d4dae8;
188
+ --muted: #5c6480;
189
+ --danger: #e05c7a;
190
+ --mono: 'IBM Plex Mono', monospace;
191
+ --sans: 'IBM Plex Sans', sans-serif;
192
+ }
193
+ * { box-sizing: border-box; margin: 0; padding: 0; }
194
+ body {
195
+ background: var(--bg);
196
+ color: var(--text);
197
+ font-family: var(--sans);
198
+ height: 100vh;
199
+ display: flex;
200
+ flex-direction: column;
201
+ }
202
+
203
+ /* ── Header ── */
204
+ header {
205
+ display: flex;
206
+ align-items: center;
207
+ justify-content: space-between;
208
+ padding: 14px 24px;
209
+ border-bottom: 1px solid var(--border);
210
+ background: var(--surface);
211
+ flex-shrink: 0;
212
+ }
213
+ .logo {
214
+ font-family: var(--mono);
215
+ font-size: 1rem;
216
+ color: var(--accent);
217
+ letter-spacing: .04em;
218
+ }
219
+ .logo span { color: var(--accent2); }
220
+ .header-actions { display: flex; gap: 10px; }
221
+ .btn {
222
+ font-family: var(--mono);
223
+ font-size: .75rem;
224
+ padding: 6px 14px;
225
+ border: 1px solid var(--border);
226
+ border-radius: 4px;
227
+ background: transparent;
228
+ color: var(--text);
229
+ cursor: pointer;
230
+ transition: all .15s;
231
+ }
232
+ .btn:hover { border-color: var(--accent); color: var(--accent); }
233
+ .btn.primary { background: var(--accent); border-color: var(--accent); color: #fff; }
234
+ .btn.primary:hover { opacity: .85; }
235
+ .btn.danger { border-color: var(--danger); color: var(--danger); }
236
+
237
+ /* ── Layout ── */
238
+ .main { display: flex; flex: 1; overflow: hidden; }
239
+
240
+ /* ── Sidebar ── */
241
+ .sidebar {
242
+ width: 260px;
243
+ border-right: 1px solid var(--border);
244
+ background: var(--surface);
245
+ display: flex;
246
+ flex-direction: column;
247
+ flex-shrink: 0;
248
+ overflow: hidden;
249
+ }
250
+ .sidebar-section { padding: 16px; border-bottom: 1px solid var(--border); }
251
+ .sidebar-title {
252
+ font-family: var(--mono);
253
+ font-size: .65rem;
254
+ text-transform: uppercase;
255
+ letter-spacing: .12em;
256
+ color: var(--muted);
257
+ margin-bottom: 10px;
258
+ }
259
+ #stats-panel { font-size: .78rem; line-height: 1.9; color: var(--muted); }
260
+ #stats-panel strong { color: var(--text); }
261
+ .doc-list { flex: 1; overflow-y: auto; padding: 8px 0; }
262
+ .doc-item {
263
+ padding: 8px 16px;
264
+ font-size: .78rem;
265
+ color: var(--muted);
266
+ display: flex;
267
+ align-items: center;
268
+ gap: 8px;
269
+ cursor: default;
270
+ }
271
+ .doc-item .dot { width: 6px; height: 6px; border-radius: 50%; background: var(--accent2); flex-shrink: 0; }
272
+ .doc-item .del {
273
+ margin-left: auto;
274
+ color: var(--danger);
275
+ cursor: pointer;
276
+ font-size: .9rem;
277
+ opacity: 0;
278
+ transition: opacity .15s;
279
+ }
280
+ .doc-item:hover .del { opacity: 1; }
281
+
282
+ /* Upload zone */
283
+ .upload-zone {
284
+ margin: 0 16px 16px;
285
+ border: 1px dashed var(--border);
286
+ border-radius: 6px;
287
+ padding: 12px;
288
+ text-align: center;
289
+ font-size: .75rem;
290
+ color: var(--muted);
291
+ cursor: pointer;
292
+ transition: border-color .15s;
293
+ }
294
+ .upload-zone:hover { border-color: var(--accent); color: var(--accent); }
295
+ #file-input { display: none; }
296
+
297
+ /* ── Chat area ── */
298
+ .chat-area {
299
+ flex: 1;
300
+ display: flex;
301
+ flex-direction: column;
302
+ overflow: hidden;
303
+ }
304
+ .messages {
305
+ flex: 1;
306
+ overflow-y: auto;
307
+ padding: 28px 32px;
308
+ display: flex;
309
+ flex-direction: column;
310
+ gap: 24px;
311
+ }
312
+ .msg { display: flex; gap: 14px; }
313
+ .msg-avatar {
314
+ width: 30px; height: 30px;
315
+ border-radius: 4px;
316
+ display: flex; align-items: center; justify-content: center;
317
+ font-family: var(--mono);
318
+ font-size: .65rem;
319
+ font-weight: 500;
320
+ flex-shrink: 0;
321
+ margin-top: 2px;
322
+ }
323
+ .msg.user .msg-avatar { background: #2b3354; color: var(--accent); }
324
+ .msg.agent .msg-avatar { background: #1e3330; color: var(--accent2); }
325
+ .msg-body { flex: 1; }
326
+ .msg-label {
327
+ font-size: .68rem;
328
+ font-family: var(--mono);
329
+ color: var(--muted);
330
+ margin-bottom: 5px;
331
+ text-transform: uppercase;
332
+ letter-spacing: .08em;
333
+ }
334
+ .msg-text {
335
+ font-size: .875rem;
336
+ line-height: 1.75;
337
+ color: var(--text);
338
+ white-space: pre-wrap;
339
+ word-break: break-word;
340
+ }
341
+ .msg.user .msg-text { color: #aebcdc; }
342
+ .sources-bar {
343
+ margin-top: 10px;
344
+ font-size: .7rem;
345
+ font-family: var(--mono);
346
+ color: var(--muted);
347
+ }
348
+ .sources-bar span {
349
+ display: inline-block;
350
+ background: #1a2238;
351
+ border: 1px solid var(--border);
352
+ border-radius: 3px;
353
+ padding: 2px 8px;
354
+ margin: 2px 3px 0 0;
355
+ color: var(--accent);
356
+ }
357
+
358
+ /* ── Input row ── */
359
+ .input-row {
360
+ padding: 18px 32px;
361
+ border-top: 1px solid var(--border);
362
+ display: flex;
363
+ gap: 10px;
364
+ background: var(--surface);
365
+ flex-shrink: 0;
366
+ }
367
+ #question-input {
368
+ flex: 1;
369
+ background: var(--bg);
370
+ border: 1px solid var(--border);
371
+ border-radius: 6px;
372
+ padding: 12px 16px;
373
+ font-family: var(--sans);
374
+ font-size: .875rem;
375
+ color: var(--text);
376
+ resize: none;
377
+ outline: none;
378
+ transition: border-color .15s;
379
+ line-height: 1.5;
380
+ min-height: 48px;
381
+ max-height: 160px;
382
+ }
383
+ #question-input:focus { border-color: var(--accent); }
384
+ #question-input::placeholder { color: var(--muted); }
385
+ #send-btn {
386
+ align-self: flex-end;
387
+ height: 48px;
388
+ padding: 0 22px;
389
+ }
390
+
391
+ /* ── Status bar ── */
392
+ #status-bar {
393
+ padding: 6px 32px;
394
+ font-size: .7rem;
395
+ font-family: var(--mono);
396
+ color: var(--muted);
397
+ background: var(--bg);
398
+ border-top: 1px solid var(--border);
399
+ flex-shrink: 0;
400
+ }
401
+
402
+ /* Scrollbar */
403
+ ::-webkit-scrollbar { width: 6px; }
404
+ ::-webkit-scrollbar-track { background: transparent; }
405
+ ::-webkit-scrollbar-thumb { background: var(--border); border-radius: 3px; }
406
+
407
+ /* Loading dots */
408
+ .typing { display: flex; gap: 5px; align-items: center; padding: 4px 0; }
409
+ .typing .dot { width: 6px; height: 6px; border-radius: 50%; background: var(--accent); animation: blink 1.2s infinite; }
410
+ .typing .dot:nth-child(2) { animation-delay: .2s; }
411
+ .typing .dot:nth-child(3) { animation-delay: .4s; }
412
+ @keyframes blink { 0%,80%,100%{opacity:.2} 40%{opacity:1} }
413
+
414
+ .empty-state {
415
+ flex: 1;
416
+ display: flex;
417
+ flex-direction: column;
418
+ align-items: center;
419
+ justify-content: center;
420
+ color: var(--muted);
421
+ font-size: .875rem;
422
+ gap: 10px;
423
+ text-align: center;
424
+ }
425
+ .empty-state .icon { font-size: 2.5rem; margin-bottom: 4px; }
426
+ </style>
427
+ </head>
428
+ <body>
429
+
430
+ <header>
431
+ <div class="logo">knowledge<span>_agent</span></div>
432
+ <div class="header-actions">
433
+ <button class="btn" onclick="loadStats()">↻ Refresh</button>
434
+ <button class="btn primary" onclick="triggerIndex()">⚑ Re-index</button>
435
+ </div>
436
+ </header>
437
+
438
+ <div class="main">
439
+ <!-- Sidebar -->
440
+ <div class="sidebar">
441
+ <div class="sidebar-section">
442
+ <div class="sidebar-title">Knowledge Base</div>
443
+ <div id="stats-panel">Loading…</div>
444
+ </div>
445
+ <div class="sidebar-section" style="flex:1;overflow:hidden;display:flex;flex-direction:column;padding-bottom:0">
446
+ <div class="sidebar-title">Indexed Documents</div>
447
+ <div class="doc-list" id="doc-list"></div>
448
+ </div>
449
+ <div class="upload-zone" onclick="document.getElementById('file-input').click()">
450
+ οΌ‹ Upload document
451
+ </div>
452
+ <input type="file" id="file-input" accept=".pdf,.docx,.txt,.md,.markdown"
453
+ onchange="uploadFile(this.files[0])">
454
+ </div>
455
+
456
+ <!-- Chat -->
457
+ <div class="chat-area">
458
+ <div class="messages" id="messages">
459
+ <div class="empty-state" id="empty-state">
460
+ <div class="icon">🧠</div>
461
+ <strong>Ask anything about your documents</strong>
462
+ <span>Add files via the sidebar, then start asking questions.</span>
463
+ </div>
464
+ </div>
465
+
466
+ <div class="input-row">
467
+ <textarea id="question-input"
468
+ placeholder="Ask a question about your documents…"
469
+ rows="1"
470
+ onkeydown="handleKey(event)"></textarea>
471
+ <button class="btn primary" id="send-btn" onclick="sendQuestion()">Ask β†’</button>
472
+ </div>
473
+ <div id="status-bar">Ready</div>
474
+ </div>
475
+ </div>
476
+
477
+ <script>
478
+ // ── State ──────────────────────────────────────────────────────────────────
479
+ let isLoading = false;
480
+
481
+ // ── On load ────────────────────────────────────────────────────────────────
482
+ window.addEventListener('DOMContentLoaded', () => {
483
+ loadStats();
484
+ loadDocs();
485
+ autoResize(document.getElementById('question-input'));
486
+ });
487
+
488
+ // ── Stats & docs ───────────────────────────────────────────────────────────
489
+ async function loadStats() {
490
+ try {
491
+ const r = await fetch('/api/stats');
492
+ const s = await r.json();
493
+ document.getElementById('stats-panel').innerHTML =
494
+ `<strong>${s.total_chunks}</strong> chunks indexed<br>
495
+ <strong>${s.indexed_files}</strong> documents<br>
496
+ <span style="font-size:.7rem">${s.embed_model}</span>`;
497
+ } catch(e) {
498
+ document.getElementById('stats-panel').textContent = 'Error loading stats.';
499
+ }
500
+ }
501
+
502
+ async function loadDocs() {
503
+ try {
504
+ const r = await fetch('/api/docs-list');
505
+ const data = await r.json();
506
+ const list = document.getElementById('doc-list');
507
+ list.innerHTML = '';
508
+ if (!data.documents.length) {
509
+ list.innerHTML = '<div class="doc-item" style="font-style:italic">No documents yet</div>';
510
+ return;
511
+ }
512
+ data.documents.forEach(doc => {
513
+ const div = document.createElement('div');
514
+ div.className = 'doc-item';
515
+ div.innerHTML = `
516
+ <span class="dot"></span>
517
+ <span style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap"
518
+ title="${doc.path}">${doc.name}</span>
519
+ <span class="del" onclick="removeDoc('${doc.name}', this)" title="Remove">βœ•</span>`;
520
+ list.appendChild(div);
521
+ });
522
+ } catch(e) {}
523
+ }
524
+
525
+ async function removeDoc(name, el) {
526
+ if (!confirm(`Remove "${name}" from the knowledge base?`)) return;
527
+ await fetch(`/api/doc/${encodeURIComponent(name)}`, { method: 'DELETE' });
528
+ el.closest('.doc-item').remove();
529
+ loadStats();
530
+ }
531
+
532
+ async function triggerIndex() {
533
+ setStatus('Indexing documents…');
534
+ try {
535
+ const r = await fetch('/api/index', { method: 'POST',
536
+ headers: {'Content-Type':'application/json'},
537
+ body: JSON.stringify({ force: false })
538
+ });
539
+ const d = await r.json();
540
+ setStatus(`Indexing complete β€” ${d.stats.total_chunks} chunks.`);
541
+ loadStats(); loadDocs();
542
+ } catch(e) { setStatus('Indexing failed.'); }
543
+ }
544
+
545
+ async function uploadFile(file) {
546
+ if (!file) return;
547
+ setStatus(`Uploading ${file.name}…`);
548
+ const fd = new FormData();
549
+ fd.append('file', file);
550
+ try {
551
+ const r = await fetch('/api/upload', { method: 'POST', body: fd });
552
+ const d = await r.json();
553
+ setStatus(d.message);
554
+ loadStats(); loadDocs();
555
+ } catch(e) { setStatus('Upload failed.'); }
556
+ document.getElementById('file-input').value = '';
557
+ }
558
+
559
+ // ── Chat ───────────────────────────────────────────────────────────────────
560
+ function handleKey(e) {
561
+ if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); sendQuestion(); }
562
+ setTimeout(() => autoResize(e.target), 0);
563
+ }
564
+
565
+ function autoResize(el) {
566
+ el.style.height = 'auto';
567
+ el.style.height = Math.min(el.scrollHeight, 160) + 'px';
568
+ }
569
+
570
+ async function sendQuestion() {
571
+ const input = document.getElementById('question-input');
572
+ const question = input.value.trim();
573
+ if (!question || isLoading) return;
574
+
575
+ isLoading = true;
576
+ input.value = '';
577
+ input.style.height = '48px';
578
+ document.getElementById('send-btn').disabled = true;
579
+ document.getElementById('empty-state')?.remove();
580
+
581
+ addMessage('user', question);
582
+ const agentMsg = addMessage('agent', '', true);
583
+ setStatus('Searching knowledge base…');
584
+
585
+ // SSE streaming
586
+ try {
587
+ const url = `/api/stream?question=${encodeURIComponent(question)}&top_k=5`;
588
+ const es = new EventSource(url);
589
+ let buffer = '';
590
+
591
+ es.onmessage = (e) => {
592
+ if (e.data === '[DONE]') {
593
+ es.close();
594
+ finishMessage(agentMsg, buffer);
595
+ isLoading = false;
596
+ document.getElementById('send-btn').disabled = false;
597
+ setStatus('Ready');
598
+ return;
599
+ }
600
+ buffer += e.data;
601
+ agentMsg.querySelector('.msg-text').textContent = buffer;
602
+ scrollToBottom();
603
+ };
604
+ es.onerror = () => {
605
+ es.close();
606
+ if (!buffer) {
607
+ agentMsg.querySelector('.msg-text').textContent =
608
+ 'Error: Could not reach the server.';
609
+ }
610
+ isLoading = false;
611
+ document.getElementById('send-btn').disabled = false;
612
+ setStatus('Error');
613
+ };
614
+ } catch(e) {
615
+ agentMsg.querySelector('.msg-text').textContent = 'Unexpected error.';
616
+ isLoading = false;
617
+ document.getElementById('send-btn').disabled = false;
618
+ setStatus('Error');
619
+ }
620
+ }
621
+
622
+ function addMessage(role, text, loading = false) {
623
+ const container = document.getElementById('messages');
624
+ const div = document.createElement('div');
625
+ div.className = `msg ${role}`;
626
+ const label = role === 'user' ? 'You' : 'Agent';
627
+ const avatarText = role === 'user' ? 'YOU' : 'KB';
628
+ div.innerHTML = `
629
+ <div class="msg-avatar">${avatarText}</div>
630
+ <div class="msg-body">
631
+ <div class="msg-label">${label}</div>
632
+ <div class="msg-text">${loading ? '<div class="typing"><div class="dot"></div><div class="dot"></div><div class="dot"></div></div>' : escapeHtml(text)}</div>
633
+ </div>`;
634
+ container.appendChild(div);
635
+ scrollToBottom();
636
+ return div;
637
+ }
638
+
639
+ function finishMessage(div, text) {
640
+ div.querySelector('.msg-text').textContent = text;
641
+ scrollToBottom();
642
+ }
643
+
644
+ function scrollToBottom() {
645
+ const c = document.getElementById('messages');
646
+ c.scrollTop = c.scrollHeight;
647
+ }
648
+
649
+ function setStatus(msg) {
650
+ document.getElementById('status-bar').textContent = msg;
651
+ }
652
+
653
+ function escapeHtml(s) {
654
+ return s.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
655
+ }
656
+ </script>
657
+ </body>
658
+ </html>"""
linkedin_agent/README.md ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # LinkedIn Agent β€” Personal AI OS
2
+
3
+ Monitors your LinkedIn feed, drafts comments and connection messages,
4
+ and generates high-quality post ideas daily.
5
+
6
+ ## What it does
7
+
8
+ | Capability | Detail |
9
+ |---|---|
10
+ | **Feed analysis** | Scans feed, finds high-engagement posts worth commenting on |
11
+ | **Smart comments** | Drafts genuine, value-adding comments (never "Great post!") |
12
+ | **Connection messages** | Personalised 300-char connection notes |
13
+ | **Post drafts** | 6 post formats: thought leadership, how-to, story, list, hot take, celebration |
14
+ | **Hook scoring** | Rates each post 1-10 on hook, value, authenticity, engagement |
15
+ | **Engagement plan** | Tells you exactly who to engage with + why + how long it'll take |
16
+ | **4 outputs** | HTML email Β· WhatsApp Β· Notion DB Β· Markdown drafts file |
17
+ | **CLI** | One-off commands for posts, connections, feed |
18
+
19
+ ---
20
+
21
+ ## File structure
22
+
23
+ ```
24
+ linkedin_agent/
25
+ β”œβ”€β”€ main_agent.py # Orchestrator, scheduler, CLI
26
+ β”œβ”€β”€ data_fetcher.py # RapidAPI + Serper fallback + mock mode
27
+ β”œβ”€β”€ llm.py # 6 Groq prompts: analysis, plan, comment, connect, post, score
28
+ β”œβ”€β”€ linkedin_store.py # Local JSON: drafts, history, connections, feed snapshots
29
+ β”œβ”€β”€ delivery.py # Email HTML, WhatsApp, Notion, Markdown
30
+ β”œβ”€β”€ .env.example
31
+ β”œβ”€β”€ requirements.txt
32
+ └── README.md
33
+ ```
34
+
35
+ ---
36
+
37
+ ## Setup
38
+
39
+ ```bash
40
+ pip install -r requirements.txt
41
+ cp .env.example .env
42
+ # Fill in your details
43
+ ```
44
+
45
+ ### 3 data source options
46
+
47
+ **Option A β€” RapidAPI (real LinkedIn data)**
48
+ 1. Go to [rapidapi.com](https://rapidapi.com)
49
+ 2. Search "LinkedIn Data API" β€” subscribe to free tier
50
+ 3. Add `RAPIDAPI_KEY` to your `.env`
51
+
52
+ **Option B β€” Serper only (public post search)**
53
+ 1. Get free key at [serper.dev](https://serper.dev)
54
+ 2. Searches Google for LinkedIn posts in your niche
55
+ 3. No LinkedIn login needed
56
+
57
+ **Option C β€” Mock mode (development/testing)**
58
+ ```env
59
+ LINKEDIN_MOCK_MODE=true
60
+ ```
61
+ Uses realistic dummy data β€” great for testing the full pipeline without any API keys.
62
+
63
+ ---
64
+
65
+ ## Usage
66
+
67
+ ```bash
68
+ # Daily digest (feed + engagement plan + post idea + delivery)
69
+ python main_agent.py --feed
70
+
71
+ # Draft connection message for someone
72
+ python main_agent.py --connect "Priya Sharma" --role "CTO" --company "Zepto"
73
+
74
+ # Draft a specific type of post
75
+ python main_agent.py --draft-post --post-type thought_leadership
76
+ python main_agent.py --draft-post --post-type story --topic "my first startup failure"
77
+ python main_agent.py --draft-post --post-type how_to --topic "building AI agents"
78
+
79
+ # Run as daemon (daily at 9:30 AM)
80
+ python main_agent.py --daemon
81
+ ```
82
+
83
+ ---
84
+
85
+ ## Post types
86
+
87
+ | Type | Format | Best for |
88
+ |---|---|---|
89
+ | `thought_leadership` | Bold claim + insight + question | Establishing expertise |
90
+ | `how_to` | Numbered steps, practical | Getting saves and shares |
91
+ | `story` | Personal narrative arc | Building connection |
92
+ | `list` | "X things I wish I knew" | Easy to consume |
93
+ | `hot_take` | Contrarian + reasoned | Debate and comments |
94
+ | `celebration` | Milestone + journey + lesson | Announcements |
95
+
96
+ ---
97
+
98
+ ## Comment strategy
99
+
100
+ The agent drafts comments with one of 5 angles:
101
+
102
+ | Angle | Style |
103
+ |---|---|
104
+ | ⚑ `contrarian` | Respectful pushback with a reason |
105
+ | πŸ‘ `supportive` | Adds data, example, or extends the idea |
106
+ | πŸ’‘ `add_value` | Brings a new angle or related insight |
107
+ | ❓ `ask_question` | Genuine curiosity about their experience |
108
+ | πŸ—£οΈ `share_experience` | "This reminded me of when I…" |
109
+
110
+ Rules baked into the prompt: no sycophantic openers, never starts with "I", max 4 sentences, sounds human.
111
+
112
+ ---
113
+
114
+ ## Personalisation
115
+
116
+ Fill these in `.env` for best results β€” they're injected into every prompt:
117
+
118
+ ```env
119
+ LINKEDIN_USER_NAME=Raj Patel
120
+ LINKEDIN_USER_ROLE=Founder & CEO at BuildFast
121
+ LINKEDIN_USER_INDUSTRY=B2B SaaS / AI Tools
122
+ LINKEDIN_NICHE_TOPICS=AI agents, developer tools, bootstrapping
123
+ LINKEDIN_WRITING_TONE=direct, slightly contrarian, data-driven
124
+ LINKEDIN_GOAL=reach 5000 followers and get 3 inbound leads/month
125
+ ```
126
+
127
+ ---
128
+
129
+ ## Integration with Personal AI OS
130
+
131
+ ```
132
+ 01 βœ… Daily Planner Agent
133
+ 02 βœ… Email Agent
134
+ 03 βœ… Meeting Prep Agent
135
+ 04 βœ… End-of-Day Review Agent
136
+ 05 βœ… Task Manager Agent
137
+ 06 βœ… Research Agent
138
+ 07 βœ… Finance Agent
139
+ 08 βœ… LinkedIn Agent ← YOU ARE HERE
140
+ 09 Knowledge Agent
141
+ 10 Master Orchestrator (Mem0 + LangGraph)
142
+ ```
linkedin_agent/data_fetcher.py ADDED
@@ -0,0 +1,352 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ data_fetcher.py β€” LinkedIn Agent
3
+ ==================================
4
+ Fetches LinkedIn data via:
5
+ 1. RapidAPI LinkedIn scraper (primary β€” needs RAPIDAPI_KEY)
6
+ 2. Serper Google search fallback (for public profile info)
7
+ 3. Local mock data for development/testing
8
+
9
+ Note: LinkedIn blocks direct scraping. This agent uses the
10
+ Fresh LinkedIn Profile Data API on RapidAPI (freemium).
11
+ """
12
+
13
+ import os
14
+ import json
15
+ import logging
16
+ import requests
17
+ from datetime import datetime
18
+ from typing import Optional
19
+ from pathlib import Path
20
+
21
+ logger = logging.getLogger("LinkedInAgent.DataFetcher")
22
+
23
+ RAPIDAPI_KEY = os.getenv("RAPIDAPI_KEY", "")
24
+ SERPER_API_KEY = os.getenv("SERPER_API_KEY", "")
25
+ REQUEST_TIMEOUT = int(os.getenv("REQUEST_TIMEOUT", "10"))
26
+ MOCK_MODE = os.getenv("LINKEDIN_MOCK_MODE", "false").lower() == "true"
27
+
28
+ # RapidAPI LinkedIn endpoints
29
+ RAPIDAPI_HOST_FEED = "linkedin-data-api.p.rapidapi.com"
30
+ RAPIDAPI_HOST_PROFILE = "fresh-linkedin-profile-data.p.rapidapi.com"
31
+
32
+ NICHE_TOPICS = os.getenv("LINKEDIN_NICHE_TOPICS", "AI, technology, startups")
33
+
34
+
35
+ # ── feed ──────────────────────────────────────────────────────────────────────
36
+
37
+ def fetch_linkedin_feed(max_posts: int = 20) -> list[dict]:
38
+ """
39
+ Fetches recent LinkedIn feed posts.
40
+ Returns: [{author, role, company, content, likes, comments, url, posted_at}]
41
+ """
42
+ if MOCK_MODE:
43
+ return _mock_feed()
44
+
45
+ if RAPIDAPI_KEY:
46
+ posts = _fetch_feed_rapidapi(max_posts)
47
+ if posts:
48
+ return posts
49
+
50
+ # Fallback: search for trending LinkedIn posts in user's niche via Serper
51
+ logger.info("RapidAPI unavailable β€” using Serper search fallback for feed")
52
+ return _fetch_feed_via_search()
53
+
54
+
55
+ def _fetch_feed_rapidapi(max_posts: int) -> list[dict]:
56
+ try:
57
+ resp = requests.get(
58
+ "https://linkedin-data-api.p.rapidapi.com/get-feed-posts",
59
+ headers={
60
+ "X-RapidAPI-Key": RAPIDAPI_KEY,
61
+ "X-RapidAPI-Host": RAPIDAPI_HOST_FEED,
62
+ },
63
+ params={"count": max_posts},
64
+ timeout=REQUEST_TIMEOUT,
65
+ )
66
+ resp.raise_for_status()
67
+ data = resp.json()
68
+ posts = []
69
+ for item in data.get("data", {}).get("items", []):
70
+ actor = item.get("actor", {})
71
+ posts.append({
72
+ "author": actor.get("name", ""),
73
+ "role": actor.get("description", ""),
74
+ "company": actor.get("subDescription", ""),
75
+ "content": item.get("commentary", {}).get("text", ""),
76
+ "likes": item.get("socialDetail", {}).get("totalLikes", 0),
77
+ "comments": item.get("socialDetail", {}).get("totalComments", 0),
78
+ "shares": item.get("socialDetail", {}).get("totalShares", 0),
79
+ "url": item.get("shareUrl", ""),
80
+ "posted_at": item.get("postedAt", ""),
81
+ "post_id": item.get("entityUrn", ""),
82
+ "source": "rapidapi",
83
+ })
84
+ return posts
85
+ except Exception as e:
86
+ logger.warning(f"RapidAPI feed fetch failed: {e}")
87
+ return []
88
+
89
+
90
+ def _fetch_feed_via_search() -> list[dict]:
91
+ """Uses Serper to find trending LinkedIn posts in the user's niche."""
92
+ if not SERPER_API_KEY:
93
+ logger.warning("No SERPER_API_KEY β€” returning mock feed")
94
+ return _mock_feed()
95
+
96
+ topics = [t.strip() for t in NICHE_TOPICS.split(",")][:3]
97
+ posts = []
98
+
99
+ for topic in topics:
100
+ try:
101
+ resp = requests.post(
102
+ "https://google.serper.dev/search",
103
+ headers={"X-API-KEY": SERPER_API_KEY, "Content-Type": "application/json"},
104
+ json={"q": f"site:linkedin.com/posts {topic} 2025", "num": 5},
105
+ timeout=REQUEST_TIMEOUT,
106
+ )
107
+ resp.raise_for_status()
108
+ for item in resp.json().get("organic", []):
109
+ posts.append({
110
+ "author": "",
111
+ "role": "",
112
+ "company": "",
113
+ "content": item.get("snippet", ""),
114
+ "title": item.get("title", ""),
115
+ "likes": 0,
116
+ "comments": 0,
117
+ "url": item.get("link", ""),
118
+ "posted_at": "",
119
+ "topic": topic,
120
+ "source": "serper",
121
+ })
122
+ except Exception as e:
123
+ logger.warning(f"Serper feed search failed for '{topic}': {e}")
124
+
125
+ return posts
126
+
127
+
128
+ # ── profile ───────────────────────────────────────────────────────────────────
129
+
130
+ def fetch_profile_info(name_or_url: str) -> Optional[dict]:
131
+ """
132
+ Fetches public LinkedIn profile info for a person.
133
+ Returns: {name, role, company, about, skills, recent_posts, url}
134
+ """
135
+ if MOCK_MODE:
136
+ return _mock_profile(name_or_url)
137
+
138
+ if RAPIDAPI_KEY:
139
+ return _fetch_profile_rapidapi(name_or_url)
140
+
141
+ return _fetch_profile_via_search(name_or_url)
142
+
143
+
144
+ def _fetch_profile_rapidapi(name_or_url: str) -> Optional[dict]:
145
+ try:
146
+ # If it's not a URL, search for the profile URL first
147
+ profile_url = name_or_url
148
+ if "linkedin.com" not in name_or_url:
149
+ profile_url = _search_profile_url(name_or_url)
150
+ if not profile_url:
151
+ return None
152
+
153
+ resp = requests.get(
154
+ "https://fresh-linkedin-profile-data.p.rapidapi.com/get-linkedin-profile",
155
+ headers={
156
+ "X-RapidAPI-Key": RAPIDAPI_KEY,
157
+ "X-RapidAPI-Host": RAPIDAPI_HOST_PROFILE,
158
+ },
159
+ params={"linkedin_url": profile_url, "include_skills": "true"},
160
+ timeout=REQUEST_TIMEOUT,
161
+ )
162
+ resp.raise_for_status()
163
+ d = resp.json().get("data", {})
164
+ return {
165
+ "name": d.get("full_name", ""),
166
+ "role": d.get("headline", ""),
167
+ "company": d.get("company", ""),
168
+ "location": d.get("location", ""),
169
+ "about": d.get("about", "")[:500],
170
+ "skills": [s.get("name") for s in d.get("skills", [])[:10]],
171
+ "connections": d.get("connections_count", 0),
172
+ "followers": d.get("followers_count", 0),
173
+ "url": profile_url,
174
+ "source": "rapidapi",
175
+ }
176
+ except Exception as e:
177
+ logger.warning(f"RapidAPI profile fetch failed: {e}")
178
+ return None
179
+
180
+
181
+ def _search_profile_url(name: str) -> Optional[str]:
182
+ """Searches Google for someone's LinkedIn profile URL."""
183
+ if not SERPER_API_KEY:
184
+ return None
185
+ try:
186
+ resp = requests.post(
187
+ "https://google.serper.dev/search",
188
+ headers={"X-API-KEY": SERPER_API_KEY, "Content-Type": "application/json"},
189
+ json={"q": f"site:linkedin.com/in {name}", "num": 1},
190
+ timeout=REQUEST_TIMEOUT,
191
+ )
192
+ results = resp.json().get("organic", [])
193
+ if results:
194
+ return results[0].get("link", "")
195
+ except Exception as e:
196
+ logger.warning(f"Profile URL search failed: {e}")
197
+ return None
198
+
199
+
200
+ def _fetch_profile_via_search(name: str) -> Optional[dict]:
201
+ """Fallback: build profile context from web search snippets."""
202
+ if not SERPER_API_KEY:
203
+ return None
204
+ try:
205
+ resp = requests.post(
206
+ "https://google.serper.dev/search",
207
+ headers={"X-API-KEY": SERPER_API_KEY, "Content-Type": "application/json"},
208
+ json={"q": f"{name} LinkedIn profile", "num": 3},
209
+ timeout=REQUEST_TIMEOUT,
210
+ )
211
+ results = resp.json().get("organic", [])
212
+ if not results:
213
+ return None
214
+ # Combine snippets as profile context
215
+ about = " ".join(r.get("snippet", "") for r in results[:2])
216
+ return {
217
+ "name": name,
218
+ "role": "",
219
+ "company": "",
220
+ "about": about[:500],
221
+ "source": "serper",
222
+ }
223
+ except Exception as e:
224
+ logger.warning(f"Profile search failed: {e}")
225
+ return None
226
+
227
+
228
+ # ── connection suggestions ────────────────────────────────────────────────────
229
+
230
+ def fetch_connection_suggestions() -> list[dict]:
231
+ """
232
+ Returns people worth connecting with.
233
+ Primary: RapidAPI | Fallback: Serper search for people in niche.
234
+ """
235
+ if MOCK_MODE:
236
+ return _mock_suggestions()
237
+
238
+ if RAPIDAPI_KEY:
239
+ try:
240
+ resp = requests.get(
241
+ "https://linkedin-data-api.p.rapidapi.com/get-profile-connections",
242
+ headers={
243
+ "X-RapidAPI-Key": RAPIDAPI_KEY,
244
+ "X-RapidAPI-Host": RAPIDAPI_HOST_FEED,
245
+ },
246
+ params={"count": 10},
247
+ timeout=REQUEST_TIMEOUT,
248
+ )
249
+ resp.raise_for_status()
250
+ data = resp.json()
251
+ items = data.get("data", {}).get("items", [])
252
+ return [
253
+ {
254
+ "name": item.get("name", ""),
255
+ "role": item.get("headline", ""),
256
+ "company": item.get("company", ""),
257
+ "url": item.get("profileUrl", ""),
258
+ "mutual": item.get("mutualConnections", 0),
259
+ "source": "rapidapi",
260
+ }
261
+ for item in items
262
+ ]
263
+ except Exception as e:
264
+ logger.warning(f"RapidAPI suggestions failed: {e}")
265
+
266
+ # Fallback: search for thought leaders in user's niche
267
+ return _fetch_suggestions_via_search()
268
+
269
+
270
+ def _fetch_suggestions_via_search() -> list[dict]:
271
+ if not SERPER_API_KEY:
272
+ return _mock_suggestions()
273
+ topics = [t.strip() for t in NICHE_TOPICS.split(",")][:2]
274
+ people = []
275
+ for topic in topics:
276
+ try:
277
+ resp = requests.post(
278
+ "https://google.serper.dev/search",
279
+ headers={"X-API-KEY": SERPER_API_KEY, "Content-Type": "application/json"},
280
+ json={"q": f"LinkedIn {topic} thought leader India 2025", "num": 4},
281
+ timeout=REQUEST_TIMEOUT,
282
+ )
283
+ for item in resp.json().get("organic", []):
284
+ if "linkedin.com/in/" in item.get("link", ""):
285
+ people.append({
286
+ "name": item.get("title", "").split(" - ")[0],
287
+ "role": item.get("title", "").split(" - ")[-1][:80],
288
+ "company": "",
289
+ "url": item.get("link", ""),
290
+ "snippet": item.get("snippet", ""),
291
+ "topic": topic,
292
+ "source": "serper",
293
+ })
294
+ except Exception as e:
295
+ logger.warning(f"Suggestion search failed: {e}")
296
+ return people[:8]
297
+
298
+
299
+ def fetch_my_profile() -> dict:
300
+ """Returns the user's own profile context from .env settings."""
301
+ return {
302
+ "name": os.getenv("LINKEDIN_USER_NAME", ""),
303
+ "role": os.getenv("LINKEDIN_USER_ROLE", ""),
304
+ "industry": os.getenv("LINKEDIN_USER_INDUSTRY", ""),
305
+ "topics": os.getenv("LINKEDIN_NICHE_TOPICS", ""),
306
+ "tone": os.getenv("LINKEDIN_WRITING_TONE", "professional yet conversational"),
307
+ "goal": os.getenv("LINKEDIN_GOAL", "build thought leadership and grow network"),
308
+ }
309
+
310
+
311
+ # ── mock data for dev/testing ──────────────────────────────────────────────────
312
+
313
+ def _mock_feed() -> list[dict]:
314
+ return [
315
+ {
316
+ "author": "Kunal Shah", "role": "Founder", "company": "CRED",
317
+ "content": "The best founders I know don't chase funding. They chase clarity. When you're clear on the problem, capital follows. Most pitches fail not because of bad ideas but because of unclear thinking.",
318
+ "likes": 3420, "comments": 187, "shares": 94,
319
+ "url": "https://linkedin.com/posts/mock1", "posted_at": "2h", "source": "mock",
320
+ },
321
+ {
322
+ "author": "Ankur Warikoo", "role": "Entrepreneur & Author", "company": "",
323
+ "content": "I failed 3 businesses before 30. Each one taught me something the previous hadn't. Failure is only permanent if you stop. The best thing you can do after failing is to write down what you learned within 24 hours.",
324
+ "likes": 5100, "comments": 342, "shares": 201,
325
+ "url": "https://linkedin.com/posts/mock2", "posted_at": "5h", "source": "mock",
326
+ },
327
+ {
328
+ "author": "Paras Chopra", "role": "Founder", "company": "Wingify",
329
+ "content": "Hot take: Most AI startups are building features, not businesses. A feature becomes a business only when it creates a switching cost. What's your moat beyond the AI model itself?",
330
+ "likes": 1890, "comments": 256, "shares": 78,
331
+ "url": "https://linkedin.com/posts/mock3", "posted_at": "8h", "source": "mock",
332
+ },
333
+ ]
334
+
335
+
336
+ def _mock_profile(name: str) -> dict:
337
+ return {
338
+ "name": name,
339
+ "role": "Senior Product Manager",
340
+ "company": "Tech Startup",
341
+ "about": f"{name} is a product leader with 8 years of experience in B2B SaaS. Previously at Razorpay and Freshworks.",
342
+ "skills": ["Product Strategy", "Growth", "AI/ML", "Leadership"],
343
+ "source": "mock",
344
+ }
345
+
346
+
347
+ def _mock_suggestions() -> list[dict]:
348
+ return [
349
+ {"name": "Nikhil Kamath", "role": "Co-founder", "company": "Zerodha", "mutual": 12, "source": "mock"},
350
+ {"name": "Sriram Krishnan", "role": "General Partner", "company": "a16z", "mutual": 8, "source": "mock"},
351
+ {"name": "Ritesh Agarwal", "role": "Founder & CEO", "company": "OYO", "mutual": 5, "source": "mock"},
352
+ ]
linkedin_agent/delivery.py ADDED
@@ -0,0 +1,348 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ delivery.py β€” LinkedIn Agent
3
+ ==============================
4
+ Output channels: Rich HTML email, WhatsApp, Notion, Markdown drafts file.
5
+ """
6
+
7
+ import os
8
+ import re
9
+ import smtplib
10
+ import logging
11
+ import requests
12
+ from email.mime.multipart import MIMEMultipart
13
+ from email.mime.text import MIMEText
14
+ from datetime import datetime
15
+ from pathlib import Path
16
+
17
+ logger = logging.getLogger("LinkedInAgent.Delivery")
18
+
19
+ SMTP_EMAIL = os.getenv("SMTP_EMAIL", "")
20
+ SMTP_PASSWORD = os.getenv("SMTP_PASSWORD", "")
21
+ SMTP_HOST = os.getenv("SMTP_HOST", "smtp.gmail.com")
22
+ SMTP_PORT = int(os.getenv("SMTP_PORT", "587"))
23
+
24
+ TWILIO_SID = os.getenv("TWILIO_ACCOUNT_SID", "")
25
+ TWILIO_AUTH = os.getenv("TWILIO_AUTH_TOKEN", "")
26
+ TWILIO_WA_FROM = os.getenv("TWILIO_WHATSAPP_FROM", "whatsapp:+14155238886")
27
+ WHATSAPP_TO = os.getenv("WHATSAPP_TO", "")
28
+
29
+ NOTION_TOKEN = os.getenv("NOTION_TOKEN", "")
30
+ NOTION_DB_ID = os.getenv("NOTION_LINKEDIN_DB_ID", "")
31
+
32
+ DRAFTS_DIR = Path(os.getenv("LINKEDIN_DRAFTS_DIR", "linkedin_drafts"))
33
+
34
+ ANGLE_EMOJI = {
35
+ "contrarian": "⚑",
36
+ "supportive": "πŸ‘",
37
+ "add_value": "πŸ’‘",
38
+ "ask_question": "❓",
39
+ "share_experience": "πŸ—£οΈ",
40
+ }
41
+
42
+
43
+ # ── markdown ──────────────────────────────────────────────────────────────────
44
+
45
+ def save_drafts_to_markdown(report: dict) -> str:
46
+ DRAFTS_DIR.mkdir(exist_ok=True)
47
+ ts = datetime.now().strftime("%Y%m%d_%H%M%S")
48
+ filename = DRAFTS_DIR / f"linkedin_drafts_{ts}.md"
49
+ date_str = datetime.now().strftime("%B %d, %Y")
50
+
51
+ lines = [f"# LinkedIn Drafts β€” {date_str}", ""]
52
+
53
+ # Post idea
54
+ post = report.get("post_idea", {})
55
+ if post.get("content"):
56
+ lines += [
57
+ "## ✍️ Post Draft",
58
+ f"**Type:** {post.get('post_type','')} | "
59
+ f"**Hook Score:** {post.get('hook_score','?')}/10 | "
60
+ f"**Best Time:** {post.get('best_time','')}",
61
+ "",
62
+ post["content"],
63
+ "",
64
+ f"**Hashtags:** {' '.join(post.get('hashtags', []))}",
65
+ "",
66
+ "---",
67
+ "",
68
+ ]
69
+
70
+ # Comments
71
+ comments = report.get("drafted_comments", [])
72
+ if comments:
73
+ lines.append("## πŸ’¬ Comments to Post")
74
+ for c in comments:
75
+ lines += [
76
+ f"### {ANGLE_EMOJI.get(c.get('suggested_angle',''), 'πŸ’¬')} {c.get('author', 'Unknown')}",
77
+ f"*{c.get('role', '')}*",
78
+ f"**Post summary:** {c.get('content_summary', '')}",
79
+ f"**Angle:** {c.get('suggested_angle', '')}",
80
+ "",
81
+ f"> {c.get('drafted_comment', '')}",
82
+ "",
83
+ f"πŸ”— {c.get('url', '')}",
84
+ "",
85
+ ]
86
+ lines.append("---\n")
87
+
88
+ # Connection messages
89
+ connections = report.get("drafted_connections", [])
90
+ if connections:
91
+ lines.append("## 🀝 Connection Requests to Send")
92
+ for p in connections:
93
+ lines += [
94
+ f"### {p.get('name', 'Unknown')}",
95
+ f"*{p.get('role', '')} @ {p.get('company', '')}*",
96
+ f"**Why connect:** {p.get('why_connect', '')}",
97
+ "",
98
+ f"**Message ({len(p.get('drafted_message',''))} chars):**",
99
+ f"> {p.get('drafted_message', '')}",
100
+ "",
101
+ f"πŸ”— {p.get('url', '')}",
102
+ "",
103
+ ]
104
+
105
+ try:
106
+ filename.write_text("\n".join(lines), encoding="utf-8")
107
+ return str(filename)
108
+ except Exception as e:
109
+ logger.error(f"Could not save markdown: {e}")
110
+ return ""
111
+
112
+
113
+ # ── email ─────────────────────────────────────────────────────────────────────
114
+
115
+ def send_digest_email(report: dict, recipient: str = "") -> bool:
116
+ if not SMTP_EMAIL or not SMTP_PASSWORD:
117
+ logger.warning("SMTP not configured β€” skipping email")
118
+ return False
119
+ recipient = recipient or SMTP_EMAIL
120
+ date_str = datetime.now().strftime("%b %d, %Y")
121
+ subject = f"πŸ”— LinkedIn Digest β€” {date_str}"
122
+ html = _build_html(report)
123
+ plain = _build_plain(report)
124
+
125
+ try:
126
+ msg = MIMEMultipart("alternative")
127
+ msg["Subject"] = subject
128
+ msg["From"] = SMTP_EMAIL
129
+ msg["To"] = recipient
130
+ msg.attach(MIMEText(plain, "plain"))
131
+ msg.attach(MIMEText(html, "html"))
132
+
133
+ with smtplib.SMTP(SMTP_HOST, SMTP_PORT) as server:
134
+ server.starttls()
135
+ server.login(SMTP_EMAIL, SMTP_PASSWORD)
136
+ server.sendmail(SMTP_EMAIL, recipient, msg.as_string())
137
+ logger.info(f"Digest email sent to {recipient}")
138
+ return True
139
+ except Exception as e:
140
+ logger.error(f"Email delivery failed: {e}", exc_info=True)
141
+ return False
142
+
143
+
144
+ def _build_plain(report: dict) -> str:
145
+ post = report.get("post_idea", {})
146
+ comments = report.get("drafted_comments", [])
147
+ connects = report.get("drafted_connections", [])
148
+ lines = [
149
+ f"LinkedIn Daily Digest β€” {datetime.now().strftime('%B %d, %Y')}",
150
+ "=" * 50,
151
+ f"Feed posts scanned: {report.get('feed_posts_scanned', 0)}",
152
+ f"Comment drafts: {len(comments)}",
153
+ f"Connection drafts: {len(connects)}",
154
+ "",
155
+ "POST IDEA:",
156
+ post.get("content", "None")[:500],
157
+ "",
158
+ ]
159
+ for c in comments:
160
+ lines += [f"COMMENT β†’ {c.get('author', '')}", c.get("drafted_comment", ""), ""]
161
+ for p in connects:
162
+ lines += [f"CONNECT β†’ {p.get('name', '')}", p.get("drafted_message", ""), ""]
163
+ return "\n".join(lines)
164
+
165
+
166
+ def _build_html(report: dict) -> str:
167
+ post = report.get("post_idea", {})
168
+ comments = report.get("drafted_comments", [])
169
+ connects = report.get("drafted_connections", [])
170
+ plan = report.get("engagement_plan", {})
171
+ analysis = report.get("feed_analysis", {})
172
+ ts = datetime.now().strftime("%B %d, %Y %H:%M")
173
+ scanned = report.get("feed_posts_scanned", 0)
174
+
175
+ # trending topics pills
176
+ topics_html = "".join(
177
+ f'<span style="background:#dbeafe;color:#1d4ed8;padding:3px 10px;border-radius:12px;font-size:12px;margin:2px">{t}</span>'
178
+ for t in analysis.get("trending_topics", [])[:5]
179
+ )
180
+
181
+ # post idea card
182
+ post_content = post.get("content", "")
183
+ hook_score = post.get("hook_score", 0)
184
+ hook_color = "#16a34a" if hook_score >= 8 else "#ca8a04" if hook_score >= 6 else "#dc2626"
185
+ hashtags_str = " ".join(post.get("hashtags", []))
186
+ post_card = f"""
187
+ <div style='background:#f0f9ff;border:1px solid #bae6fd;border-radius:8px;padding:16px;margin:12px 0'>
188
+ <div style='display:flex;justify-content:space-between;align-items:center;margin-bottom:8px'>
189
+ <strong style='color:#0369a1'>✍️ Today's Post Draft</strong>
190
+ <span style='background:{hook_color};color:white;padding:2px 8px;border-radius:12px;font-size:12px'>
191
+ Hook {hook_score}/10
192
+ </span>
193
+ </div>
194
+ <p style='white-space:pre-wrap;font-size:14px;line-height:1.6;margin:0'>{post_content}</p>
195
+ <p style='margin:8px 0 0;font-size:12px;color:#6b7280'>{hashtags_str} Β· Best time: {post.get("best_time","")}</p>
196
+ </div>""" if post_content else ""
197
+
198
+ # comment cards
199
+ comment_cards = ""
200
+ for c in comments:
201
+ angle = c.get("suggested_angle", "add_value")
202
+ emoji = ANGLE_EMOJI.get(angle, "πŸ’¬")
203
+ url = c.get("url", "#")
204
+ comment_cards += f"""
205
+ <div style='border:1px solid #e2e8f0;border-radius:8px;padding:14px;margin:8px 0'>
206
+ <div style='font-weight:600;margin-bottom:4px'>{emoji} {c.get("author","")} <span style='font-weight:normal;color:#6b7280;font-size:13px'>β€” {c.get("role","")}</span></div>
207
+ <p style='font-size:13px;color:#64748b;margin:4px 0;font-style:italic'>{c.get("content_summary","")}</p>
208
+ <div style='background:#f8fafc;border-left:3px solid #3b82f6;padding:8px 12px;margin:8px 0;border-radius:0 4px 4px 0'>
209
+ <p style='margin:0;font-size:14px'>{c.get("drafted_comment","")}</p>
210
+ </div>
211
+ <a href='{url}' style='font-size:12px;color:#3b82f6'>View post β†’</a>
212
+ </div>"""
213
+
214
+ # connection cards
215
+ connect_cards = ""
216
+ for p in connects:
217
+ msg_len = len(p.get("drafted_message", ""))
218
+ connect_cards += f"""
219
+ <div style='border:1px solid #e2e8f0;border-radius:8px;padding:14px;margin:8px 0'>
220
+ <div style='font-weight:600;margin-bottom:2px'>🀝 {p.get("name","")}</div>
221
+ <div style='font-size:13px;color:#6b7280;margin-bottom:8px'>{p.get("role","")} @ {p.get("company","")}</div>
222
+ <div style='background:#f0fdf4;border-left:3px solid #16a34a;padding:8px 12px;border-radius:0 4px 4px 0'>
223
+ <p style='margin:0;font-size:14px'>{p.get("drafted_message","")}</p>
224
+ </div>
225
+ <p style='font-size:12px;color:#9ca3af;margin:4px 0'>{msg_len}/300 chars</p>
226
+ </div>"""
227
+
228
+ daily_goal = plan.get("daily_goal", "")
229
+ est_time = plan.get("estimated_time_minutes", 15)
230
+
231
+ return f"""<!DOCTYPE html><html><body style='font-family:sans-serif;max-width:720px;margin:0 auto;color:#111;font-size:14px'>
232
+
233
+ <div style='background:#0a66c2;color:white;padding:20px 24px;border-radius:8px 8px 0 0'>
234
+ <h2 style='margin:0'>πŸ”— LinkedIn Agent Digest</h2>
235
+ <p style='margin:4px 0 0;opacity:0.8;font-size:13px'>{ts} Β· {scanned} posts scanned</p>
236
+ </div>
237
+
238
+ <div style='padding:16px 24px;background:#f8fafc;border:1px solid #e2e8f0;border-top:none'>
239
+ <div style='display:flex;gap:16px;flex-wrap:wrap;margin-bottom:12px'>
240
+ <div style='text-align:center'><strong style='font-size:20px;color:#0a66c2'>{len(comments)}</strong><br><span style='font-size:12px;color:#6b7280'>Comments</span></div>
241
+ <div style='text-align:center'><strong style='font-size:20px;color:#0a66c2'>{len(connects)}</strong><br><span style='font-size:12px;color:#6b7280'>Connections</span></div>
242
+ <div style='text-align:center'><strong style='font-size:20px;color:#0a66c2'>{est_time}m</strong><br><span style='font-size:12px;color:#6b7280'>Est. time</span></div>
243
+ </div>
244
+ {f'<p style="margin:0;font-size:13px;color:#1d4ed8;font-style:italic">🎯 {daily_goal}</p>' if daily_goal else ''}
245
+ </div>
246
+
247
+ <div style='padding:16px 24px;background:#fff;border:1px solid #e2e8f0;border-top:none'>
248
+ <strong style='font-size:13px;color:#64748b'>TRENDING IN YOUR FEED</strong><br>
249
+ <div style='margin-top:8px'>{topics_html}</div>
250
+ </div>
251
+
252
+ <div style='padding:16px 24px;background:#fff;border:1px solid #e2e8f0;border-top:none'>
253
+ {post_card}
254
+ </div>
255
+
256
+ {"<div style='padding:16px 24px;background:#fff;border:1px solid #e2e8f0;border-top:none'><h3 style='margin:0 0 8px;color:#1e293b'>πŸ’¬ Comment Drafts</h3>" + comment_cards + "</div>" if comment_cards else ""}
257
+
258
+ {"<div style='padding:16px 24px;background:#fff;border:1px solid #e2e8f0;border-top:none'><h3 style='margin:0 0 8px;color:#1e293b'>🀝 Connection Drafts</h3>" + connect_cards + "</div>" if connect_cards else ""}
259
+
260
+ <p style='font-size:11px;color:#9ca3af;padding:8px 24px'>Personal AI OS Β· LinkedIn Agent</p>
261
+ </body></html>"""
262
+
263
+
264
+ # ── whatsapp ──────────────────────────────────────────────────────────────────
265
+
266
+ def send_whatsapp_alert(report: dict) -> bool:
267
+ if not all([TWILIO_SID, TWILIO_AUTH, WHATSAPP_TO]):
268
+ logger.warning("Twilio not configured β€” skipping WhatsApp")
269
+ return False
270
+ try:
271
+ from twilio.rest import Client
272
+ comments = report.get("drafted_comments", [])
273
+ connects = report.get("drafted_connections", [])
274
+ post = report.get("post_idea", {})
275
+ plan = report.get("engagement_plan", {})
276
+
277
+ comment_names = ", ".join(c.get("author", "") for c in comments[:3])
278
+ connect_names = ", ".join(p.get("name", "") for p in connects[:3])
279
+
280
+ body = (
281
+ f"πŸ”— *LinkedIn Digest*\n"
282
+ f"{'─'*28}\n"
283
+ f"🎯 {plan.get('daily_goal','Engage and grow')}\n\n"
284
+ f"πŸ’¬ Comment on: {comment_names or 'none'}\n"
285
+ f"🀝 Connect with: {connect_names or 'none'}\n"
286
+ f"✍️ Post idea ({post.get('post_type','')}, score {post.get('hook_score','?')}/10)\n\n"
287
+ f"_{plan.get('estimated_time_minutes',15)} min total_"
288
+ )
289
+
290
+ client = Client(TWILIO_SID, TWILIO_AUTH)
291
+ msg = client.messages.create(
292
+ from_=TWILIO_WA_FROM, to=f"whatsapp:{WHATSAPP_TO}", body=body,
293
+ )
294
+ logger.info(f"WhatsApp sent: {msg.sid}")
295
+ return True
296
+ except Exception as e:
297
+ logger.error(f"WhatsApp failed: {e}", exc_info=True)
298
+ return False
299
+
300
+
301
+ # ── notion ────────────────────────────────────────────────────────────────────
302
+
303
+ def save_to_notion(report: dict) -> str:
304
+ if not NOTION_TOKEN or not NOTION_DB_ID:
305
+ logger.debug("Notion not configured β€” skipping")
306
+ return ""
307
+ try:
308
+ headers = {
309
+ "Authorization": f"Bearer {NOTION_TOKEN}",
310
+ "Notion-Version": "2022-06-28",
311
+ "Content-Type": "application/json",
312
+ }
313
+ date_str = datetime.now().strftime("%Y-%m-%d")
314
+ post = report.get("post_idea", {})
315
+ comments = report.get("drafted_comments", [])
316
+ connects = report.get("drafted_connections", [])
317
+
318
+ payload = {
319
+ "parent": {"database_id": NOTION_DB_ID},
320
+ "properties": {
321
+ "Name": {"title": [{"text": {"content": f"LinkedIn Digest β€” {date_str}"}}]},
322
+ "Date": {"date": {"start": date_str}},
323
+ "Post Type": {"select": {"name": post.get("post_type", "thought_leadership").replace("_", " ").title()}},
324
+ "Hook Score": {"number": post.get("hook_score", 0)},
325
+ "Comments Drafted": {"number": len(comments)},
326
+ "Connections Drafted": {"number": len(connects)},
327
+ "Trigger": {"rich_text": [{"text": {"content": report.get("trigger", "")}}]},
328
+ },
329
+ "children": [
330
+ {
331
+ "object": "block", "type": "heading_2",
332
+ "heading_2": {"rich_text": [{"type": "text", "text": {"content": "Post Draft"}}]},
333
+ },
334
+ {
335
+ "object": "block", "type": "paragraph",
336
+ "paragraph": {"rich_text": [{"type": "text", "text": {"content": post.get("content", "")[:2000]}}]},
337
+ },
338
+ ],
339
+ }
340
+ resp = requests.post(
341
+ "https://api.notion.com/v1/pages",
342
+ headers=headers, json=payload, timeout=10,
343
+ )
344
+ resp.raise_for_status()
345
+ return resp.json().get("url", "")
346
+ except Exception as e:
347
+ logger.error(f"Notion save failed: {e}", exc_info=True)
348
+ return ""
linkedin_agent/linkedin_data/drafts.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "id": "d04c1500-39b4-4730-90f2-79e9b05dca62",
4
+ "created_at": "2026-05-14T07:44:35.925500+00:00",
5
+ "date": "2026-05-14",
6
+ "comments": [
7
+ {
8
+ "author": "Ankur Warikoo",
9
+ "content_summary": "The author shares his experience of failing 3 businesses before 30 and emphasizes the importance of learning from failures",
10
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:1234567890/",
11
+ "why_comment": "The post is engaging and relevant to startup building, and sharing a personal story can help build thought leadership",
12
+ "suggested_angle": "share_experience",
13
+ "drafted_comment": "Having navigated the challenges of building and scaling multiple SaaS companies, it's clear that resilience and adaptability are key to transforming failures into valuable lessons. The ability to analyze and learn from setbacks has been instrumental in shaping our company's growth strategy, allowing us to pivot and refine our approach in response to changing market conditions. What role do you think mentorship and external guidance play in helping entrepreneurs bounce back from failure and apply those lessons to future ventures?"
14
+ },
15
+ {
16
+ "author": "Paras Chopra",
17
+ "content_summary": "The author argues that most AI startups are building features, not businesses, and emphasizes the need for a moat beyond the AI model",
18
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:2345678901/",
19
+ "why_comment": "The post is thought-provoking and relevant to AI agents, and sharing an example can add value to the conversation",
20
+ "suggested_angle": "add_value",
21
+ "drafted_comment": "Building a defensible moat beyond the AI model is crucial for long-term sustainability, and this can be achieved by focusing on the data flywheel effect, where the AI model improves with more data, creating a self-reinforcing cycle. Successful AI startups have also emphasized the importance of integrating their AI models with human workflows, creating a seamless user experience that drives adoption and retention. By doing so, they can create a competitive advantage that goes beyond just the technology itself, what strategies have been most effective in your experience for building this type of moat?"
22
+ },
23
+ {
24
+ "author": "Kunal Shah",
25
+ "content_summary": "The author emphasizes the importance of clarity in founding a successful startup, rather than just chasing funding",
26
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:3456789012/",
27
+ "why_comment": "The post is engaging and relevant to startup building, and discussing the importance of clarity can help build thought leadership",
28
+ "suggested_angle": "supportive",
29
+ "drafted_comment": "Clarity is indeed crucial in the early stages of a startup, as it allows founders to focus on solving a specific problem and build a strong foundation for growth. Many successful SaaS companies have demonstrated that prioritizing clarity over funding can lead to more sustainable and long-term success. For instance, companies like Atlassian and Zoom have achieved remarkable growth by staying true to their core mission and values, what role do you think company culture plays in maintaining this clarity as the organization scales?"
30
+ }
31
+ ],
32
+ "connections": [
33
+ {
34
+ "name": "Nikhil Kamath",
35
+ "role": "Co-founder",
36
+ "company": "Zerodha",
37
+ "url": "https://www.linkedin.com/in/nikhilkamath/",
38
+ "why_connect": "Nikhil Kamath is a relevant connection in the startup space with 12 mutual connections, increasing the likelihood of engagement",
39
+ "drafted_message": "Nikhil, impressed by Zerodha's innovative approach, I'm reaching out as a fellow startup founder with similar interests in building scalable products."
40
+ },
41
+ {
42
+ "name": "Sriram Krishnan",
43
+ "role": "General Partner",
44
+ "company": "a16z",
45
+ "url": "https://www.linkedin.com/in/sriramkrishnan/",
46
+ "why_connect": "Sriram Krishnan is a key player in the VC space with 8 mutual connections, and connecting with him can lead to valuable networking opportunities",
47
+ "drafted_message": "Sriram, impressed by your work at a16z, particularly in areas like startup building and product management, looking forward to learning from your experience and exploring potential synergies."
48
+ },
49
+ {
50
+ "name": "Ritesh Agarwal",
51
+ "role": "Founder & CEO",
52
+ "company": "OYO",
53
+ "url": "https://www.linkedin.com/in/riteshagarwal/",
54
+ "why_connect": "Ritesh Agarwal is a successful entrepreneur in the startup space with 5 mutual connections, and connecting with him can help build relationships with other founders",
55
+ "drafted_message": "Impressed by OYO's growth, Ritesh. As a fellow founder and CEO, I'd appreciate learning from your startup building experiences and exploring potential synergies."
56
+ }
57
+ ],
58
+ "post_idea": {
59
+ "content": "**85% of startups fail within the first three years, often due to poor market timing or lack of product-market fit.**\nThis staggering statistic highlights the importance of careful planning and adaptability in the startup ecosystem.\n\nSuccessful entrepreneurs, such as Steve Jobs and Elon Musk, have demonstrated the ability to pivot and adjust their strategies in response to changing market conditions.\nTheir willingness to take calculated risks and experiment with new approaches has been key to their success.\n\nIn today's fast-paced technology landscape, the use of AI agents is becoming increasingly prevalent, with companies like Salesforce and Microsoft leveraging AI to drive innovation and growth.\nThe ability to effectively integrate AI into business operations will be crucial for startups looking to stay ahead of the curve.\n\nAs the founder of a SaaS company, navigating these challenges is a daily reality.\nWhat strategies have been most effective for you in building a successful startup, and how do you see AI shaping the future of entrepreneurship?",
60
+ "post_type": "thought_leadership",
61
+ "hook_score": 8,
62
+ "hashtags": [
63
+ "#startups",
64
+ "#AI",
65
+ "#entrepreneurship"
66
+ ],
67
+ "best_time": "Morning (7-9am) | Lunch (12-1pm) | Evening (6-8pm)",
68
+ "word_count": 163,
69
+ "char_count": 1063
70
+ }
71
+ }
72
+ ]
linkedin_agent/linkedin_data/feed_snapshots.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "date": "2026-05-14",
4
+ "timestamp": "2026-05-14T07:44:35.924598+00:00",
5
+ "post_count": 3,
6
+ "engagement_plan": {
7
+ "comment_targets": [
8
+ {
9
+ "author": "Ankur Warikoo",
10
+ "content_summary": "The author shares his experience of failing 3 businesses before 30 and emphasizes the importance of learning from failures",
11
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:1234567890/",
12
+ "why_comment": "The post is engaging and relevant to startup building, and sharing a personal story can help build thought leadership",
13
+ "suggested_angle": "share_experience",
14
+ "drafted_comment": "Having navigated the challenges of building and scaling multiple SaaS companies, it's clear that resilience and adaptability are key to transforming failures into valuable lessons. The ability to analyze and learn from setbacks has been instrumental in shaping our company's growth strategy, allowing us to pivot and refine our approach in response to changing market conditions. What role do you think mentorship and external guidance play in helping entrepreneurs bounce back from failure and apply those lessons to future ventures?"
15
+ },
16
+ {
17
+ "author": "Paras Chopra",
18
+ "content_summary": "The author argues that most AI startups are building features, not businesses, and emphasizes the need for a moat beyond the AI model",
19
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:2345678901/",
20
+ "why_comment": "The post is thought-provoking and relevant to AI agents, and sharing an example can add value to the conversation",
21
+ "suggested_angle": "add_value",
22
+ "drafted_comment": "Building a defensible moat beyond the AI model is crucial for long-term sustainability, and this can be achieved by focusing on the data flywheel effect, where the AI model improves with more data, creating a self-reinforcing cycle. Successful AI startups have also emphasized the importance of integrating their AI models with human workflows, creating a seamless user experience that drives adoption and retention. By doing so, they can create a competitive advantage that goes beyond just the technology itself, what strategies have been most effective in your experience for building this type of moat?"
23
+ },
24
+ {
25
+ "author": "Kunal Shah",
26
+ "content_summary": "The author emphasizes the importance of clarity in founding a successful startup, rather than just chasing funding",
27
+ "url": "https://www.linkedin.com/feed/update/urn:li:activity:3456789012/",
28
+ "why_comment": "The post is engaging and relevant to startup building, and discussing the importance of clarity can help build thought leadership",
29
+ "suggested_angle": "supportive",
30
+ "drafted_comment": "Clarity is indeed crucial in the early stages of a startup, as it allows founders to focus on solving a specific problem and build a strong foundation for growth. Many successful SaaS companies have demonstrated that prioritizing clarity over funding can lead to more sustainable and long-term success. For instance, companies like Atlassian and Zoom have achieved remarkable growth by staying true to their core mission and values, what role do you think company culture plays in maintaining this clarity as the organization scales?"
31
+ }
32
+ ],
33
+ "connect_targets": [
34
+ {
35
+ "name": "Nikhil Kamath",
36
+ "role": "Co-founder",
37
+ "company": "Zerodha",
38
+ "url": "https://www.linkedin.com/in/nikhilkamath/",
39
+ "why_connect": "Nikhil Kamath is a relevant connection in the startup space with 12 mutual connections, increasing the likelihood of engagement",
40
+ "drafted_message": "Nikhil, impressed by Zerodha's innovative approach, I'm reaching out as a fellow startup founder with similar interests in building scalable products."
41
+ },
42
+ {
43
+ "name": "Sriram Krishnan",
44
+ "role": "General Partner",
45
+ "company": "a16z",
46
+ "url": "https://www.linkedin.com/in/sriramkrishnan/",
47
+ "why_connect": "Sriram Krishnan is a key player in the VC space with 8 mutual connections, and connecting with him can lead to valuable networking opportunities",
48
+ "drafted_message": "Sriram, impressed by your work at a16z, particularly in areas like startup building and product management, looking forward to learning from your experience and exploring potential synergies."
49
+ },
50
+ {
51
+ "name": "Ritesh Agarwal",
52
+ "role": "Founder & CEO",
53
+ "company": "OYO",
54
+ "url": "https://www.linkedin.com/in/riteshagarwal/",
55
+ "why_connect": "Ritesh Agarwal is a successful entrepreneur in the startup space with 5 mutual connections, and connecting with him can help build relationships with other founders",
56
+ "drafted_message": "Impressed by OYO's growth, Ritesh. As a fellow founder and CEO, I'd appreciate learning from your startup building experiences and exploring potential synergies."
57
+ }
58
+ ],
59
+ "daily_goal": "Engage with relevant posts and connect with key players in the startup and AI spaces to build thought leadership and grow the network",
60
+ "estimated_time_minutes": 15
61
+ },
62
+ "trending_topics": []
63
+ }
64
+ ]
linkedin_agent/linkedin_drafts/linkedin_drafts_20260514_131435.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # LinkedIn Drafts β€” May 14, 2026
2
+
3
+ ## ✍️ Post Draft
4
+ **Type:** thought_leadership | **Hook Score:** 8/10 | **Best Time:** Morning (7-9am) | Lunch (12-1pm) | Evening (6-8pm)
5
+
6
+ **85% of startups fail within the first three years, often due to poor market timing or lack of product-market fit.**
7
+ This staggering statistic highlights the importance of careful planning and adaptability in the startup ecosystem.
8
+
9
+ Successful entrepreneurs, such as Steve Jobs and Elon Musk, have demonstrated the ability to pivot and adjust their strategies in response to changing market conditions.
10
+ Their willingness to take calculated risks and experiment with new approaches has been key to their success.
11
+
12
+ In today's fast-paced technology landscape, the use of AI agents is becoming increasingly prevalent, with companies like Salesforce and Microsoft leveraging AI to drive innovation and growth.
13
+ The ability to effectively integrate AI into business operations will be crucial for startups looking to stay ahead of the curve.
14
+
15
+ As the founder of a SaaS company, navigating these challenges is a daily reality.
16
+ What strategies have been most effective for you in building a successful startup, and how do you see AI shaping the future of entrepreneurship?
17
+
18
+ **Hashtags:** #startups #AI #entrepreneurship
19
+
20
+ ---
21
+
22
+ ## πŸ’¬ Comments to Post
23
+ ### πŸ—£οΈ Ankur Warikoo
24
+ **
25
+ **Post summary:** The author shares his experience of failing 3 businesses before 30 and emphasizes the importance of learning from failures
26
+ **Angle:** share_experience
27
+
28
+ > Having navigated the challenges of building and scaling multiple SaaS companies, it's clear that resilience and adaptability are key to transforming failures into valuable lessons. The ability to analyze and learn from setbacks has been instrumental in shaping our company's growth strategy, allowing us to pivot and refine our approach in response to changing market conditions. What role do you think mentorship and external guidance play in helping entrepreneurs bounce back from failure and apply those lessons to future ventures?
29
+
30
+ πŸ”— https://www.linkedin.com/feed/update/urn:li:activity:1234567890/
31
+
32
+ ### πŸ’‘ Paras Chopra
33
+ **
34
+ **Post summary:** The author argues that most AI startups are building features, not businesses, and emphasizes the need for a moat beyond the AI model
35
+ **Angle:** add_value
36
+
37
+ > Building a defensible moat beyond the AI model is crucial for long-term sustainability, and this can be achieved by focusing on the data flywheel effect, where the AI model improves with more data, creating a self-reinforcing cycle. Successful AI startups have also emphasized the importance of integrating their AI models with human workflows, creating a seamless user experience that drives adoption and retention. By doing so, they can create a competitive advantage that goes beyond just the technology itself, what strategies have been most effective in your experience for building this type of moat?
38
+
39
+ πŸ”— https://www.linkedin.com/feed/update/urn:li:activity:2345678901/
40
+
41
+ ### πŸ‘ Kunal Shah
42
+ **
43
+ **Post summary:** The author emphasizes the importance of clarity in founding a successful startup, rather than just chasing funding
44
+ **Angle:** supportive
45
+
46
+ > Clarity is indeed crucial in the early stages of a startup, as it allows founders to focus on solving a specific problem and build a strong foundation for growth. Many successful SaaS companies have demonstrated that prioritizing clarity over funding can lead to more sustainable and long-term success. For instance, companies like Atlassian and Zoom have achieved remarkable growth by staying true to their core mission and values, what role do you think company culture plays in maintaining this clarity as the organization scales?
47
+
48
+ πŸ”— https://www.linkedin.com/feed/update/urn:li:activity:3456789012/
49
+
50
+ ---
51
+
52
+ ## 🀝 Connection Requests to Send
53
+ ### Nikhil Kamath
54
+ *Co-founder @ Zerodha*
55
+ **Why connect:** Nikhil Kamath is a relevant connection in the startup space with 12 mutual connections, increasing the likelihood of engagement
56
+
57
+ **Message (150 chars):**
58
+ > Nikhil, impressed by Zerodha's innovative approach, I'm reaching out as a fellow startup founder with similar interests in building scalable products.
59
+
60
+ πŸ”— https://www.linkedin.com/in/nikhilkamath/
61
+
62
+ ### Sriram Krishnan
63
+ *General Partner @ a16z*
64
+ **Why connect:** Sriram Krishnan is a key player in the VC space with 8 mutual connections, and connecting with him can lead to valuable networking opportunities
65
+
66
+ **Message (191 chars):**
67
+ > Sriram, impressed by your work at a16z, particularly in areas like startup building and product management, looking forward to learning from your experience and exploring potential synergies.
68
+
69
+ πŸ”— https://www.linkedin.com/in/sriramkrishnan/
70
+
71
+ ### Ritesh Agarwal
72
+ *Founder & CEO @ OYO*
73
+ **Why connect:** Ritesh Agarwal is a successful entrepreneur in the startup space with 5 mutual connections, and connecting with him can help build relationships with other founders
74
+
75
+ **Message (161 chars):**
76
+ > Impressed by OYO's growth, Ritesh. As a fellow founder and CEO, I'd appreciate learning from your startup building experiences and exploring potential synergies.
77
+
78
+ πŸ”— https://www.linkedin.com/in/riteshagarwal/
linkedin_agent/linkedin_store.py ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ linkedin_store.py β€” LinkedIn Agent
3
+ =====================================
4
+ Local JSON store for feed snapshots, drafted content,
5
+ posting history, and connection tracking.
6
+ """
7
+
8
+ import os
9
+ import json
10
+ import uuid
11
+ import logging
12
+ from datetime import datetime, timezone, timedelta
13
+ from pathlib import Path
14
+
15
+ logger = logging.getLogger("LinkedInAgent.Store")
16
+
17
+ STORE_DIR = Path(os.getenv("LINKEDIN_STORE_DIR", "linkedin_data"))
18
+ DRAFTS_FILE = STORE_DIR / "drafts.json"
19
+ HISTORY_FILE = STORE_DIR / "post_history.json"
20
+ CONNECTIONS_FILE = STORE_DIR / "connections.json"
21
+ FEED_FILE = STORE_DIR / "feed_snapshots.json"
22
+ MAX_RECORDS = 500
23
+
24
+
25
+ class LinkedInStore:
26
+ def __init__(self):
27
+ STORE_DIR.mkdir(exist_ok=True)
28
+ self._drafts = self._load(DRAFTS_FILE)
29
+ self._history = self._load(HISTORY_FILE)
30
+ self._connections = self._load(CONNECTIONS_FILE)
31
+ self._feed = self._load(FEED_FILE)
32
+
33
+ def _load(self, path: Path) -> list:
34
+ if path.exists():
35
+ try:
36
+ with open(path) as f:
37
+ return json.load(f)
38
+ except Exception as e:
39
+ logger.warning(f"Could not load {path}: {e}")
40
+ return []
41
+
42
+ def _save(self, path: Path, data: list):
43
+ try:
44
+ with open(path, "w") as f:
45
+ json.dump(data[-MAX_RECORDS:], f, indent=2, default=str)
46
+ except Exception as e:
47
+ logger.error(f"Could not save {path}: {e}")
48
+
49
+ # ── drafts ────────────────────────────────────────────────────────────────
50
+
51
+ def save_drafts(
52
+ self,
53
+ comments: list[dict],
54
+ connections: list[dict],
55
+ post_idea: dict,
56
+ ):
57
+ entry = {
58
+ "id": str(uuid.uuid4()),
59
+ "created_at": datetime.now(timezone.utc).isoformat(),
60
+ "date": datetime.now().strftime("%Y-%m-%d"),
61
+ "comments": comments,
62
+ "connections": connections,
63
+ "post_idea": post_idea,
64
+ }
65
+ self._drafts.append(entry)
66
+ self._save(DRAFTS_FILE, self._drafts)
67
+
68
+ def get_todays_drafts(self) -> dict:
69
+ today = datetime.now().strftime("%Y-%m-%d")
70
+ for entry in reversed(self._drafts):
71
+ if entry.get("date") == today:
72
+ return entry
73
+ return {}
74
+
75
+ def get_all_draft_posts(self) -> list[dict]:
76
+ posts = []
77
+ for entry in self._drafts:
78
+ if entry.get("post_idea", {}).get("content"):
79
+ posts.append({
80
+ "date": entry["date"],
81
+ "content": entry["post_idea"]["content"],
82
+ "post_type": entry["post_idea"].get("post_type", ""),
83
+ "hook_score": entry["post_idea"].get("hook_score", 0),
84
+ "hashtags": entry["post_idea"].get("hashtags", []),
85
+ "status": entry["post_idea"].get("status", "draft"),
86
+ })
87
+ return posts
88
+
89
+ # ── posting history ───────────────────────────────────────────────────────
90
+
91
+ def mark_posted(self, post_content: str, post_type: str = "", url: str = ""):
92
+ entry = {
93
+ "id": str(uuid.uuid4()),
94
+ "posted_at": datetime.now(timezone.utc).isoformat(),
95
+ "date": datetime.now().strftime("%Y-%m-%d"),
96
+ "content": post_content[:500],
97
+ "post_type": post_type,
98
+ "url": url,
99
+ }
100
+ self._history.append(entry)
101
+ self._save(HISTORY_FILE, self._history)
102
+
103
+ def get_post_history(self, days: int = 30) -> list[dict]:
104
+ cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
105
+ return [h for h in self._history if h.get("date", "") >= cutoff]
106
+
107
+ def get_topics_posted_recently(self, days: int = 14) -> list[str]:
108
+ """Returns recently used topics to avoid repetition."""
109
+ recent = self.get_post_history(days=days)
110
+ return [r.get("post_type", "") for r in recent]
111
+
112
+ # ── connections ───────────────────────────────────────────────────────────
113
+
114
+ def save_connection_sent(self, person: dict, message: str):
115
+ entry = {
116
+ "id": str(uuid.uuid4()),
117
+ "sent_at": datetime.now(timezone.utc).isoformat(),
118
+ "date": datetime.now().strftime("%Y-%m-%d"),
119
+ "name": person.get("name", ""),
120
+ "role": person.get("role", ""),
121
+ "company": person.get("company", ""),
122
+ "url": person.get("url", ""),
123
+ "message": message,
124
+ "status": "sent",
125
+ }
126
+ self._connections.append(entry)
127
+ self._save(CONNECTIONS_FILE, self._connections)
128
+
129
+ def get_recent_connections(self, days: int = 7) -> list[dict]:
130
+ cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
131
+ return [c for c in self._connections if c.get("date", "") >= cutoff]
132
+
133
+ # ── feed snapshots ────────────────────────────────────────────────────────
134
+
135
+ def save_feed_snapshot(self, posts: list[dict], engagement_plan: dict):
136
+ entry = {
137
+ "date": datetime.now().strftime("%Y-%m-%d"),
138
+ "timestamp": datetime.now(timezone.utc).isoformat(),
139
+ "post_count": len(posts),
140
+ "engagement_plan": engagement_plan,
141
+ "trending_topics": engagement_plan.get("trending_topics", []),
142
+ }
143
+ self._feed.append(entry)
144
+ self._save(FEED_FILE, self._feed)
145
+
146
+ def get_recent_feed_snapshots(self, days: int = 7) -> list[dict]:
147
+ cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
148
+ return [f for f in self._feed if f.get("date", "") >= cutoff]
149
+
150
+ # ── stats ─────────────────────────────────────────────────────────────────
151
+
152
+ def get_weekly_stats(self) -> dict:
153
+ return {
154
+ "posts_drafted": len(self.get_all_draft_posts()),
155
+ "posts_published": len(self.get_post_history(days=7)),
156
+ "connections_sent": len(self.get_recent_connections(days=7)),
157
+ "days_active": len({f["date"] for f in self.get_recent_feed_snapshots(days=7)}),
158
+ }
linkedin_agent/llm.py ADDED
@@ -0,0 +1,385 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ llm.py β€” LinkedIn Agent
3
+ =========================
4
+ All Groq (llama-3.3-70b-versatile) calls:
5
+ - Feed analysis + topic extraction
6
+ - Engagement plan generation
7
+ - Comment drafting
8
+ - Connection message drafting
9
+ - Post drafting (6 formats)
10
+ - Post idea scoring
11
+ """
12
+
13
+ import os
14
+ import json
15
+ import re
16
+ import logging
17
+ from datetime import datetime
18
+ from dotenv import load_dotenv
19
+ load_dotenv()
20
+
21
+ from groq import Groq
22
+
23
+ logger = logging.getLogger("LinkedInAgent.LLM")
24
+
25
+ client = Groq(api_key=os.getenv("GROQ_API_KEY"))
26
+ MODEL = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")
27
+ TODAY = datetime.now().strftime("%A, %B %d, %Y")
28
+
29
+
30
+ # ── helpers ───────────────────────────────────────────────────────────────────
31
+
32
+ def _chat(system: str, user: str, temperature: float = 0.6, max_tokens: int = 1500) -> str:
33
+ resp = client.chat.completions.create(
34
+ model=MODEL,
35
+ temperature=temperature,
36
+ max_tokens=max_tokens,
37
+ messages=[
38
+ {"role": "system", "content": system},
39
+ {"role": "user", "content": user},
40
+ ],
41
+ )
42
+ return resp.choices[0].message.content.strip()
43
+
44
+
45
+ def _safe_json(text: str, fallback=None):
46
+ text = re.sub(r"```(?:json)?", "", text).strip().rstrip("`")
47
+ for s, e in [("[", "]"), ("{", "}")]:
48
+ si, ei = text.find(s), text.rfind(e)
49
+ if si != -1 and ei != -1:
50
+ try:
51
+ return json.loads(text[si: ei + 1])
52
+ except json.JSONDecodeError:
53
+ pass
54
+ return fallback if fallback is not None else {}
55
+
56
+
57
+ # ── feed analysis ─────────────────────────────────────────────────────────────
58
+
59
+ FEED_ANALYSIS_SYSTEM = f"""You are a LinkedIn growth strategist. Today is {TODAY}.
60
+
61
+ Analyse a LinkedIn feed and return a JSON object with:
62
+ {{
63
+ "trending_topics": ["topic1", "topic2", "topic3"],
64
+ "top_posts": [
65
+ {{
66
+ "author": "...",
67
+ "content_summary": "1 sentence summary",
68
+ "why_engaging": "why this post is getting traction",
69
+ "engagement_opportunity": "high|medium|low",
70
+ "suggested_angle": "what angle to take when commenting"
71
+ }}
72
+ ],
73
+ "feed_sentiment": "positive|neutral|mixed|negative",
74
+ "content_gaps": ["topic not covered that the user could post about"],
75
+ "best_posting_time": "observation about when posts in the feed were made"
76
+ }}
77
+
78
+ Focus on posts with high engagement or interesting ideas worth engaging with.
79
+ Return top 5 posts maximum in top_posts."""
80
+
81
+
82
+ def analyze_feed(posts: list[dict], user_context: dict) -> dict:
83
+ if not posts:
84
+ return {"trending_topics": [], "top_posts": [], "content_gaps": []}
85
+
86
+ posts_text = "\n\n".join(
87
+ f"[POST {i+1}]\nAuthor: {p.get('author','')} ({p.get('role','')} @ {p.get('company','')})\n"
88
+ f"Likes: {p.get('likes',0)} | Comments: {p.get('comments',0)}\n"
89
+ f"Content: {p.get('content','')[:400]}"
90
+ for i, p in enumerate(posts[:15])
91
+ )
92
+
93
+ user_str = (
94
+ f"User: {user_context.get('name')} | {user_context.get('role')} | "
95
+ f"Topics: {user_context.get('topics')}"
96
+ )
97
+
98
+ try:
99
+ raw = _chat(FEED_ANALYSIS_SYSTEM, f"{user_str}\n\nFEED:\n{posts_text}", temperature=0.3)
100
+ result = _safe_json(raw, {})
101
+ return result if isinstance(result, dict) else {}
102
+ except Exception as e:
103
+ logger.error(f"Feed analysis failed: {e}")
104
+ return {}
105
+
106
+
107
+ # ── engagement plan ───────────────────────────────────────────────────────────
108
+
109
+ ENGAGEMENT_PLAN_SYSTEM = f"""You are a LinkedIn growth strategist. Today is {TODAY}.
110
+
111
+ Based on feed analysis and connection suggestions, create a daily engagement plan.
112
+
113
+ Return JSON:
114
+ {{
115
+ "comment_targets": [
116
+ {{
117
+ "author": "...",
118
+ "content_summary": "...",
119
+ "url": "...",
120
+ "why_comment": "strategic reason",
121
+ "suggested_angle": "contrarian|supportive|add_value|ask_question|share_experience"
122
+ }}
123
+ ],
124
+ "connect_targets": [
125
+ {{
126
+ "name": "...",
127
+ "role": "...",
128
+ "company": "...",
129
+ "url": "...",
130
+ "why_connect": "strategic reason"
131
+ }}
132
+ ],
133
+ "daily_goal": "one sentence goal for today's LinkedIn activity",
134
+ "estimated_time_minutes": 15
135
+ }}
136
+
137
+ Limit: 3 comment targets, 3 connect targets.
138
+ Prioritise people who are active, relevant to the user's niche, and likely to engage back."""
139
+
140
+
141
+ def generate_engagement_plan(
142
+ feed_analysis: dict,
143
+ suggestions: list[dict],
144
+ user_context: dict,
145
+ ) -> dict:
146
+ user_str = (
147
+ f"User: {user_context.get('name')} | {user_context.get('role')} | "
148
+ f"Goal: {user_context.get('goal', 'grow network and build thought leadership')} | "
149
+ f"Topics: {user_context.get('topics')}"
150
+ )
151
+ feed_str = json.dumps(feed_analysis.get("top_posts", [])[:5], indent=2)
152
+ suggestions_str = json.dumps(suggestions[:8], indent=2)
153
+
154
+ try:
155
+ raw = _chat(
156
+ ENGAGEMENT_PLAN_SYSTEM,
157
+ f"{user_str}\n\nFEED ANALYSIS:\n{feed_str}\n\nCONNECTION SUGGESTIONS:\n{suggestions_str}",
158
+ temperature=0.4,
159
+ )
160
+ result = _safe_json(raw, {})
161
+ return result if isinstance(result, dict) else {}
162
+ except Exception as e:
163
+ logger.error(f"Engagement plan failed: {e}")
164
+ return {}
165
+
166
+
167
+ # ── comment drafting ──────────────────────────────────────────────────────────
168
+
169
+ COMMENT_SYSTEM = f"""You are a LinkedIn expert ghostwriter. Today is {TODAY}.
170
+
171
+ Draft a genuine, high-value LinkedIn comment. Rules:
172
+ - 2-4 sentences maximum
173
+ - Add real value: insight, experience, a question, or a respectful different angle
174
+ - NEVER sycophantic openers ("Great post!", "Love this!", "So true!")
175
+ - Sound human β€” conversational but professional
176
+ - Match the angle: contrarian=respectful pushback, supportive=add data/example, ask_question=genuine curiosity
177
+ - End with a question only if it flows naturally
178
+ - No hashtags in comments
179
+ - Do not start with "I"
180
+
181
+ Return ONLY the comment text, nothing else."""
182
+
183
+
184
+ def draft_comment(post: dict, user_context: dict) -> str:
185
+ post_text = (
186
+ f"Post by {post.get('author','')} ({post.get('role','')}):\n"
187
+ f"{post.get('content_summary', post.get('content', ''))}\n"
188
+ f"Angle to take: {post.get('suggested_angle', 'add_value')}"
189
+ )
190
+ user_str = (
191
+ f"Commenter: {user_context.get('name')} | {user_context.get('role')} | "
192
+ f"Industry: {user_context.get('industry')}"
193
+ )
194
+ try:
195
+ return _chat(COMMENT_SYSTEM, f"{user_str}\n\nPOST:\n{post_text}", temperature=0.7)
196
+ except Exception as e:
197
+ logger.error(f"Comment draft failed: {e}")
198
+ return "Interesting perspective. What's been the biggest challenge in applying this at scale?"
199
+
200
+
201
+ # ── connection message drafting ───────────────────────────────────────────────
202
+
203
+ CONNECTION_SYSTEM = f"""You are a LinkedIn networking expert. Today is {TODAY}.
204
+
205
+ Draft a LinkedIn connection request note. Hard rules:
206
+ - 300 characters MAXIMUM (LinkedIn limit)
207
+ - Personalised β€” mention something specific about them
208
+ - Clear why you want to connect (shared interest, mutual goal, their work)
209
+ - No generic lines ("I'd like to add you to my network")
210
+ - Warm but not creepy β€” professional peer tone
211
+ - No emojis
212
+ - First person, direct
213
+
214
+ Return ONLY the message text."""
215
+
216
+
217
+ def draft_connection_message(person: dict, user_context: dict) -> str:
218
+ person_str = (
219
+ f"Connect with: {person.get('name')} | {person.get('role','')} @ {person.get('company','')}\n"
220
+ f"About them: {person.get('about', person.get('snippet', ''))[:300]}\n"
221
+ f"Why connecting: {person.get('why_connect', 'shared professional interest')}"
222
+ )
223
+ user_str = (
224
+ f"Sender: {user_context.get('name')} | {user_context.get('role')} | "
225
+ f"Topics: {user_context.get('topics')}"
226
+ )
227
+ try:
228
+ msg = _chat(CONNECTION_SYSTEM, f"{user_str}\n\n{person_str}", temperature=0.65)
229
+ # Enforce 300 char limit
230
+ if len(msg) > 300:
231
+ msg = msg[:297] + "..."
232
+ return msg
233
+ except Exception as e:
234
+ logger.error(f"Connection message draft failed: {e}")
235
+ return f"Hi {person.get('name','')}, your work in {person.get('role','')} aligns with what I'm exploring. Would love to connect and exchange perspectives."[:300]
236
+
237
+
238
+ # ── post drafting ─────────────────────────────────────────────────────────────
239
+
240
+ POST_TEMPLATES = {
241
+ "thought_leadership": """Write a thought leadership LinkedIn post. Structure:
242
+ - Hook: Bold first line that stops the scroll (no question hooks, make a statement or share a number)
243
+ - Body: 3-5 short paragraphs or lines. Share a specific insight, observation, or lesson.
244
+ - CTA: End with a question or invitation to share perspective
245
+ - Use line breaks generously β€” short paragraphs (1-2 lines each)
246
+ - NO corporate jargon, NO hashtags in the body
247
+ - Aim for 150-250 words""",
248
+
249
+ "how_to": """Write a practical how-to LinkedIn post. Structure:
250
+ - Hook: "How to [do X] without [common mistake]" or similar
251
+ - 5-7 numbered actionable steps
252
+ - Each step: 1-2 lines, specific and concrete
253
+ - Closing line: insight or invitation to comment
254
+ - 150-200 words""",
255
+
256
+ "story": """Write a personal story LinkedIn post. Structure:
257
+ - Hook: Start in the middle of the action ("I was about to quit...")
258
+ - Setup: Brief context (2-3 lines)
259
+ - Conflict: What went wrong or what you struggled with
260
+ - Resolution: What you learned or did
261
+ - Lesson: 1-2 sentence takeaway
262
+ - 200-280 words. First person. Vulnerable but professional.""",
263
+
264
+ "list": """Write a listicle LinkedIn post. Structure:
265
+ - Hook: "X things I wish I knew about [topic]" or similar
266
+ - 5-8 items, each 1-2 lines
267
+ - Items should be specific, non-obvious insights β€” not generic advice
268
+ - End with a question
269
+ - 150-220 words""",
270
+
271
+ "hot_take": """Write a contrarian/hot take LinkedIn post. Structure:
272
+ - Bold opening statement that challenges conventional wisdom
273
+ - 2-3 short paragraphs explaining your reasoning with evidence
274
+ - Acknowledge the counterargument briefly
275
+ - Restate your position with nuance
276
+ - End with a question inviting debate
277
+ - 150-200 words. Confident but not arrogant.""",
278
+
279
+ "celebration": """Write a celebration/milestone LinkedIn post. Structure:
280
+ - Share the achievement directly (no false humility)
281
+ - Brief story of the journey
282
+ - Credit people who helped
283
+ - Lesson or insight from the experience
284
+ - Forward-looking closing
285
+ - 150-200 words. Genuine, warm, not braggy.""",
286
+ }
287
+
288
+ POST_SYSTEM_BASE = f"""You are an expert LinkedIn ghostwriter. Today is {TODAY}.
289
+
290
+ You write posts that get 1000+ likes β€” not because they're viral bait,
291
+ but because they're genuinely insightful and human.
292
+
293
+ Writing rules:
294
+ - Never start with "I" as the first word
295
+ - Short paragraphs (1-3 lines each) with line breaks between them
296
+ - Specific over generic (use numbers, names, timeframes)
297
+ - Conversational but substantive
298
+ - No emojis in the main content
299
+ - Hashtags: 3-5 relevant ones at the very end only
300
+
301
+ After the post, on a new line write:
302
+ HOOK_SCORE: X/10
303
+ HASHTAGS: #tag1 #tag2 #tag3
304
+ BEST_TIME: Morning (7-9am) | Lunch (12-1pm) | Evening (6-8pm)"""
305
+
306
+
307
+ def draft_post(
308
+ user_context: dict,
309
+ trending_topics: list[str],
310
+ post_type: str = "thought_leadership",
311
+ ) -> dict:
312
+ template = POST_TEMPLATES.get(post_type, POST_TEMPLATES["thought_leadership"])
313
+ topics_str = ", ".join(trending_topics[:3]) if trending_topics else user_context.get("topics", "")
314
+
315
+ user_str = (
316
+ f"Author: {user_context.get('name')} | {user_context.get('role')} | "
317
+ f"Industry: {user_context.get('industry')}\n"
318
+ f"Writing tone: {user_context.get('tone', 'professional yet conversational')}\n"
319
+ f"Goal: {user_context.get('goal', 'build thought leadership')}\n"
320
+ f"Trending topics to consider: {topics_str}"
321
+ )
322
+
323
+ system = POST_SYSTEM_BASE + "\n\n" + template
324
+
325
+ try:
326
+ raw = _chat(system, user_str, temperature=0.75, max_tokens=800)
327
+
328
+ # Parse structured output
329
+ content = raw
330
+ hook_score = 7
331
+ hashtags = []
332
+ best_time = "Morning"
333
+
334
+ if "HOOK_SCORE:" in raw:
335
+ parts = raw.split("HOOK_SCORE:")
336
+ content = parts[0].strip()
337
+ metadata = parts[1] if len(parts) > 1 else ""
338
+
339
+ score_match = re.search(r"(\d+)/10", metadata)
340
+ if score_match:
341
+ hook_score = int(score_match.group(1))
342
+
343
+ hash_match = re.search(r"HASHTAGS:(.*?)(?:BEST_TIME:|$)", metadata, re.DOTALL)
344
+ if hash_match:
345
+ hashtags = re.findall(r"#\w+", hash_match.group(1))
346
+
347
+ time_match = re.search(r"BEST_TIME:(.*?)$", metadata, re.DOTALL)
348
+ if time_match:
349
+ best_time = time_match.group(1).strip()
350
+
351
+ return {
352
+ "content": content,
353
+ "post_type": post_type,
354
+ "hook_score": hook_score,
355
+ "hashtags": hashtags[:5],
356
+ "best_time": best_time,
357
+ "word_count": len(content.split()),
358
+ "char_count": len(content),
359
+ }
360
+ except Exception as e:
361
+ logger.error(f"Post draft failed: {e}")
362
+ return {"content": "", "post_type": post_type, "hook_score": 0, "hashtags": []}
363
+
364
+
365
+ # ── post scoring ──────────────────────────────────────────────────────────────
366
+
367
+ SCORE_SYSTEM = """You are a LinkedIn content expert. Score this post idea on:
368
+ - Hook strength (1-10): Does the first line stop the scroll?
369
+ - Value delivery (1-10): Is there real insight or is it generic?
370
+ - Authenticity (1-10): Does it sound human, not corporate?
371
+ - Engagement potential (1-10): Will people comment?
372
+
373
+ Return JSON only:
374
+ {"hook": 7, "value": 8, "authenticity": 9, "engagement": 7, "overall": 7.8,
375
+ "top_strength": "...", "top_weakness": "...", "quick_fix": "..."}"""
376
+
377
+
378
+ def score_post_idea(post_content: str) -> dict:
379
+ try:
380
+ raw = _chat(SCORE_SYSTEM, f"POST:\n{post_content}", temperature=0.2)
381
+ result = _safe_json(raw, {})
382
+ return result if isinstance(result, dict) else {}
383
+ except Exception as e:
384
+ logger.error(f"Post scoring failed: {e}")
385
+ return {}
linkedin_agent/main_agent.py ADDED
@@ -0,0 +1,279 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ LinkedIn Agent β€” Personal AI OS
3
+ =================================
4
+ Monitors your LinkedIn feed, drafts connection messages,
5
+ suggests engagement targets, helps build professional presence.
6
+
7
+ Triggers:
8
+ - Daily 9:30 AM feed scan + engagement suggestions
9
+ - On-demand: draft connection request, comment, post
10
+ - CLI: python main_agent.py --feed | --draft-post | --connect "Name"
11
+ """
12
+
13
+ import os
14
+ import time
15
+ import logging
16
+ import argparse
17
+ import schedule
18
+ from datetime import datetime
19
+ import os as _os
20
+ from dotenv import load_dotenv
21
+
22
+ from data_fetcher import (
23
+ fetch_linkedin_feed,
24
+ fetch_profile_info,
25
+ fetch_connection_suggestions,
26
+ fetch_my_profile,
27
+ )
28
+ from llm import (
29
+ analyze_feed,
30
+ draft_connection_message,
31
+ draft_comment,
32
+ draft_post,
33
+ generate_engagement_plan,
34
+ score_post_idea,
35
+ )
36
+ from linkedin_store import LinkedInStore
37
+ from delivery import (
38
+ send_digest_email,
39
+ send_whatsapp_alert,
40
+ save_to_notion,
41
+ save_drafts_to_markdown,
42
+ )
43
+
44
+ load_dotenv(_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "..", ".env"))
45
+ logging.basicConfig(
46
+ level=logging.INFO,
47
+ format="%(asctime)s [%(levelname)s] %(name)s β€” %(message)s",
48
+ handlers=[
49
+ logging.FileHandler("linkedin_agent.log"),
50
+ logging.StreamHandler(),
51
+ ],
52
+ )
53
+ logger = logging.getLogger("LinkedInAgent")
54
+
55
+ # ── config ────────────────────────────────────────────────────────────────────
56
+ DELIVER_EMAIL = os.getenv("DELIVER_LINKEDIN_EMAIL", "true").lower() == "true"
57
+ DELIVER_WHATSAPP = os.getenv("DELIVER_LINKEDIN_WHATSAPP", "false").lower() == "true"
58
+ SAVE_NOTION = os.getenv("SAVE_LINKEDIN_NOTION", "true").lower() == "true"
59
+ SAVE_MARKDOWN = os.getenv("SAVE_LINKEDIN_MARKDOWN", "true").lower() == "true"
60
+ USER_EMAIL = os.getenv("USER_EMAIL", "")
61
+ USER_NAME = os.getenv("LINKEDIN_USER_NAME", "")
62
+ USER_ROLE = os.getenv("LINKEDIN_USER_ROLE", "")
63
+ USER_INDUSTRY = os.getenv("LINKEDIN_USER_INDUSTRY", "")
64
+ NICHE_TOPICS = os.getenv("LINKEDIN_NICHE_TOPICS", "AI, technology, startups")
65
+
66
+
67
+ # ── core pipelines ────────────────────────────────────────────────────────────
68
+
69
+ def run_daily_digest(trigger: str = "daily_930am") -> dict:
70
+ """
71
+ Full daily LinkedIn pipeline:
72
+ feed scan β†’ engagement suggestions β†’ post ideas β†’ delivery
73
+ """
74
+ logger.info(f"πŸ”— LinkedIn Agent pipeline | trigger={trigger}")
75
+ start = datetime.now()
76
+ store = LinkedInStore()
77
+
78
+ user_context = {
79
+ "name": USER_NAME,
80
+ "role": USER_ROLE,
81
+ "industry": USER_INDUSTRY,
82
+ "topics": NICHE_TOPICS,
83
+ }
84
+
85
+ # 1. Fetch feed posts
86
+ logger.info("πŸ“° Fetching LinkedIn feed …")
87
+ feed_posts = fetch_linkedin_feed()
88
+ logger.info(f" Fetched {len(feed_posts)} feed posts")
89
+
90
+ # 2. Fetch connection suggestions
91
+ logger.info("🀝 Fetching connection suggestions …")
92
+ suggestions = fetch_connection_suggestions()
93
+ logger.info(f" Found {len(suggestions)} suggestions")
94
+
95
+ # 3. Analyse feed β€” find posts worth engaging with
96
+ logger.info("πŸ€– Analysing feed via Groq …")
97
+ feed_analysis = analyze_feed(feed_posts, user_context)
98
+
99
+ # 4. Generate engagement plan (who to comment on, what to say)
100
+ logger.info("πŸ“‹ Generating engagement plan …")
101
+ engagement_plan = generate_engagement_plan(
102
+ feed_analysis=feed_analysis,
103
+ suggestions=suggestions,
104
+ user_context=user_context,
105
+ )
106
+
107
+ # 5. Draft comments for top engagement targets
108
+ drafted_comments = []
109
+ for target in engagement_plan.get("comment_targets", [])[:3]:
110
+ comment = draft_comment(
111
+ post=target,
112
+ user_context=user_context,
113
+ )
114
+ target["drafted_comment"] = comment
115
+ drafted_comments.append(target)
116
+ logger.info(f" Drafted comment for: {target.get('author', '?')}")
117
+
118
+ # 6. Draft connection messages for top suggestions
119
+ drafted_connections = []
120
+ for person in engagement_plan.get("connect_targets", [])[:3]:
121
+ msg = draft_connection_message(person=person, user_context=user_context)
122
+ person["drafted_message"] = msg
123
+ drafted_connections.append(person)
124
+ logger.info(f" Drafted connection message for: {person.get('name', '?')}")
125
+
126
+ # 7. Generate a post idea for today
127
+ logger.info("✍️ Generating post idea …")
128
+ post_idea = draft_post(
129
+ user_context=user_context,
130
+ trending_topics=feed_analysis.get("trending_topics", []),
131
+ post_type="thought_leadership",
132
+ )
133
+
134
+ # 8. Store
135
+ store.save_feed_snapshot(feed_posts, engagement_plan)
136
+ store.save_drafts(drafted_comments, drafted_connections, post_idea)
137
+
138
+ elapsed = (datetime.now() - start).seconds
139
+ report = {
140
+ "trigger": trigger,
141
+ "timestamp": datetime.now().isoformat(),
142
+ "feed_posts_scanned": len(feed_posts),
143
+ "feed_analysis": feed_analysis,
144
+ "engagement_plan": engagement_plan,
145
+ "drafted_comments": drafted_comments,
146
+ "drafted_connections": drafted_connections,
147
+ "post_idea": post_idea,
148
+ "elapsed_seconds": elapsed,
149
+ }
150
+
151
+ logger.info(f"βœ… LinkedIn digest ready in {elapsed}s")
152
+ _deliver(report)
153
+ return report
154
+
155
+
156
+ def run_connect(name: str, role: str = "", company: str = "") -> str:
157
+ """Draft a connection request message for a specific person."""
158
+ logger.info(f"🀝 Drafting connection message for: {name}")
159
+ user_context = {
160
+ "name": USER_NAME, "role": USER_ROLE,
161
+ "industry": USER_INDUSTRY, "topics": NICHE_TOPICS,
162
+ }
163
+ person = {"name": name, "role": role, "company": company}
164
+ profile_info = fetch_profile_info(name)
165
+ if profile_info:
166
+ person.update(profile_info)
167
+
168
+ message = draft_connection_message(person=person, user_context=user_context)
169
+ logger.info(" Message drafted")
170
+ return message
171
+
172
+
173
+ def run_draft_post(post_type: str = "thought_leadership", topic: str = "") -> dict:
174
+ """Draft a LinkedIn post on demand."""
175
+ logger.info(f"✍️ Drafting post | type={post_type} topic={topic}")
176
+ user_context = {
177
+ "name": USER_NAME, "role": USER_ROLE,
178
+ "industry": USER_INDUSTRY, "topics": NICHE_TOPICS,
179
+ }
180
+ post = draft_post(
181
+ user_context=user_context,
182
+ trending_topics=[topic] if topic else [],
183
+ post_type=post_type,
184
+ )
185
+ store = LinkedInStore()
186
+ store.save_drafts([], [], post)
187
+ return post
188
+
189
+
190
+ def _deliver(report: dict):
191
+ if SAVE_MARKDOWN:
192
+ path = save_drafts_to_markdown(report)
193
+ logger.info(f"πŸ“„ Drafts saved to {path}")
194
+
195
+ if SAVE_NOTION:
196
+ url = save_to_notion(report)
197
+ if url:
198
+ logger.info(f"πŸ““ Saved to Notion: {url}")
199
+
200
+ if DELIVER_EMAIL and USER_EMAIL:
201
+ send_digest_email(report, recipient=USER_EMAIL)
202
+ logger.info("πŸ“§ Digest email sent")
203
+
204
+ if DELIVER_WHATSAPP:
205
+ send_whatsapp_alert(report)
206
+ logger.info("πŸ“± WhatsApp alert sent")
207
+
208
+
209
+ # ── scheduler ─────────────────────────────────────────────────────────────────
210
+
211
+ def run_scheduler():
212
+ schedule.every().day.at("09:30").do(
213
+ lambda: run_daily_digest(trigger="daily_930am")
214
+ )
215
+ logger.info("⏰ Daily digest scheduled at 09:30")
216
+ while True:
217
+ schedule.run_pending()
218
+ time.sleep(30)
219
+
220
+
221
+ # ── CLI ───────────────────────────────────────────────────────────────────────
222
+
223
+ def _parse_args():
224
+ parser = argparse.ArgumentParser(description="LinkedIn Agent β€” Personal AI OS")
225
+ parser.add_argument("--feed", action="store_true", help="Run full daily digest")
226
+ parser.add_argument("--connect", type=str, help="Draft connection message for NAME")
227
+ parser.add_argument("--role", type=str, default="", help="Role of person to connect with")
228
+ parser.add_argument("--company", type=str, default="", help="Company of person to connect with")
229
+ parser.add_argument("--draft-post", action="store_true", help="Draft a LinkedIn post")
230
+ parser.add_argument("--topic", type=str, default="", help="Topic for the post")
231
+ parser.add_argument(
232
+ "--post-type",
233
+ choices=["thought_leadership", "how_to", "story", "list", "hot_take", "celebration"],
234
+ default="thought_leadership",
235
+ )
236
+ parser.add_argument("--daemon", action="store_true", help="Run with daily scheduler")
237
+ return parser.parse_args()
238
+
239
+
240
+ def main():
241
+ logger.info("=" * 60)
242
+ logger.info(" LINKEDIN AGENT β€” Personal AI OS")
243
+ logger.info("=" * 60)
244
+
245
+ args = _parse_args()
246
+
247
+ if args.feed:
248
+ report = run_daily_digest(trigger="cli_feed")
249
+ plan = report.get("engagement_plan", {})
250
+ print(f"\nπŸ“° Scanned {report['feed_posts_scanned']} posts")
251
+ print(f"πŸ’¬ Comment targets: {len(report['drafted_comments'])}")
252
+ print(f"🀝 Connect targets: {len(report['drafted_connections'])}")
253
+ print(f"\n✍️ Post idea:\n{report['post_idea'].get('content','')[:300]}")
254
+
255
+ elif args.connect:
256
+ message = run_connect(args.connect, role=args.role, company=args.company)
257
+ print(f"\n🀝 Connection message for {args.connect}:\n{'-'*40}\n{message}\n{'-'*40}")
258
+
259
+ elif args.draft_post:
260
+ post = run_draft_post(post_type=args.post_type, topic=args.topic)
261
+ print(f"\n✍️ Draft post ({args.post_type}):\n{'='*50}")
262
+ print(post.get("content", ""))
263
+ print(f"\nπŸ“Š Hook score: {post.get('hook_score', '?')}/10")
264
+ print(f"🏷️ Hashtags: {' '.join(post.get('hashtags', []))}")
265
+
266
+ elif args.daemon:
267
+ run_daily_digest(trigger="startup")
268
+ run_scheduler()
269
+
270
+ else:
271
+ print("Usage:")
272
+ print(" python main_agent.py --feed")
273
+ print(" python main_agent.py --connect 'Priya Sharma' --role 'CTO' --company 'Zepto'")
274
+ print(" python main_agent.py --draft-post --post-type thought_leadership --topic 'AI agents'")
275
+ print(" python main_agent.py --daemon")
276
+
277
+
278
+ if __name__ == "__main__":
279
+ main()
linkedin_agent/requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ groq>=0.8.0
2
+ requests>=2.31.0
3
+ schedule>=1.2.1
4
+ python-dotenv>=1.0.1
5
+ twilio>=8.13.0
task_manager_agent/README.md ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Task Manager Agent β€” Personal AI OS
2
+
3
+ Automatically turns emails and meeting notes into prioritized, synced tasks.
4
+
5
+ ## What it does
6
+
7
+ | Capability | Detail |
8
+ |---|---|
9
+ | **Email β†’ Tasks** | Scans inbox every 15 min, extracts actionable items via Groq |
10
+ | **Meeting β†’ Tasks** | Parses Google Calendar descriptions for action items and follow-ups |
11
+ | **Prioritization** | LLM scores each task 1–10 using urgency + impact + effort |
12
+ | **Deduplication** | Never creates the same task twice |
13
+ | **Sync** | Pushes to Notion DB and/or Todoist |
14
+ | **Digest** | Sends a formatted email + optional WhatsApp summary |
15
+ | **Triggers** | New email detection (polling) + daily 9 AM scheduled sync |
16
+
17
+ ---
18
+
19
+ ## File structure
20
+
21
+ ```
22
+ task_manager_agent/
23
+ β”œβ”€β”€ main_agent.py # Orchestrator, scheduler, email watcher
24
+ β”œβ”€β”€ data_fetcher.py # Gmail + Calendar + existing task fetch
25
+ β”œβ”€β”€ llm.py # Groq extraction + prioritization prompts
26
+ β”œβ”€β”€ task_store.py # Local JSON persistence + dedup
27
+ β”œβ”€β”€ delivery.py # Email SMTP, WhatsApp, Notion, Todoist
28
+ β”œβ”€β”€ .env.example # All env vars with explanations
29
+ β”œβ”€β”€ requirements.txt
30
+ └── README.md
31
+ ```
32
+
33
+ ---
34
+
35
+ ## Setup
36
+
37
+ ### 1. Install dependencies
38
+ ```bash
39
+ pip install -r requirements.txt
40
+ ```
41
+
42
+ ### 2. Configure environment
43
+ ```bash
44
+ cp .env.example .env
45
+ # Edit .env with your credentials
46
+ ```
47
+
48
+ ### 3. Google credentials
49
+ Share the same `credentials.json` and `token.json` from your earlier agents.
50
+ The agent needs scopes:
51
+ - `gmail.readonly`
52
+ - `calendar.readonly`
53
+
54
+ ### 4. Notion Database setup
55
+
56
+ Create a Notion database with these properties:
57
+
58
+ | Property | Type |
59
+ |---|---|
60
+ | Name | Title |
61
+ | Status | Select: `To Do`, `In Progress`, `Done` |
62
+ | Priority | Select: `Critical`, `High`, `Medium`, `Low` |
63
+ | Due Date | Date |
64
+ | Category | Select: `Work`, `Personal`, `Admin`, `Communication`, `Research`, `Finance`, `Health`, `Other` |
65
+ | Source | Rich Text |
66
+ | Priority Score | Number |
67
+ | Notes | Rich Text |
68
+
69
+ Copy the DB ID from the Notion URL:
70
+ `https://notion.so/your-workspace/`**`THIS-IS-YOUR-DB-ID`**`?v=...`
71
+
72
+ ### 5. Run
73
+ ```bash
74
+ python main_agent.py
75
+ ```
76
+
77
+ On startup the agent runs immediately, then enters the continuous loop:
78
+ - Email watcher polls every `EMAIL_POLL_INTERVAL_MINUTES` minutes
79
+ - Full sync fires daily at **09:00**
80
+
81
+ ---
82
+
83
+ ## How prioritization works
84
+
85
+ The LLM scores each task using four axes:
86
+
87
+ ```
88
+ Priority Score (1-10) = f(urgency, impact, effort, dependencies)
89
+ ```
90
+
91
+ | Score | Label | Example |
92
+ |---|---|---|
93
+ | 9–10 | Critical | "Contract due tomorrow, client blocked" |
94
+ | 7–8 | High | "Respond to investor by Friday" |
95
+ | 5–6 | Medium | "Update project docs" |
96
+ | 1–4 | Low | "Read that article someone forwarded" |
97
+
98
+ ---
99
+
100
+ ## Cron alternative
101
+
102
+ To run via system cron instead of the built-in scheduler:
103
+
104
+ ```cron
105
+ # Daily 9 AM sync
106
+ 0 9 * * * cd /path/to/task_manager_agent && python -c "from main_agent import run_task_extraction_pipeline; run_task_extraction_pipeline('cron_9am')"
107
+
108
+ # Email polling every 15 min
109
+ */15 * * * * cd /path/to/task_manager_agent && python -c "from main_agent import run_task_extraction_pipeline; run_task_extraction_pipeline('email_poll')"
110
+ ```
111
+
112
+ ---
113
+
114
+ ## Agent position in Personal AI OS
115
+
116
+ ```
117
+ 01 βœ… Daily Planner Agent
118
+ 02 βœ… Email Agent
119
+ 03 βœ… Meeting Prep Agent
120
+ 04 βœ… End-of-Day Review Agent
121
+ 05 βœ… Task Manager Agent ← YOU ARE HERE
122
+ 06 Research Agent
123
+ 07 Finance Agent
124
+ 08 LinkedIn Agent
125
+ 09 Knowledge Agent
126
+ 10 Master Orchestrator (Mem0 + LangGraph)
127
+ ```
128
+
129
+ The Master Orchestrator will call `run_task_extraction_pipeline()` directly
130
+ and read from `TaskStore` to feed task context into other agents.
task_manager_agent/data_fetcher.py ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ data_fetcher.py β€” Task Manager Agent
3
+ =====================================
4
+ Pulls raw data from Gmail, Google Calendar, and existing task stores.
5
+ """
6
+
7
+ import os
8
+ import base64
9
+ import logging
10
+ from datetime import datetime, timedelta, timezone
11
+ from typing import Optional
12
+
13
+ from google.oauth2.credentials import Credentials
14
+ from google.auth.transport.requests import Request
15
+ from google_auth_oauthlib.flow import InstalledAppFlow
16
+ from googleapiclient.discovery import build
17
+
18
+ logger = logging.getLogger("TaskManagerAgent.DataFetcher")
19
+
20
+ SCOPES = [
21
+ "https://www.googleapis.com/auth/gmail.readonly",
22
+ "https://www.googleapis.com/auth/calendar.readonly",
23
+ ]
24
+
25
+ CREDENTIALS_FILE = os.getenv("GOOGLE_CREDENTIALS_FILE")
26
+ TOKEN_FILE = os.getenv("GOOGLE_TOKEN_FILE")
27
+
28
+ # Labels that signal actionable emails (expand as needed)
29
+ ACTIONABLE_LABELS = {"INBOX", "IMPORTANT", "STARRED"}
30
+
31
+
32
+ # ── Google auth ───────────────────────────────────────────────────────────────
33
+
34
+ def _get_google_creds() -> Credentials:
35
+ creds = None
36
+ if os.path.exists(TOKEN_FILE):
37
+ creds = Credentials.from_authorized_user_file(TOKEN_FILE, SCOPES)
38
+ if not creds or not creds.valid:
39
+ if creds and creds.expired and creds.refresh_token:
40
+ creds.refresh(Request())
41
+ else:
42
+ flow = InstalledAppFlow.from_client_secrets_file(CREDENTIALS_FILE, SCOPES)
43
+ creds = flow.run_local_server(port=0)
44
+ with open(TOKEN_FILE, "w") as f:
45
+ f.write(creds.to_json())
46
+ return creds
47
+
48
+
49
+ def _gmail_service():
50
+ return build("gmail", "v1", credentials=_get_google_creds())
51
+
52
+
53
+ def _calendar_service():
54
+ return build("calendar", "v3", credentials=_get_google_creds())
55
+
56
+
57
+ # ── Gmail ─────────────────────────────────────────────────────────────────────
58
+
59
+ def fetch_recent_emails(
60
+ max_results: int = 20,
61
+ hours_back: int = 24,
62
+ since_id: Optional[str] = None,
63
+ ) -> list[dict]:
64
+ """
65
+ Returns list of dicts with keys:
66
+ id, subject, sender, date, snippet, body (plain text, truncated)
67
+ """
68
+ try:
69
+ service = _gmail_service()
70
+ after_ts = int((datetime.now(timezone.utc) - timedelta(hours=hours_back)).timestamp())
71
+ query = f"after:{after_ts} -category:promotions -category:social"
72
+
73
+ result = service.users().messages().list(
74
+ userId="me", q=query, maxResults=max_results
75
+ ).execute()
76
+ messages = result.get("messages", [])
77
+
78
+ emails = []
79
+ for msg_ref in messages:
80
+ if since_id and msg_ref["id"] == since_id:
81
+ break
82
+ try:
83
+ msg = service.users().messages().get(
84
+ userId="me", id=msg_ref["id"], format="full"
85
+ ).execute()
86
+ emails.append(_parse_email(msg))
87
+ except Exception as e:
88
+ logger.warning(f"Could not fetch message {msg_ref['id']}: {e}")
89
+
90
+ logger.debug(f"Fetched {len(emails)} emails")
91
+ return emails
92
+
93
+ except Exception as e:
94
+ logger.error(f"Gmail fetch error: {e}", exc_info=True)
95
+ return []
96
+
97
+
98
+ def _parse_email(msg: dict) -> dict:
99
+ headers = {h["name"]: h["value"] for h in msg["payload"].get("headers", [])}
100
+ body = _extract_body(msg["payload"])
101
+ return {
102
+ "id": msg["id"],
103
+ "thread_id": msg.get("threadId", ""),
104
+ "subject": headers.get("Subject", "(no subject)"),
105
+ "sender": headers.get("From", ""),
106
+ "to": headers.get("To", ""),
107
+ "date": headers.get("Date", ""),
108
+ "snippet": msg.get("snippet", ""),
109
+ "body": body[:3000], # cap at 3k chars
110
+ "labels": msg.get("labelIds", []),
111
+ }
112
+
113
+
114
+ def _extract_body(payload: dict) -> str:
115
+ """Recursively extracts plain-text body from MIME payload."""
116
+ if payload.get("mimeType") == "text/plain":
117
+ data = payload.get("body", {}).get("data", "")
118
+ if data:
119
+ return base64.urlsafe_b64decode(data).decode("utf-8", errors="replace")
120
+ for part in payload.get("parts", []):
121
+ text = _extract_body(part)
122
+ if text:
123
+ return text
124
+ return ""
125
+
126
+
127
+ # ── Google Calendar ───────────────────────────────────────────────────────────
128
+
129
+ def fetch_calendar_meetings(
130
+ days_back: int = 1,
131
+ days_ahead: int = 0,
132
+ calendar_id: str = "primary",
133
+ ) -> list[dict]:
134
+ """
135
+ Returns meetings from [now - days_back, now + days_ahead].
136
+ Each dict: id, title, start, end, attendees, description, location
137
+ """
138
+ try:
139
+ service = _calendar_service()
140
+ now = datetime.now(timezone.utc)
141
+ time_min = (now - timedelta(days=days_back)).isoformat()
142
+ time_max = (now + timedelta(days=days_ahead + 1)).isoformat()
143
+
144
+ events_result = service.events().list(
145
+ calendarId=calendar_id,
146
+ timeMin=time_min,
147
+ timeMax=time_max,
148
+ maxResults=50,
149
+ singleEvents=True,
150
+ orderBy="startTime",
151
+ ).execute()
152
+ events = events_result.get("items", [])
153
+
154
+ meetings = []
155
+ for e in events:
156
+ if e.get("status") == "cancelled":
157
+ continue
158
+ meetings.append({
159
+ "id": e.get("id", ""),
160
+ "title": e.get("summary", "(untitled)"),
161
+ "start": e.get("start", {}).get("dateTime", e.get("start", {}).get("date", "")),
162
+ "end": e.get("end", {}).get("dateTime", e.get("end", {}).get("date", "")),
163
+ "attendees": [
164
+ a.get("email", "") for a in e.get("attendees", [])
165
+ ],
166
+ "description": e.get("description", "")[:2000],
167
+ "location": e.get("location", ""),
168
+ })
169
+
170
+ logger.debug(f"Fetched {len(meetings)} meetings")
171
+ return meetings
172
+
173
+ except Exception as e:
174
+ logger.error(f"Calendar fetch error: {e}", exc_info=True)
175
+ return []
176
+
177
+
178
+ # ── Existing tasks (local store read) ─────────────────────────────────────────
179
+
180
+ def fetch_existing_tasks() -> list[dict]:
181
+ """Loads tasks from local task store + Notion for deduplication context."""
182
+ tasks = []
183
+
184
+ # local
185
+ local_path = os.getenv("LOCAL_TASKS_FILE", "tasks.json")
186
+ if os.path.exists(local_path):
187
+ import json
188
+ try:
189
+ with open(local_path) as f:
190
+ tasks.extend(json.load(f))
191
+ logger.debug(f"Loaded {len(tasks)} local tasks")
192
+ except Exception as e:
193
+ logger.warning(f"Could not load local tasks: {e}")
194
+
195
+ return tasks
196
+
197
+
198
+ def fetch_notion_tasks() -> list[dict]:
199
+ """Fetches open tasks from Notion DB."""
200
+ import requests
201
+
202
+ notion_token = os.getenv("NOTION_TOKEN")
203
+ db_id = os.getenv("NOTION_TASKS_DB_ID")
204
+ if not notion_token or not db_id:
205
+ logger.debug("Notion credentials not set β€” skipping Notion task fetch")
206
+ return []
207
+
208
+ try:
209
+ headers = {
210
+ "Authorization": f"Bearer {notion_token}",
211
+ "Notion-Version": "2022-06-28",
212
+ "Content-Type": "application/json",
213
+ }
214
+ payload = {
215
+ "filter": {
216
+ "property": "Status",
217
+ "select": {"does_not_equal": "Done"},
218
+ }
219
+ }
220
+ resp = requests.post(
221
+ f"https://api.notion.com/v1/databases/{db_id}/query",
222
+ headers=headers, json=payload, timeout=10,
223
+ )
224
+ resp.raise_for_status()
225
+ results = resp.json().get("results", [])
226
+
227
+ tasks = []
228
+ for page in results:
229
+ props = page.get("properties", {})
230
+ title_prop = props.get("Name", {}).get("title", [])
231
+ title = title_prop[0]["plain_text"] if title_prop else "(no title)"
232
+ tasks.append({
233
+ "id": page["id"],
234
+ "title": title,
235
+ "source": "notion",
236
+ "url": page.get("url", ""),
237
+ })
238
+ logger.debug(f"Fetched {len(tasks)} tasks from Notion")
239
+ return tasks
240
+
241
+ except Exception as e:
242
+ logger.error(f"Notion fetch error: {e}", exc_info=True)
243
+ return []
task_manager_agent/delivery.py ADDED
@@ -0,0 +1,362 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ delivery.py β€” Task Manager Agent
3
+ =================================
4
+ Handles all delivery methods for task digests:
5
+ - Email (Gmail SMTP)
6
+ - WhatsApp (Twilio)
7
+ - Notion sync
8
+ - Todoist sync
9
+ """
10
+
11
+ import os
12
+ import json
13
+ import logging
14
+ import smtplib
15
+ from email.mime.text import MIMEText
16
+ from email.mime.multipart import MIMEMultipart
17
+ from datetime import datetime
18
+ from typing import Optional
19
+
20
+ from dotenv import load_dotenv
21
+ load_dotenv()
22
+
23
+ logger = logging.getLogger("TaskManagerAgent.Delivery")
24
+
25
+ # ── Config ────────────────────────────────────────────────────────────────────
26
+ GMAIL_ADDRESS = os.getenv("GMAIL_ADDRESS", "")
27
+ GMAIL_APP_PASSWORD = os.getenv("GMAIL_APP_PASSWORD", "")
28
+ TWILIO_SID = os.getenv("TWILIO_ACCOUNT_SID", "")
29
+ TWILIO_TOKEN = os.getenv("TWILIO_AUTH_TOKEN", "")
30
+ TWILIO_FROM = os.getenv("TWILIO_WHATSAPP_FROM", "")
31
+ WHATSAPP_TO = os.getenv("WHATSAPP_TO_NUMBER", "")
32
+ NOTION_API_KEY = os.getenv("NOTION_API_KEY", "")
33
+ NOTION_DATABASE_ID = os.getenv("NOTION_DATABASE_ID", "")
34
+ TODOIST_API_KEY = os.getenv("TODOIST_API_KEY", "")
35
+
36
+
37
+ # ── Formatters ────────────────────────────────────────────────────────────────
38
+
39
+ def _format_task_digest_text(tasks: list[dict], trigger: str = "scheduled") -> str:
40
+ """Format tasks into a plain-text digest."""
41
+ now = datetime.now().strftime("%A, %d %b %Y %H:%M")
42
+ lines = [
43
+ f"TASK DIGEST β€” {now}",
44
+ f"Trigger: {trigger}",
45
+ f"Total new tasks: {len(tasks)}",
46
+ "=" * 50,
47
+ ]
48
+ for i, task in enumerate(tasks, 1):
49
+ title = task.get("title", "Untitled")
50
+ priority = task.get("priority", "medium")
51
+ deadline = task.get("deadline", "No deadline")
52
+ category = task.get("category", "general")
53
+ source = task.get("source", "unknown")
54
+ lines.append(
55
+ f"\n{i}. [{priority.upper()}] {title}\n"
56
+ f" Category : {category}\n"
57
+ f" Deadline : {deadline}\n"
58
+ f" Source : {source}"
59
+ )
60
+ lines.append("\n" + "=" * 50)
61
+ return "\n".join(lines)
62
+
63
+
64
+ def _format_task_digest_html(tasks: list[dict], trigger: str = "scheduled") -> str:
65
+ """Format tasks into an HTML email digest."""
66
+ now = datetime.now().strftime("%A, %d %b %Y %H:%M")
67
+
68
+ priority_colors = {
69
+ "high": "#ef4444",
70
+ "medium": "#f59e0b",
71
+ "low": "#22c55e",
72
+ }
73
+
74
+ rows = ""
75
+ for i, task in enumerate(tasks, 1):
76
+ title = task.get("title", "Untitled")
77
+ priority = task.get("priority", "medium").lower()
78
+ deadline = task.get("deadline", "No deadline")
79
+ category = task.get("category", "general")
80
+ source = task.get("source", "unknown")
81
+ color = priority_colors.get(priority, "#f59e0b")
82
+ rows += f"""
83
+ <tr>
84
+ <td style="padding:10px;border-bottom:1px solid #1e2840;color:#c8d6f0">{i}</td>
85
+ <td style="padding:10px;border-bottom:1px solid #1e2840;color:#e8f0ff;font-weight:600">{title}</td>
86
+ <td style="padding:10px;border-bottom:1px solid #1e2840">
87
+ <span style="background:{color}22;color:{color};padding:2px 8px;border-radius:12px;font-size:12px;border:1px solid {color}44">
88
+ {priority.upper()}
89
+ </span>
90
+ </td>
91
+ <td style="padding:10px;border-bottom:1px solid #1e2840;color:#8aaccc">{deadline}</td>
92
+ <td style="padding:10px;border-bottom:1px solid #1e2840;color:#6b8ab0">{category}</td>
93
+ <td style="padding:10px;border-bottom:1px solid #1e2840;color:#4a6a90">{source}</td>
94
+ </tr>"""
95
+
96
+ return f"""
97
+ <html><body style="background:#0a0e1a;color:#c8d6f0;font-family:sans-serif;padding:24px">
98
+ <h2 style="color:#e8f0ff">πŸ“‹ Task Digest</h2>
99
+ <p style="color:#6b8ab0">{now} &nbsp;Β·&nbsp; Trigger: {trigger} &nbsp;Β·&nbsp; {len(tasks)} new tasks</p>
100
+ <table style="width:100%;border-collapse:collapse;background:#0d1220;border-radius:10px;overflow:hidden">
101
+ <thead>
102
+ <tr style="background:#111827">
103
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">#</th>
104
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">TASK</th>
105
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">PRIORITY</th>
106
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">DEADLINE</th>
107
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">CATEGORY</th>
108
+ <th style="padding:10px;text-align:left;color:#4a6a90;font-size:12px">SOURCE</th>
109
+ </tr>
110
+ </thead>
111
+ <tbody>{rows}</tbody>
112
+ </table>
113
+ <p style="color:#3a5070;font-size:12px;margin-top:16px">Personal AI OS β€” Task Manager Agent</p>
114
+ </body></html>"""
115
+
116
+
117
+ # ── Email delivery ────────────────────────────────────────────────────────────
118
+
119
+ def send_task_digest_email(
120
+ tasks: list[dict],
121
+ trigger: str = "scheduled",
122
+ recipient: str = "",
123
+ ) -> bool:
124
+ """
125
+ Send task digest via Gmail SMTP.
126
+
127
+ Args:
128
+ tasks: List of task dicts.
129
+ trigger: What triggered this run (for subject line).
130
+ recipient: Email address to send to.
131
+
132
+ Returns:
133
+ True if sent successfully, False otherwise.
134
+ """
135
+ if not GMAIL_ADDRESS or not GMAIL_APP_PASSWORD:
136
+ logger.warning("Email delivery skipped β€” GMAIL_ADDRESS or GMAIL_APP_PASSWORD not set")
137
+ return False
138
+
139
+ to_addr = recipient or GMAIL_ADDRESS
140
+
141
+ try:
142
+ msg = MIMEMultipart("alternative")
143
+ msg["Subject"] = f"πŸ“‹ Task Digest β€” {len(tasks)} new tasks ({trigger})"
144
+ msg["From"] = GMAIL_ADDRESS
145
+ msg["To"] = to_addr
146
+
147
+ text_part = MIMEText(_format_task_digest_text(tasks, trigger), "plain")
148
+ html_part = MIMEText(_format_task_digest_html(tasks, trigger), "html")
149
+ msg.attach(text_part)
150
+ msg.attach(html_part)
151
+
152
+ with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
153
+ server.login(GMAIL_ADDRESS, GMAIL_APP_PASSWORD)
154
+ server.sendmail(GMAIL_ADDRESS, to_addr, msg.as_string())
155
+
156
+ logger.info(f"βœ“ Task digest email sent to {to_addr}")
157
+ return True
158
+
159
+ except Exception as e:
160
+ logger.error(f"βœ— Email delivery failed: {e}")
161
+ return False
162
+
163
+
164
+ # ── WhatsApp delivery ─────────────────────────────────────────────────────────
165
+
166
+ def send_task_digest_whatsapp(
167
+ tasks: list[dict],
168
+ trigger: str = "scheduled",
169
+ ) -> bool:
170
+ """
171
+ Send task digest via Twilio WhatsApp.
172
+
173
+ Returns:
174
+ True if sent successfully, False otherwise.
175
+ """
176
+ if not TWILIO_SID or not TWILIO_TOKEN or not TWILIO_FROM or not WHATSAPP_TO:
177
+ logger.warning("WhatsApp delivery skipped β€” Twilio credentials not set")
178
+ return False
179
+
180
+ try:
181
+ from twilio.rest import Client
182
+ client = Client(TWILIO_SID, TWILIO_TOKEN)
183
+
184
+ # Build compact WhatsApp message
185
+ now = datetime.now().strftime("%d %b %H:%M")
186
+ lines = [f"πŸ“‹ *Task Digest* β€” {now}", f"_{trigger} Β· {len(tasks)} new tasks_\n"]
187
+
188
+ for i, task in enumerate(tasks[:10], 1): # cap at 10 for WhatsApp
189
+ pri = task.get("priority", "medium").upper()
190
+ title = task.get("title", "Untitled")
191
+ dl = task.get("deadline", "")
192
+ dl_str = f" Β· {dl}" if dl and dl != "No deadline" else ""
193
+ lines.append(f"{i}. [{pri}] {title}{dl_str}")
194
+
195
+ if len(tasks) > 10:
196
+ lines.append(f"\n_...and {len(tasks) - 10} more tasks_")
197
+
198
+ message = "\n".join(lines)
199
+
200
+ client.messages.create(
201
+ from_=f"whatsapp:{TWILIO_FROM}",
202
+ to=f"whatsapp:{WHATSAPP_TO}",
203
+ body=message,
204
+ )
205
+
206
+ logger.info(f"βœ“ Task digest WhatsApp sent to {WHATSAPP_TO}")
207
+ return True
208
+
209
+ except Exception as e:
210
+ logger.error(f"βœ— WhatsApp delivery failed: {e}")
211
+ return False
212
+
213
+
214
+ # ── Notion sync ───────────────────────────────────────────────────────────────
215
+
216
+ def sync_to_notion(tasks: list[dict]) -> int:
217
+ """
218
+ Sync tasks to a Notion database.
219
+
220
+ Each task becomes a Notion page with:
221
+ - Title, Priority, Deadline, Category, Source, Status properties.
222
+
223
+ Returns:
224
+ Number of tasks successfully synced.
225
+ """
226
+ if not NOTION_API_KEY or not NOTION_DATABASE_ID:
227
+ logger.warning("Notion sync skipped β€” NOTION_API_KEY or NOTION_DATABASE_ID not set")
228
+ return 0
229
+
230
+ try:
231
+ import requests
232
+
233
+ headers = {
234
+ "Authorization": f"Bearer {NOTION_API_KEY}",
235
+ "Content-Type": "application/json",
236
+ "Notion-Version": "2022-06-28",
237
+ }
238
+
239
+ synced = 0
240
+ for task in tasks:
241
+ title = task.get("title", "Untitled Task")
242
+ priority = task.get("priority", "medium").capitalize()
243
+ deadline = task.get("deadline", "")
244
+ category = task.get("category", "General")
245
+ source = task.get("source", "AI OS")
246
+
247
+ properties = {
248
+ "Name": {
249
+ "title": [{"text": {"content": title}}]
250
+ },
251
+ "Priority": {
252
+ "select": {"name": priority}
253
+ },
254
+ "Category": {
255
+ "select": {"name": category}
256
+ },
257
+ "Source": {
258
+ "rich_text": [{"text": {"content": source}}]
259
+ },
260
+ "Status": {
261
+ "select": {"name": "To Do"}
262
+ },
263
+ }
264
+
265
+ # Only add deadline if it's a valid date string
266
+ if deadline and deadline not in ("No deadline", "none", ""):
267
+ try:
268
+ # Try to parse common date formats
269
+ for fmt in ("%Y-%m-%d", "%d %b %Y", "%B %d, %Y"):
270
+ try:
271
+ parsed = datetime.strptime(deadline, fmt)
272
+ properties["Deadline"] = {
273
+ "date": {"start": parsed.strftime("%Y-%m-%d")}
274
+ }
275
+ break
276
+ except ValueError:
277
+ continue
278
+ except Exception:
279
+ pass
280
+
281
+ payload = {
282
+ "parent": {"database_id": NOTION_DATABASE_ID},
283
+ "properties": properties,
284
+ }
285
+
286
+ response = requests.post(
287
+ "https://api.notion.com/v1/pages",
288
+ headers=headers,
289
+ json=payload,
290
+ timeout=10,
291
+ )
292
+
293
+ if response.status_code in (200, 201):
294
+ synced += 1
295
+ logger.debug(f" βœ“ Notion: {title}")
296
+ else:
297
+ logger.warning(f" βœ— Notion failed for '{title}': {response.text[:200]}")
298
+
299
+ logger.info(f"βœ“ Synced {synced}/{len(tasks)} tasks to Notion")
300
+ return synced
301
+
302
+ except Exception as e:
303
+ logger.error(f"βœ— Notion sync failed: {e}")
304
+ return 0
305
+
306
+
307
+ # ── Todoist sync ──────────────────────────────────────────────────────────────
308
+
309
+ def sync_to_todoist(tasks: list[dict]) -> int:
310
+ """
311
+ Sync tasks to Todoist via REST API v2.
312
+
313
+ Returns:
314
+ Number of tasks successfully synced.
315
+ """
316
+ if not TODOIST_API_KEY:
317
+ logger.warning("Todoist sync skipped β€” TODOIST_API_KEY not set")
318
+ return 0
319
+
320
+ try:
321
+ import requests
322
+
323
+ headers = {
324
+ "Authorization": f"Bearer {TODOIST_API_KEY}",
325
+ "Content-Type": "application/json",
326
+ }
327
+
328
+ priority_map = {"high": 4, "medium": 3, "low": 2}
329
+ synced = 0
330
+
331
+ for task in tasks:
332
+ title = task.get("title", "Untitled Task")
333
+ priority = priority_map.get(task.get("priority", "medium").lower(), 3)
334
+ deadline = task.get("deadline", "")
335
+
336
+ payload: dict = {
337
+ "content": title,
338
+ "priority": priority,
339
+ }
340
+
341
+ if deadline and deadline not in ("No deadline", "none", ""):
342
+ payload["due_string"] = deadline
343
+
344
+ response = requests.post(
345
+ "https://api.todoist.com/rest/v2/tasks",
346
+ headers=headers,
347
+ json=payload,
348
+ timeout=10,
349
+ )
350
+
351
+ if response.status_code in (200, 204):
352
+ synced += 1
353
+ logger.debug(f" βœ“ Todoist: {title}")
354
+ else:
355
+ logger.warning(f" βœ— Todoist failed for '{title}': {response.text[:200]}")
356
+
357
+ logger.info(f"βœ“ Synced {synced}/{len(tasks)} tasks to Todoist")
358
+ return synced
359
+
360
+ except Exception as e:
361
+ logger.error(f"βœ— Todoist sync failed: {e}")
362
+ return 0
task_manager_agent/llm.py ADDED
@@ -0,0 +1,303 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ llm.py β€” Task Manager Agent
3
+ =============================
4
+ All Groq (llama-3.3-70b-versatile) calls:
5
+ - extract tasks from emails
6
+ - extract tasks from meeting notes
7
+ - prioritize + enrich tasks
8
+ """
9
+
10
+ import os
11
+ import json
12
+ import logging
13
+ import re
14
+ from datetime import datetime
15
+ from typing import Any
16
+
17
+ from groq import Groq
18
+ from dotenv import load_dotenv
19
+ load_dotenv()
20
+
21
+ logger = logging.getLogger("TaskManagerAgent.LLM")
22
+
23
+ client = Groq(api_key=os.getenv("GROQ_API_KEY"))
24
+ MODEL = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")
25
+ TODAY = datetime.now().strftime("%A, %B %d, %Y")
26
+
27
+
28
+ # ── helpers ───────────────────────────────────────────────────────────────────
29
+
30
+ def _chat(system: str, user: str, temperature: float = 0.3) -> str:
31
+ resp = client.chat.completions.create(
32
+ model=MODEL,
33
+ temperature=temperature,
34
+ messages=[
35
+ {"role": "system", "content": system},
36
+ {"role": "user", "content": user},
37
+ ],
38
+ )
39
+ return resp.choices[0].message.content.strip()
40
+
41
+
42
+ def _safe_json(text: str) -> Any:
43
+ """Extract and parse first JSON block from LLM output."""
44
+ # Strip markdown fences
45
+ text = re.sub(r"```(?:json)?", "", text).strip().rstrip("`")
46
+ # Find outermost array or object
47
+ for start_char, end_char in [("[", "]"), ("{", "}")]:
48
+ start = text.find(start_char)
49
+ end = text.rfind(end_char)
50
+ if start != -1 and end != -1:
51
+ try:
52
+ return json.loads(text[start : end + 1])
53
+ except json.JSONDecodeError:
54
+ pass
55
+ logger.warning("Could not parse JSON from LLM response")
56
+ return []
57
+
58
+
59
+ # ── email task extraction ─────────────────────────────────────────────────────
60
+
61
+ EXTRACT_EMAIL_SYSTEM = f"""You are an expert executive assistant AI.
62
+ Today is {TODAY}.
63
+
64
+ Your job is to read emails and extract ACTIONABLE tasks β€” things the user must DO.
65
+ Focus on:
66
+ - Direct requests or assignments ("Please send me…", "Can you prepare…", "I need you to…")
67
+ - Commitments the user made ("I'll get back to you", "Will share by Friday")
68
+ - Deadlines or follow-ups buried in email threads
69
+ - Approvals, decisions, or reviews required
70
+
71
+ Return ONLY a JSON array of task objects. Each task object must have:
72
+ {{
73
+ "title": "short imperative action (verb + object)",
74
+ "description": "1-2 sentence context from the email",
75
+ "source": "email",
76
+ "source_ref": "<email subject>",
77
+ "sender": "<from address>",
78
+ "deadline": "YYYY-MM-DD or null",
79
+ "deadline_confidence": "high|medium|low",
80
+ "estimated_minutes": <number or null>,
81
+ "tags": ["tag1", "tag2"],
82
+ "priority_raw": "urgent|high|medium|low"
83
+ }}
84
+
85
+ Return [] if no actionable tasks are found.
86
+ Do NOT include tasks that are just FYI, news, or promotions.
87
+ Do NOT duplicate tasks already in the existing list."""
88
+
89
+
90
+ def extract_tasks_from_emails(
91
+ emails: list[dict],
92
+ existing_tasks: list[dict],
93
+ ) -> list[dict]:
94
+ if not emails:
95
+ return []
96
+
97
+ existing_titles = [t.get("title", "") for t in existing_tasks]
98
+ existing_context = "\n".join(f"- {t}" for t in existing_titles[:30]) if existing_titles else "None"
99
+
100
+ email_blocks = []
101
+ for i, e in enumerate(emails, 1):
102
+ email_blocks.append(
103
+ f"=== EMAIL {i} ===\n"
104
+ f"From: {e['sender']}\n"
105
+ f"Subject: {e['subject']}\n"
106
+ f"Date: {e['date']}\n"
107
+ f"Body:\n{e['body']}\n"
108
+ )
109
+
110
+ user_prompt = (
111
+ f"EXISTING TASKS (do not duplicate):\n{existing_context}\n\n"
112
+ f"EMAILS TO ANALYSE:\n{''.join(email_blocks)}"
113
+ )
114
+
115
+ try:
116
+ raw = _chat(EXTRACT_EMAIL_SYSTEM, user_prompt)
117
+ tasks = _safe_json(raw)
118
+ if not isinstance(tasks, list):
119
+ tasks = []
120
+ logger.info(f"Email extraction: {len(tasks)} tasks found")
121
+ return tasks
122
+ except Exception as e:
123
+ logger.error(f"Email task extraction failed: {e}", exc_info=True)
124
+ return []
125
+
126
+
127
+ # ── meeting task extraction ───────────────────────────────────────────────────
128
+
129
+ EXTRACT_MEETING_SYSTEM = f"""You are an expert executive assistant AI.
130
+ Today is {TODAY}.
131
+
132
+ Your job is to extract ACTIONABLE tasks from meeting descriptions and calendar events.
133
+ Look for:
134
+ - Action items mentioned in descriptions ("Action: …", "TODO:", "Next steps:")
135
+ - Implied follow-ups (prep materials, send recap, schedule next meeting)
136
+ - Deadlines or deliverables tied to the meeting
137
+
138
+ Return ONLY a JSON array of task objects. Each must have:
139
+ {{
140
+ "title": "short imperative action",
141
+ "description": "context from the meeting",
142
+ "source": "meeting",
143
+ "source_ref": "<meeting title>",
144
+ "sender": null,
145
+ "deadline": "YYYY-MM-DD or null",
146
+ "deadline_confidence": "high|medium|low",
147
+ "estimated_minutes": <number or null>,
148
+ "tags": ["tag1", "tag2"],
149
+ "priority_raw": "urgent|high|medium|low"
150
+ }}
151
+
152
+ Return [] if nothing actionable found.
153
+ Do NOT duplicate tasks already in the existing list."""
154
+
155
+
156
+ def extract_tasks_from_meeting_notes(
157
+ meetings: list[dict],
158
+ existing_tasks: list[dict],
159
+ ) -> list[dict]:
160
+ if not meetings:
161
+ return []
162
+
163
+ existing_titles = [t.get("title", "") for t in existing_tasks]
164
+ existing_context = "\n".join(f"- {t}" for t in existing_titles[:30]) if existing_titles else "None"
165
+
166
+ meeting_blocks = []
167
+ for i, m in enumerate(meetings, 1):
168
+ attendees = ", ".join(m.get("attendees", [])[:5]) or "N/A"
169
+ meeting_blocks.append(
170
+ f"=== MEETING {i} ===\n"
171
+ f"Title: {m['title']}\n"
172
+ f"When: {m['start']} β†’ {m['end']}\n"
173
+ f"Attendees: {attendees}\n"
174
+ f"Description:\n{m.get('description', '(none)')}\n"
175
+ )
176
+
177
+ user_prompt = (
178
+ f"EXISTING TASKS (do not duplicate):\n{existing_context}\n\n"
179
+ f"MEETINGS TO ANALYSE:\n{''.join(meeting_blocks)}"
180
+ )
181
+
182
+ try:
183
+ raw = _chat(EXTRACT_MEETING_SYSTEM, user_prompt)
184
+ tasks = _safe_json(raw)
185
+ if not isinstance(tasks, list):
186
+ tasks = []
187
+ logger.info(f"Meeting extraction: {len(tasks)} tasks found")
188
+ return tasks
189
+ except Exception as e:
190
+ logger.error(f"Meeting task extraction failed: {e}", exc_info=True)
191
+ return []
192
+
193
+
194
+ # ── prioritization ────────────────────────────────────────────────────────────
195
+
196
+ PRIORITIZE_SYSTEM = f"""You are a world-class productivity coach and AI assistant.
197
+ Today is {TODAY}.
198
+
199
+ You will receive a list of raw tasks. Your job is to:
200
+ 1. Score each task's PRIORITY on a 1-10 scale using these criteria:
201
+ - Urgency (deadline proximity)
202
+ - Impact (business / personal significance)
203
+ - Effort required (lower effort = slightly higher priority, all else equal)
204
+ - Dependencies (if others are blocked, bump up)
205
+
206
+ 2. Assign a CATEGORY from: [work, personal, admin, communication, research, finance, health, other]
207
+
208
+ 3. Suggest a DUE DATE if none exists (or confirm/adjust if one does).
209
+
210
+ 4. Write a SHORT "why_urgent" note (1 sentence) for tasks scored 7+.
211
+
212
+ Return ONLY a JSON array. Each object must include ALL original fields PLUS:
213
+ {{
214
+ ...original fields...,
215
+ "priority_score": <1-10>,
216
+ "priority_label": "critical|high|medium|low",
217
+ "category": "<category>",
218
+ "suggested_due_date": "YYYY-MM-DD or null",
219
+ "why_urgent": "one sentence or null",
220
+ "order": <integer starting from 1, lowest = highest priority>
221
+ }}
222
+
223
+ Sort the array by priority_score descending (order 1 = most important)."""
224
+
225
+
226
+ def prioritize_tasks(
227
+ new_tasks: list[dict],
228
+ all_context_tasks: list[dict],
229
+ ) -> list[dict]:
230
+ if not new_tasks:
231
+ return []
232
+
233
+ # Give the LLM context about the full task landscape
234
+ context_summary = []
235
+ for t in all_context_tasks[:20]:
236
+ context_summary.append(
237
+ f"- [{t.get('priority_raw', '?')}] {t.get('title', '?')} "
238
+ f"(deadline: {t.get('deadline', 'none')})"
239
+ )
240
+ context_str = "\n".join(context_summary) if context_summary else "No existing tasks"
241
+
242
+ user_prompt = (
243
+ f"FULL TASK CONTEXT (existing + new):\n{context_str}\n\n"
244
+ f"TASKS TO PRIORITIZE:\n{json.dumps(new_tasks, indent=2)}"
245
+ )
246
+
247
+ try:
248
+ raw = _chat(PRIORITIZE_SYSTEM, user_prompt, temperature=0.2)
249
+ tasks = _safe_json(raw)
250
+ if not isinstance(tasks, list) or not tasks:
251
+ # Fallback: return original tasks with default priority
252
+ logger.warning("Prioritization returned empty β€” using defaults")
253
+ return _apply_default_priority(new_tasks)
254
+ logger.info(f"Prioritized {len(tasks)} tasks")
255
+ return tasks
256
+ except Exception as e:
257
+ logger.error(f"Prioritization failed: {e}", exc_info=True)
258
+ return _apply_default_priority(new_tasks)
259
+
260
+
261
+ def _apply_default_priority(tasks: list[dict]) -> list[dict]:
262
+ priority_map = {"urgent": 9, "high": 7, "medium": 5, "low": 3}
263
+ for i, t in enumerate(tasks):
264
+ score = priority_map.get(t.get("priority_raw", "medium"), 5)
265
+ t["priority_score"] = score
266
+ t["priority_label"] = t.get("priority_raw", "medium")
267
+ t["category"] = "work"
268
+ t["suggested_due_date"] = t.get("deadline")
269
+ t["why_urgent"] = None
270
+ t["order"] = i + 1
271
+ return sorted(tasks, key=lambda x: x["priority_score"], reverse=True)
272
+
273
+
274
+ # ── daily digest summary ──────────────────────────────────────────────────────
275
+
276
+ DIGEST_SYSTEM = f"""You are a sharp, concise executive assistant.
277
+ Today is {TODAY}.
278
+
279
+ Write a crisp task digest. Structure:
280
+ 1. One-line headline ("You have N high-priority tasks today")
281
+ 2. Top 3 critical tasks with a one-line action each
282
+ 3. Quick summary of remaining tasks grouped by category
283
+ 4. One motivational closing line
284
+
285
+ Keep the total under 250 words. Use bullet points sparingly β€” prefer clean paragraphs.
286
+ Do NOT use emoji."""
287
+
288
+
289
+ def generate_digest_summary(tasks: list[dict]) -> str:
290
+ if not tasks:
291
+ return "No new tasks extracted today. Your slate is clean."
292
+ try:
293
+ task_list = json.dumps(
294
+ [{"title": t.get("title"), "priority_score": t.get("priority_score"),
295
+ "deadline": t.get("deadline") or t.get("suggested_due_date"),
296
+ "category": t.get("category"), "why_urgent": t.get("why_urgent")}
297
+ for t in tasks[:15]],
298
+ indent=2
299
+ )
300
+ return _chat(DIGEST_SYSTEM, f"TASKS:\n{task_list}", temperature=0.5)
301
+ except Exception as e:
302
+ logger.error(f"Digest generation failed: {e}")
303
+ return f"{len(tasks)} new tasks extracted and prioritized."
task_manager_agent/main_agent.py ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Task Manager Agent
3
+ ==================
4
+ Turns emails + meeting notes into tasks automatically.
5
+ Prioritizes by deadline and impact.
6
+ Syncs with Notion or Todoist.
7
+ Triggers: new email arrival + daily 9 AM sync.
8
+ """
9
+
10
+ import os
11
+ import time
12
+ import logging
13
+ import schedule
14
+ import threading
15
+ from datetime import datetime
16
+ import os as _os
17
+ from dotenv import load_dotenv
18
+
19
+ from data_fetcher import (
20
+ fetch_recent_emails,
21
+ fetch_calendar_meetings,
22
+ fetch_existing_tasks,
23
+ fetch_notion_tasks,
24
+ )
25
+ from llm import extract_tasks_from_emails, extract_tasks_from_meeting_notes, prioritize_tasks
26
+ from task_store import TaskStore
27
+ import sys, os
28
+ sys.path.insert(0, os.path.dirname(__file__))
29
+ from delivery import (
30
+ send_task_digest_email,
31
+ send_task_digest_whatsapp,
32
+ sync_to_notion,
33
+ sync_to_todoist,
34
+ )
35
+
36
+ load_dotenv(_os.path.join(_os.path.dirname(_os.path.abspath(__file__)), "..", ".env"))
37
+ logging.basicConfig(
38
+ level=logging.INFO,
39
+ format="%(asctime)s [%(levelname)s] %(name)s β€” %(message)s",
40
+ handlers=[
41
+ logging.FileHandler("task_manager.log"),
42
+ logging.StreamHandler(),
43
+ ],
44
+ )
45
+ logger = logging.getLogger("TaskManagerAgent")
46
+
47
+ # ── config ────────────────────────────────────────────────────────────────────
48
+ EMAIL_POLL_INTERVAL_MINUTES = int(os.getenv("EMAIL_POLL_INTERVAL_MINUTES", "15"))
49
+ SYNC_BACKEND = os.getenv("TASK_SYNC_BACKEND", "notion") # "notion" | "todoist" | "both"
50
+ DELIVER_EMAIL = os.getenv("DELIVER_TASK_EMAIL", "true").lower() == "true"
51
+ DELIVER_WHATSAPP = os.getenv("DELIVER_TASK_WHATSAPP", "false").lower() == "true"
52
+ MAX_EMAILS_TO_SCAN = int(os.getenv("MAX_EMAILS_TO_SCAN", "20"))
53
+ USER_EMAIL = os.getenv("USER_EMAIL", "")
54
+
55
+
56
+ # ── core pipeline ─────────────────────────────────────────────────────────────
57
+
58
+ def run_task_extraction_pipeline(trigger: str = "scheduled") -> dict:
59
+ """Full pipeline: fetch β†’ extract β†’ prioritize β†’ sync β†’ deliver."""
60
+ logger.info(f"πŸš€ Task Manager pipeline started (trigger={trigger})")
61
+ start = datetime.now()
62
+ results = {"new_tasks": [], "updated_tasks": [], "errors": []}
63
+
64
+ try:
65
+ # 1. Fetch inputs
66
+ logger.info("πŸ“₯ Fetching recent emails …")
67
+ emails = fetch_recent_emails(max_results=MAX_EMAILS_TO_SCAN)
68
+ logger.info(f" Found {len(emails)} emails to analyse")
69
+
70
+ logger.info("πŸ“… Fetching today's meetings for context …")
71
+ meetings = fetch_calendar_meetings(days_back=1, days_ahead=0)
72
+ logger.info(f" Found {len(meetings)} recent meetings")
73
+
74
+ logger.info("πŸ“‹ Fetching existing tasks to avoid duplicates …")
75
+ existing_tasks = fetch_existing_tasks()
76
+
77
+ # 2. Extract tasks from emails
78
+ if emails:
79
+ logger.info("πŸ€– Extracting tasks from emails via Groq …")
80
+ email_tasks = extract_tasks_from_emails(emails, existing_tasks)
81
+ logger.info(f" Extracted {len(email_tasks)} tasks from emails")
82
+ results["new_tasks"].extend(email_tasks)
83
+
84
+ # 3. Extract tasks from meeting notes / descriptions
85
+ if meetings:
86
+ logger.info("πŸ€– Extracting tasks from meeting context …")
87
+ meeting_tasks = extract_tasks_from_meeting_notes(meetings, existing_tasks)
88
+ logger.info(f" Extracted {len(meeting_tasks)} tasks from meetings")
89
+ results["new_tasks"].extend(meeting_tasks)
90
+
91
+ if not results["new_tasks"]:
92
+ logger.info("βœ… No new tasks found β€” nothing to sync")
93
+ return results
94
+
95
+ # 4. Prioritize all new tasks together
96
+ logger.info("🎯 Prioritizing tasks …")
97
+ all_existing = existing_tasks + results["new_tasks"]
98
+ prioritized = prioritize_tasks(results["new_tasks"], all_existing)
99
+ results["new_tasks"] = prioritized
100
+
101
+ # 5. Persist locally
102
+ store = TaskStore()
103
+ saved_count = store.save_tasks(prioritized)
104
+ logger.info(f"πŸ’Ύ Saved {saved_count} tasks to local store")
105
+
106
+ # 6. Sync to external backend(s)
107
+ if SYNC_BACKEND in ("notion", "both"):
108
+ logger.info("πŸ”„ Syncing to Notion …")
109
+ synced = sync_to_notion(prioritized)
110
+ logger.info(f" Synced {synced} tasks to Notion")
111
+
112
+ if SYNC_BACKEND in ("todoist", "both"):
113
+ logger.info("πŸ”„ Syncing to Todoist …")
114
+ synced = sync_to_todoist(prioritized)
115
+ logger.info(f" Synced {synced} tasks to Todoist")
116
+
117
+ # 7. Deliver digest
118
+ if DELIVER_EMAIL and USER_EMAIL:
119
+ logger.info("πŸ“§ Sending task digest email …")
120
+ send_task_digest_email(prioritized, trigger=trigger, recipient=USER_EMAIL)
121
+
122
+ if DELIVER_WHATSAPP:
123
+ logger.info("πŸ“± Sending WhatsApp digest …")
124
+ send_task_digest_whatsapp(prioritized, trigger=trigger)
125
+
126
+ elapsed = (datetime.now() - start).seconds
127
+ logger.info(f"βœ… Pipeline complete in {elapsed}s β€” {len(prioritized)} tasks processed")
128
+
129
+ except Exception as e:
130
+ logger.error(f"❌ Pipeline error: {e}", exc_info=True)
131
+ results["errors"].append(str(e))
132
+
133
+ return results
134
+
135
+
136
+ # ── email-trigger watcher ─────────────────────────────────────────────────────
137
+
138
+ class EmailWatcher(threading.Thread):
139
+ """Polls Gmail every N minutes; fires pipeline when new actionable emails arrive."""
140
+
141
+ def __init__(self):
142
+ super().__init__(daemon=True)
143
+ self._stop_event = threading.Event()
144
+ self._last_email_id: str | None = None
145
+
146
+ def run(self):
147
+ logger.info(f"πŸ‘οΈ EmailWatcher started (polling every {EMAIL_POLL_INTERVAL_MINUTES} min)")
148
+ while not self._stop_event.is_set():
149
+ try:
150
+ self._check_for_new_emails()
151
+ except Exception as e:
152
+ logger.warning(f"EmailWatcher error: {e}")
153
+ self._stop_event.wait(EMAIL_POLL_INTERVAL_MINUTES * 60)
154
+
155
+ def _check_for_new_emails(self):
156
+ emails = fetch_recent_emails(max_results=5, since_id=self._last_email_id)
157
+ if emails:
158
+ newest_id = emails[0].get("id")
159
+ if newest_id != self._last_email_id:
160
+ logger.info(f"πŸ“¬ {len(emails)} new email(s) detected β€” triggering extraction")
161
+ self._last_email_id = newest_id
162
+ run_task_extraction_pipeline(trigger="email_trigger")
163
+
164
+ def stop(self):
165
+ self._stop_event.set()
166
+
167
+
168
+ # ── scheduler ─────────────────────────────────────────────────────────────────
169
+
170
+ def schedule_daily_sync():
171
+ schedule.every().day.at("09:00").do(
172
+ lambda: run_task_extraction_pipeline(trigger="daily_9am")
173
+ )
174
+ logger.info("⏰ Daily sync scheduled at 09:00")
175
+
176
+
177
+ def run_scheduler():
178
+ logger.info("πŸ—“οΈ Scheduler running …")
179
+ while True:
180
+ schedule.run_pending()
181
+ time.sleep(30)
182
+
183
+
184
+ # ── entry point ───────────────────────────────────────────────────────────────
185
+
186
+ def main():
187
+ logger.info("=" * 60)
188
+ logger.info(" TASK MANAGER AGENT β€” Personal AI OS")
189
+ logger.info("=" * 60)
190
+
191
+ # Immediate run on startup
192
+ run_task_extraction_pipeline(trigger="startup")
193
+
194
+ # Continuous email watcher
195
+ watcher = EmailWatcher()
196
+ watcher.start()
197
+
198
+ # Daily 9 AM scheduled sync
199
+ schedule_daily_sync()
200
+ run_scheduler() # blocks
201
+
202
+
203
+ if __name__ == "__main__":
204
+ main()
task_manager_agent/requirements.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ groq>=0.8.0
2
+ google-auth>=2.28.0
3
+ google-auth-oauthlib>=1.2.0
4
+ google-auth-httplib2>=0.2.0
5
+ google-api-python-client>=2.120.0
6
+ schedule>=1.2.1
7
+ python-dotenv>=1.0.1
8
+ requests>=2.31.0
9
+ twilio>=8.13.0
task_manager_agent/task_store.py ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ task_store.py β€” Task Manager Agent
3
+ =====================================
4
+ Local JSON-based task store with deduplication, status tracking,
5
+ and a simple query interface.
6
+ """
7
+
8
+ import json
9
+ import os
10
+ import uuid
11
+ import logging
12
+ from datetime import datetime, timezone
13
+ from pathlib import Path
14
+ from typing import Optional
15
+
16
+ logger = logging.getLogger("TaskManagerAgent.TaskStore")
17
+
18
+ TASKS_FILE = os.getenv("LOCAL_TASKS_FILE", "tasks.json")
19
+ MAX_LOCAL_TASKS = 500 # rotate after this many
20
+
21
+
22
+ class TaskStore:
23
+ def __init__(self, filepath: str = TASKS_FILE):
24
+ self.path = Path(filepath)
25
+ self._tasks: list[dict] = self._load()
26
+
27
+ # ── I/O ──────────────────────────────────────────────────────────────────
28
+
29
+ def _load(self) -> list[dict]:
30
+ if self.path.exists():
31
+ try:
32
+ with open(self.path) as f:
33
+ data = json.load(f)
34
+ return data if isinstance(data, list) else []
35
+ except Exception as e:
36
+ logger.warning(f"Could not load tasks file: {e}")
37
+ return []
38
+
39
+ def _persist(self):
40
+ try:
41
+ with open(self.path, "w") as f:
42
+ json.dump(self._tasks[-MAX_LOCAL_TASKS:], f, indent=2, default=str)
43
+ except Exception as e:
44
+ logger.error(f"Could not persist tasks: {e}")
45
+
46
+ # ── public API ────────────────────────────────────────────────────────────
47
+
48
+ def save_tasks(self, tasks: list[dict]) -> int:
49
+ """Upsert tasks by title similarity. Returns count of new tasks added."""
50
+ existing_titles = {t.get("title", "").lower().strip() for t in self._tasks}
51
+ added = 0
52
+ for task in tasks:
53
+ title_key = task.get("title", "").lower().strip()
54
+ if title_key in existing_titles:
55
+ logger.debug(f"Skipping duplicate task: {task.get('title')}")
56
+ continue
57
+ task["id"] = task.get("id") or str(uuid.uuid4())
58
+ task["created_at"] = datetime.now(timezone.utc).isoformat()
59
+ task["status"] = task.get("status", "todo")
60
+ self._tasks.append(task)
61
+ existing_titles.add(title_key)
62
+ added += 1
63
+
64
+ if added:
65
+ self._persist()
66
+ return added
67
+
68
+ def get_tasks(
69
+ self,
70
+ status: Optional[str] = None,
71
+ category: Optional[str] = None,
72
+ priority_min: int = 0,
73
+ ) -> list[dict]:
74
+ result = self._tasks
75
+ if status:
76
+ result = [t for t in result if t.get("status") == status]
77
+ if category:
78
+ result = [t for t in result if t.get("category") == category]
79
+ if priority_min:
80
+ result = [t for t in result if (t.get("priority_score") or 0) >= priority_min]
81
+ return sorted(result, key=lambda x: x.get("priority_score", 0), reverse=True)
82
+
83
+ def mark_done(self, task_id: str) -> bool:
84
+ for t in self._tasks:
85
+ if t.get("id") == task_id:
86
+ t["status"] = "done"
87
+ t["completed_at"] = datetime.now(timezone.utc).isoformat()
88
+ self._persist()
89
+ return True
90
+ return False
91
+
92
+ def get_open_count(self) -> int:
93
+ return sum(1 for t in self._tasks if t.get("status") == "todo")
94
+
95
+ def all_as_context(self) -> list[dict]:
96
+ """Lightweight list for LLM context / dedup checks."""
97
+ return [{"title": t.get("title"), "status": t.get("status")} for t in self._tasks]
task_manager_agent/token.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"token": "ya29.a0AQvPyIM5ItOxxVnGPnWfIzXS7aQpCYvu09vLmS42BhjeV2p4Z0V2Q3bYPiub9yY0UkU09AvuIX75cyfS7nxn2oFEW49QLvPcHZU5PzDiNrvitPB6C4LS75CxQUjR63OLP8hpELr07zgEM594L7m0D0BV3GBdayxJqiIF_X4rt_fg0-raP4suW4sSW_FoX-8IYCCkdBEaCgYKAcgSARUSFQHGX2MiAHBi0PWVvembfcbAGKUFSg0206", "refresh_token": "1//0gpNfTgwl7MUnCgYIARAAGBASNwF-L9IrcTUaFCTszG_Z0BLQZw-RC4pZaXl0B1m8AjuP0dnEaOdZDUIIK-2EOh7WndaHl-X4fiQ", "token_uri": "https://oauth2.googleapis.com/token", "client_id": "426626406279-ib9ps1kchej0eqt4ot2l7mbng7gaac5m.apps.googleusercontent.com", "client_secret": "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", "scopes": ["https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/calendar.readonly"], "universe_domain": "googleapis.com", "account": "", "expiry": "2026-05-17T20:40:05Z"}