knowledge-net / backend /scraper.py

Commit History

fix: scrape error handling
f7da48c

Soham Waghmare commited on

fix: duckduckgo search index error; improved prompts
56e3a38

Soham Waghmare commited on

fix: error handling
963ab6b

Soham Waghmare commited on

feat: README, cleanup
514da67

Soham Waghmare commited on

feat: frontend - QOL; backend - QOL
79ae05b

Soham Waghmare commited on

feat: better error and data handling
62283c0

Soham Waghmare commited on

feat: migration from google-generativeai to google-genai
0a3d9b7

Soham Waghmare commited on

feat: improves logging
4e3ab6e

Soham Waghmare commited on

feat: refactor, abstract, simplify
63a0765

Soham Waghmare commited on

feat: format
02298d2

Soham Waghmare commited on

fix: scraper
b6e302a

sohamw03 commited on

fix: types, off-by-one graph sraping logic, pull depth, breadth, num_sites
88139f0

Soham Waghmare commited on

feat: concurrent ws conns, parallel scraping, better image extraction
9635653

Soham Waghmare commited on

format
9155a62

Soham Waghmare commited on

feat: improve scraping and context handling
54a7d14

Soham Waghmare commited on

feat: image extraction logic
ceae8b0

Soham Waghmare commited on

feat: integrate crawl4ai minimal scraping
15b6036

Soham Waghmare commited on

feat: adds a logistic update to backend and init frontend
afce8a8

Soham Waghmare commited on

refactor: Improve error handling and update web scraping to use DuckDuckGo
81dc032

Soham Waghmare commited on

init: Add initial backend structure with Flask, WebSocket support, and web scraping functionality
2d96b3b

Soham Waghmare commited on