Spaces:

MSGEncrypted
/

lesson-agent-dev

Running

MSG

Feat/research tab agent skills (#5)

e7fd66f 30 days ago

317 Bytes

	# HTML cleanup

	ResearchMind uses trafilatura to strip boilerplate and keep main article text.

	- `include_tables=true` for data-heavy pages
	- `include_comments=false`
	- Fallback: first 50k chars of raw HTML if extraction returns empty

	Raw snapshot saved under `RESEARCHMIND_DATA_DIR/raw/{doc_id}/snapshot.txt`.