Add max_tokens parameter to NexAGI endpoint

Add privacy column to Document model + migration 005

Remove PII from research mode and response handlers

Tune Presidio: strict mode masks names+orgs for NDA, light mode skips them for research

Add privacy flag to upload — PII runs only when privacy=True (NDA mode)

Fix tuple unpacking error in gather result for PDF parsing

Skip MarkItDown for PDFs — run only as fallback if pypdf returns empty

Swap pdfplumber → pypdf for upload parsing (3x faster text extraction)

Fix upload dedup crash: scalar_one_or_none → scalars().first() for sha256 lookup

Remove forced section headers from SYSTEM_WHITE_BOX — /query returns raw LLM output

Fix NexAGIResponse: reasoning_details is list[dict], not dict

Fix NexAGI: reasoning_details is already a list of dicts

Fix NexAGI: reasoning_details list model_dump, add reasoning_details to message schema

Add /NexAGI endpoint — OpenRouter reasoning via nex-agi/nex-n2-pro:free

Parallelize multi-doc retrieval with asyncio.gather

Restore JSON parsing in analyze_document — populate all DocumentAnalysis fields

Reduce default concurrent jobs to 5

Append verification warning to answer so frontend always shows it

Handle null content in LLM response (NoneType len error)

Auto-fallback to mercury when minimax fails (chat + stream)

Skip PERSON/LOCATION/ORG in PII masking — only mask contact & financial IDs

Safe minimax defaults: no thinking, temp=0.3, max_tokens=4096

Profile-aware API key: mercury uses INCEPTION_API_KEY, minimax uses LLM_API_KEY

Minimax uses LLM_BASE_URL env var; mercury always uses inceptionlabs.ai

Remove LLM_BASE_URL env override — use profile's own base_url

Keep only mercury + minimax profiles; remove custom profile logic entirely

Drop custom profile as default — always default to mercury; minimax remains selectable

Speed up analyze: use black_box mode (no thinking, 2K tokens) instead of white_box

Remove _strip_headers post-processing — let LLM output pass through as-is

Strip headers server-side + stronger prompt to enforce plain-text output

Force plain-text output: remove all headers/structure from LLM prompt

Simplify document_agent: drop JSON parsing, return raw LLM output; fix llm.py syntax errors

fix: strict query-only output in analyze — no extra sections

feat: query-driven analyze + context window bump

fix: make analyze JSON structure query-driven instead of rigid template

fix: normalize all LLM JSON fields + wire cross_doc/per_doc

fix: normalize list-typed limitations/methodology from LLM JSON

fix: thinking.type enabled for analyze mode + status event

fix: increase httpx timeout 120s -> 300s/600s for thinking models

fix: thinking.type enabled instead of adaptive for NVIDIA API

feat: minimax temp=0.1 + thinking adaptive + json output for analyze

fix: handle LLM_BASE_URL with or without /chat/completions suffix

feat: provider-based LLM profiles + per-request model switching

fix: reduce max_citations_total=10, retrieve_k=15, rerank_top_k=10 + filter chunks <50 chars

chore: white-box temperature 0.75 -> 0.3 (stream + analyze)

feat: remove per-doc citation cap, bump rerank_top_k=20, max_citations_total=50, expose all retrieved citations

fix: retrieve_k=20, strengthen system prompts to prevent citation hallucination, add _sanitize_citations post-processing

fix: black-box concise -> medium reasoning, white-box researcher + analyze -> high

fix: use high reasoning for verify, medium for gaps + full citation text for gaps

fix: remove stale page_texts_pdf references from PII block