RAG-document-assistant / docs /api_reference.md
vn6295337's picture
Initial commit: RAG Document Assistant with Zero-Storage Privacy
f866820

API Reference

Base URL: https://vn6295337-rag-document-assistant.hf.space/api


Zero-Storage Endpoints

POST /embed-chunks

Generate embeddings for text chunks and store in Pinecone. Text is discarded immediately after embedding.

Request Body:

{
  "chunks": [
    {
      "text": "The actual text content...",
      "metadata": {
        "filename": "document.pdf",
        "filePath": "/Documents/document.pdf",
        "fileId": "dropbox_file_id",
        "chunkIndex": 0,
        "startChar": 0,
        "endChar": 1000
      }
    }
  ]
}

Response:

{
  "status": "success",
  "vectors_upserted": 5,
  "error": null
}

Privacy Note: Text is used only for embedding generation, then immediately deleted. Only embeddings and metadata (file paths, positions) are stored.


POST /query-secure

Execute a zero-storage query. Re-fetches text from user's Dropbox at query time.

Request Body:

{
  "query": "What are the payment terms?",
  "access_token": "dropbox_access_token",
  "top_k": 3
}

Response:

{
  "answer": "According to the contract, payment is due within 30 days...",
  "citations": [
    {
      "id": "file_id::0",
      "score": 0.85,
      "snippet": "Payment terms: Net 30 days from invoice date..."
    }
  ],
  "error": null
}

Flow:

  1. Generate query embedding
  2. Search Pinecone for similar chunks
  3. Re-fetch files from user's Dropbox using provided token
  4. Extract chunk text using stored positions
  5. Send to LLM for answer generation
  6. Return answer (text never stored)

DELETE /clear-index

Clear all vectors from Pinecone index.

Response:

{
  "status": "success",
  "message": "Index cleared"
}

Dropbox Integration Endpoints

POST /dropbox/token

Exchange Dropbox authorization code for access token. Client secret is kept server-side.

Request Body:

{
  "code": "authorization_code_from_oauth",
  "redirect_uri": "https://your-app.com/callback"
}

Response:

{
  "access_token": "sl.xxxxx",
  "token_type": "bearer",
  "expires_in": 14400
}

POST /dropbox/folder

List contents of a Dropbox folder (proxy to avoid CORS).

Request Body:

{
  "path": "",
  "access_token": "dropbox_access_token"
}

Response:

{
  "entries": [
    {
      ".tag": "file",
      "name": "document.pdf",
      "id": "id:xxxxx",
      "path_lower": "/documents/document.pdf",
      "size": 12345
    }
  ],
  "has_more": false
}

POST /dropbox/file

Download file content from Dropbox. Supports text files and PDFs (with text extraction).

Request Body:

{
  "path": "/documents/document.pdf",
  "access_token": "dropbox_access_token"
}

Response:

{
  "content": "Extracted text content from the file..."
}

Supported File Types:

  • .txt - Plain text
  • .md - Markdown
  • .pdf - PDF (text extraction via PyPDF2)

Utility Endpoints

GET /health

Health check endpoint.

Response:

{
  "status": "ok"
}

GET /status

Get current index status.

Query Parameters:

  • chunks_path (optional): Path to chunks file (default: data/chunks.jsonl)

Response:

{
  "exists": true,
  "chunks": 44,
  "documents": 5,
  "path": "data/chunks.jsonl",
  "error": null
}

Legacy Endpoints

POST /query

Standard RAG query (uses local chunks file, not zero-storage).

Request Body:

{
  "query": "What is GDPR?",
  "top_k": 3,
  "use_hybrid": false,
  "use_reranking": false
}

POST /ingest

Ingest documents from a directory (server-side processing).

Request Body:

{
  "docs_dir": "sample_docs",
  "output_path": "data/chunks.jsonl",
  "provider": "sentence-transformers"
}

Error Responses

All endpoints return errors in this format:

{
  "status": "error",
  "error": "Description of the error"
}

Common error codes:

  • Missing required parameters
  • Invalid access token
  • Dropbox API errors
  • Pinecone connection errors
  • LLM provider failures

Rate Limits

  • Embedding: No explicit limit (Pinecone free tier: 100K operations/month)
  • Queries: Subject to LLM provider limits:
    • Gemini: 15 RPM
    • Groq: 30 RPM
    • OpenRouter: Varies by model

Environment Variables

Required on backend:

  • PINECONE_API_KEY - Pinecone vector database
  • DROPBOX_APP_KEY - Dropbox OAuth app key
  • DROPBOX_APP_SECRET - Dropbox OAuth app secret
  • GEMINI_API_KEY - Google Gemini API (primary LLM)
  • GROQ_API_KEY - Groq API (fallback LLM)