berohan commited on
Commit
0f53057
Β·
verified Β·
1 Parent(s): 98d5a9a

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -234
README.md DELETED
@@ -1,234 +0,0 @@
1
- # Studyson - RAG Document QA & Summarization API
2
-
3
- A full-stack Retrieval-Augmented Generation (RAG) system for intelligent document question-answering and summarization. Built with FastAPI, LlamaIndex, and Groq AI.
4
-
5
- ## Features
6
-
7
- - **πŸ“„ PDF Document Processing**: Upload and index PDF documents with intelligent text extraction
8
- - **🌐 Web Content Scraping**: Scrape and index content from URLs
9
- - **πŸ’¬ Interactive Q&A Chat**: Ask questions about your documents with streaming responses
10
- - **πŸ“ Smart Summarization**: Generate concise summaries of indexed documents
11
- - **πŸ” Source Citations**: Get verifiable citations with exact source snippets
12
- - **⚑ Real-time Streaming**: Token-by-token streaming for responsive user experience
13
- - **🎨 Modern UI**: Clean, responsive web interface with tabbed navigation
14
- - **🐳 Docker Support**: Easy deployment with Docker and Docker Compose
15
-
16
- ## Tech Stack
17
-
18
- ### Backend
19
- - **FastAPI**: Modern Python web framework
20
- - **LlamaIndex**: RAG orchestration and document indexing
21
- - **Groq**: Lightning-fast LLM inference (Llama 3.1)
22
- - **FastEmbed**: Lightweight embeddings (BGE-small)
23
- - **PyMuPDF**: Advanced PDF text extraction
24
- - **BeautifulSoup**: HTML parsing and web scraping
25
- - **Pydantic**: Data validation and settings management
26
-
27
- ### Frontend
28
- - **HTML5/CSS3/JavaScript**: Vanilla web technologies
29
- - **Server-Sent Events (SSE)**: Real-time streaming responses
30
-
31
- ## Architecture
32
-
33
- ### Ingestion Pipeline
34
- 1. User uploads PDF or provides URL
35
- 2. Content extraction (PyMuPDF for PDFs, BeautifulSoup for web)
36
- 3. Text chunking and embedding via LlamaIndex + FastEmbed
37
- 4. In-memory vector index creation
38
-
39
- ### Query Pipeline
40
- 1. Question embedding generation
41
- 2. Semantic similarity search for relevant chunks
42
- 3. Context + question sent to Groq LLM
43
- 4. Streaming response with source citations
44
-
45
- ## Installation
46
-
47
- ### Prerequisites
48
- - Python 3.10 or higher
49
- - Groq API key ([Get it free here](https://console.groq.com))
50
-
51
- ### Local Setup
52
-
53
- 1. **Clone the repository**
54
- ```bash
55
- git clone <repository-url>
56
- cd studyrag
57
- ```
58
-
59
- 2. **Create virtual environment**
60
- ```bash
61
- python -m venv venv
62
- source venv/bin/activate # On Windows: venv\Scripts\activate
63
- ```
64
-
65
- 3. **Install dependencies**
66
- ```bash
67
- pip install -r requirements.txt
68
- ```
69
-
70
- 4. **Set up environment variables**
71
- ```bash
72
- cp .env.example .env
73
- ```
74
-
75
- Edit `.env` and add your Groq API key:
76
- ```
77
- GROQ_API_KEY=your_groq_api_key_here
78
- PORT=7860
79
- HOST=0.0.0.0
80
- ```
81
-
82
- 5. **Run the application**
83
- ```bash
84
- uvicorn app.main:app --reload --port 7860
85
- ```
86
-
87
- 6. **Access the application**
88
-
89
- Open your browser and navigate to: `http://localhost:7860`
90
-
91
- ### Docker Setup
92
-
93
- 1. **Set environment variables**
94
- ```bash
95
- cp .env.example .env
96
- # Edit .env with your Groq API key
97
- ```
98
-
99
- 2. **Build and run with Docker Compose**
100
- ```bash
101
- docker-compose up --build
102
- ```
103
-
104
- ## API Endpoints
105
-
106
- | Method | Endpoint | Description |
107
- |--------|----------|-------------|
108
- | GET | `/` | Serves the web UI |
109
- | POST | `/upload` | Upload PDF document |
110
- | POST | `/scrape` | Scrape URL content |
111
- | POST | `/stream_query` | Stream Q&A response |
112
- | POST | `/query` | Get Q&A response |
113
- | POST | `/summarize` | Generate summary |
114
- | POST | `/reset` | Clear all documents |
115
- | GET | `/status` | Get system status |
116
-
117
- ## Project Structure
118
-
119
- ```
120
- studyrag/
121
- β”œβ”€β”€ app/
122
- β”‚ β”œβ”€β”€ __init__.py
123
- β”‚ β”œβ”€β”€ main.py # FastAPI application
124
- β”‚ β”œβ”€β”€ config.py # Configuration settings
125
- β”‚ β”œβ”€β”€ models/
126
- β”‚ β”‚ └── schemas.py # Pydantic models
127
- β”‚ β”œβ”€β”€ services/
128
- β”‚ β”‚ └── rag_service.py # RAG logic
129
- β”‚ └── utils/
130
- β”‚ └── document_processor.py
131
- β”œβ”€β”€ static/
132
- β”‚ β”œβ”€β”€ css/style.css
133
- β”‚ β”œβ”€β”€ js/app.js
134
- β”‚ └── index.html
135
- β”œβ”€β”€ .env.example
136
- β”œβ”€β”€ .gitignore
137
- β”œβ”€β”€ Dockerfile
138
- β”œβ”€β”€ docker-compose.yml
139
- β”œβ”€β”€ Procfile
140
- β”œβ”€β”€ requirements.txt
141
- └── README.md
142
- ```
143
-
144
- ## Configuration
145
-
146
- ### Environment Variables
147
-
148
- - `GROQ_API_KEY`: Your Groq API key (required, free tier available)
149
- - `HOST`: Server host (default: 0.0.0.0)
150
- - `PORT`: Server port (default: 7860)
151
-
152
- ### Application Settings
153
-
154
- Edit `app/config.py` to modify:
155
- - `upload_dir`: Upload directory path
156
- - `max_file_size`: Maximum file size (default: 10MB)
157
-
158
- ## Deployment
159
-
160
- ### Deploy to Hugging Face Spaces (Recommended - Free)
161
-
162
- 1. Push code to GitHub
163
- 2. Go to [huggingface.co](https://huggingface.co) and create an account
164
- 3. Click your profile β†’ **New Space**
165
- 4. Configure:
166
- - **Space name**: `studyson`
167
- - **SDK**: Select **Docker**
168
- - **Hardware**: CPU basic (free)
169
- 5. Under **Files** β†’ Link to GitHub repo (or upload files)
170
- 6. Add secret: `GROQ_API_KEY` in Space Settings β†’ Variables
171
- 7. The Space will auto-build and deploy!
172
-
173
- **Your app will be live at:** `https://huggingface.co/spaces/YOUR_USERNAME/studyson`
174
-
175
- ## Features in Detail
176
-
177
- ### RAG Pipeline
178
- - **Chunking**: Intelligent text splitting for optimal context windows
179
- - **Embeddings**: FastEmbed BGE-small for semantic understanding (lightweight)
180
- - **Retrieval**: Top-k similarity search with configurable parameters
181
- - **Generation**: Groq Llama 3.1 for fast, accurate responses
182
-
183
- ### Streaming
184
- - Server-Sent Events (SSE) for real-time token delivery
185
- - Progressive rendering in the UI
186
- - Graceful error handling
187
-
188
- ### Source Attribution
189
- - Exact text snippets from source documents
190
- - Similarity scores for transparency
191
- - Multiple source support per answer
192
-
193
- ## Limitations
194
-
195
- - In-memory vector storage (resets on restart)
196
- - PDF-only document support (extensible to other formats)
197
- - Single-user session management
198
- - No authentication/authorization
199
-
200
- ## Troubleshooting
201
-
202
- ### Common Issues
203
-
204
- **Import errors:**
205
- ```bash
206
- pip install --upgrade -r requirements.txt
207
- ```
208
-
209
- **API key errors:**
210
- - Verify your `.env` file has the correct `GROQ_API_KEY`
211
- - Check API key validity at [console.groq.com](https://console.groq.com)
212
-
213
- **Port already in use:**
214
- ```bash
215
- uvicorn app.main:app --port 8000
216
- ```
217
-
218
- **File upload fails:**
219
- - Check file size is under 10MB
220
-
221
- ## License
222
-
223
- MIT License - feel free to use this project for learning and development.
224
-
225
- ## Acknowledgments
226
-
227
- - [LlamaIndex](https://www.llamaindex.ai/) for RAG orchestration
228
- - [Groq](https://groq.com/) for lightning-fast LLM inference
229
- - [FastEmbed](https://github.com/qdrant/fastembed) for lightweight embeddings
230
- - [FastAPI](https://fastapi.tiangolo.com/) for the web framework
231
-
232
- ---
233
-
234
- Built with ❀️ using RAG technology