Spaces:
Sleeping
Sleeping
| title: Samarth | |
| emoji: 🧑💻 | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: streamlit | |
| pinned: false | |
| short_description: 'Ask questions about Agriculture — Samarth gives answer from ' | |
| sdk_version: 1.51.0 | |
| # 📊 Samarth: Data-Aware Question Answering System | |
| An AI-powered **Question Answering System** built using **LangChain** and **Streamlit**, designed to intelligently answer user queries based on multiple government data sources. | |
| --- | |
| ## 🧠 Project Overview | |
| This system enables natural language querying over diverse datasets. | |
| Each dataset is first analyzed to automatically generate structured **metadata** (including summary, columns, and use cases) using an LLM. | |
| These metadata representations are embedded into a **vector database** for semantic similarity search. | |
| When a user asks a question: | |
| 1. The system finds the **most relevant dataset** using semantic search. | |
| 2. It uses an **LLM to generate an appropriate SQL query** for that dataset. | |
| 3. The query is executed on the dataset, and the **result is interpreted into a human-readable answer** by another LLM. | |
| 4. If the dataset lacks relevant information, the system responds gracefully that no answer is available. | |
| --- | |
| ## ⚙️ How to Use | |
| 1. **Run the Streamlit App** | |
| > streamlit run app.py | |
| 2. **Upload or choose your dataset(s)** | |
| - The app supports multiple tabular datasets (CSV). | |
| 3. **Ask a natural language question** | |
| Example: | |
| > “What was the average annual rainfall in Telangana in 2000?” | |
| 4. **View the result** | |
| * The system automatically identifies the most relevant dataset. | |
| * It generates, executes, and interprets a SQL query. | |
| * You receive a concise, natural answer with a verified data source link. | |
| ## 🗂️ Adding New Datasets | |
| To include new datasets in the system, follow these simple steps: | |
| 1. **Download and place your dataset file** | |
| - Save the new dataset (CSV format) inside the `/datasets` folder. | |
| 2. **Update the metadata generation script** | |
| - Open `generate_metadata.py`. | |
| - Add the dataset details in the respective lists: | |
| - `dataset_links` → the dataset’s source link | |
| - `dataset_names` → a short descriptive name for the dataset | |
| - `datasets_list` → the filename of the dataset (as saved in the `/datasets` folder) | |
| 3. **Generate metadata** | |
| - Run the following command to generate structured metadata for all datasets: | |
| > python generate_metadata.py | |
| 4. **Restart the Streamlit app** | |
| - Once metadata is generated, rerun the app: | |
| > streamlit run app.py | |
| --- | |
| ## 🧩 Tech Stack | |
| * **LangChain** – for prompt orchestration and LLM integration | |
| * **Sentence Transformers + FAISS** – for vector similarity search | |
| * **Streamlit** – for interactive web UI | |
| * **Google Gemini / Generative AI** – for SQL and natural-language generation | |
| --- | |
| ## 📁 Project Flow Summary | |
| User Query | |
| ↓ | |
| Semantic Search (Vector DB) | |
| ↓ | |
| SQL Query Generation (LLM) | |
| ↓ | |
| SQL Execution on Dataset | |
| ↓ | |
| Natural Language Answer (LLM) | |
| ↓ | |
| Final Answer + Source Link | |
| --- |