README.md

---
title: Smart Search LLM
emoji: 🦀
colorFrom: blue
colorTo: gray
sdk: streamlit
sdk_version: 1.39.0
app_file: app.py
pinned: false
license: mit
short_description: LLM based search tool for free courses on Analytics Vidhya
---

# 📚 Analytics Vidhya Free Courses Search App

This project is a **Streamlit-based web application** designed to help users find the most relevant free courses on Analytics Vidhya’s platform. Using **Hugging Face’s `all-MiniLM-L6-v2`** for embeddings and **FAISS** for fast similarity search, the tool provides an efficient and intuitive search experience.

## 🌟 Key Features

- **Embeddings for Smart Search**: Course titles are converted into embeddings using Hugging Face’s `all-MiniLM-L6-v2` model, enabling accurate matching with user queries.
- **Real-time Course Data**: The app scrapes and updates course data directly from Analytics Vidhya's free courses page.
- **Ranked Recommendations**: Each course is ranked based on its relevance to the user query, displaying only the top results.
- **User-Friendly Interface**: Built with Streamlit, the app offers a clean and intuitive interface for easy navigation.
- **Fast Similarity Search**: Uses FAISS to quickly retrieve the top relevant courses based on query embeddings.

## 🧠 How It Works

1. **Web Scraping**: The tool scrapes course titles, images, and URLs from Analytics Vidhya.
2. **Embedding Generation**: Course titles are transformed into embeddings using the `all-MiniLM-L6-v2` model from Hugging Face.
3. **Similarity Search**: FAISS is employed to efficiently search for the closest matches to the user’s query embedding.
4. **Results Display**: The top 10 relevant courses are displayed with their titles, images, and relevance scores.

## 🚀 Technologies

- **Python**: Backend processing.
- **Streamlit**: UI framework for building the web application.
- **BeautifulSoup**: For web scraping to gather course data.
- **Hugging Face Transformers**: For generating embeddings using the `all-MiniLM-L6-v2` model.
- **FAISS**: For fast similarity search across course embeddings.
- **Pandas**: For data management and manipulation.


## 📥 Installation and Setup

1. **Clone the Repository**
   ```bash
   git clone https://huggingface.co/spaces/metechmohit/Smart_Search_LLM
   cd\Smart_Search_LLM
   ```

2. **Set Up Virtual Environment**
   ```bash
   python -m venv myenv
   source myenv/bin/activate   # On Windows, use myenv\Scripts\activate
   ```

3. **Install Dependencies**
   ```bash
   pip install -r requirements.txt
   ```

4. **Run the Application**
   ```bash
   streamlit run app.py
   ```

## 🌐 Deployed Version

You can try the live version on Hugging Face Spaces:  
[**Live Demo**](https://huggingface.co/spaces/metechmohit/Smart_Search_LLM)

---

Developed by [@metechmohit](https://github.com/metechmohit).