README.md

metadata

title: Smart Search LLM
emoji: 🦀
colorFrom: blue
colorTo: gray
sdk: streamlit
sdk_version: 1.39.0
app_file: app.py
pinned: false
license: mit
short_description: LLM based search tool for free courses on Analytics Vidhya

📚 Analytics Vidhya Free Courses Search App

This project is a Streamlit-based web application designed to help users find the most relevant free courses on Analytics Vidhya’s platform. Using Hugging Face’s all-MiniLM-L6-v2 for embeddings and FAISS for fast similarity search, the tool provides an efficient and intuitive search experience.

🌟 Key Features

• Embeddings for Smart Search: Course titles are converted into embeddings using Hugging Face’s all-MiniLM-L6-v2 model, enabling accurate matching with user queries.
• Real-time Course Data: The app scrapes and updates course data directly from Analytics Vidhya's free courses page.
• Ranked Recommendations: Each course is ranked based on its relevance to the user query, displaying only the top results.
• User-Friendly Interface: Built with Streamlit, the app offers a clean and intuitive interface for easy navigation.
• Fast Similarity Search: Uses FAISS to quickly retrieve the top relevant courses based on query embeddings.

🧠 How It Works

1. Web Scraping: The tool scrapes course titles, images, and URLs from Analytics Vidhya.
2. Embedding Generation: Course titles are transformed into embeddings using the all-MiniLM-L6-v2 model from Hugging Face.
3. Similarity Search: FAISS is employed to efficiently search for the closest matches to the user’s query embedding.
4. Results Display: The top 10 relevant courses are displayed with their titles, images, and relevance scores.

🚀 Technologies

• Python: Backend processing.
• Streamlit: UI framework for building the web application.
• BeautifulSoup: For web scraping to gather course data.
• Hugging Face Transformers: For generating embeddings using the all-MiniLM-L6-v2 model.
• FAISS: For fast similarity search across course embeddings.
• Pandas: For data management and manipulation.

📥 Installation and Setup

1. Clone the Repository

   git clone https://huggingface.co/spaces/metechmohit/Smart_Search_LLM
   cd\Smart_Search_LLM
   

2. Set Up Virtual Environment

   python -m venv myenv
   source myenv/bin/activate   # On Windows, use myenv\Scripts\activate
   

3. Install Dependencies

   pip install -r requirements.txt
   

4. Run the Application

   streamlit run app.py
   

🌐 Deployed Version

You can try the live version on Hugging Face Spaces:
Live Demo

---

Developed by @metechmohit.