--- title: ProfillyBot emoji: 🤖 colorFrom: gray colorTo: indigo sdk: gradio sdk_version: 6.0.2 python_version: '3.11' app_file: gradio_app.py pinned: true license: mit short_description: RAG chatbot for professional profile Q&A thumbnail: >- https://cdn-uploads.huggingface.co/production/uploads/656f08be4893a2a26d8b17c9/1jY92qjVxCyZ0_BVHVAOF.png --- # 🤖 ProfillyBot

ProfillyBot Banner

Python 3.10 Python 3.11 Python 3.12 Docker Lint Code Quality Security Scan Codecov

**ProfillyBot** is a RAG (Retrieval Augmented Generation) chatbot that answers questions about your professional profile using your resume, project reports, and other documents. ## ✨ Features - 📄 **Multi-format Support**: Process PDF, Word, HTML, and text documents - 🔍 **Hybrid Search**: Combines BM25 (keyword) + Vector (semantic) search with Reciprocal Rank Fusion - 🧠 **Extensible Retrieval**: Pluggable strategy system - easily add new retrieval methods (PageIndex, GraphRAG, etc.) - 📌 **Main Document Support**: Guaranteed context - critical information always available, auto-format detection - 🦙 **Ollama Integration**: Run small language models locally - 🎨 **Clean UI**: Streamlit-based interface - ⚙️ **Highly Configurable**: YAML-based settings for easy customization - 🚀 **HuggingFace Spaces Ready**: Deploy with one click - ✨ **Smart Response Enhancement**: Automatically removes negative language and adds professional, recruiter-friendly tone ## 🏗️ Architecture

System Architecture Diagram

Overall system design showing the RAG pipeline and component interactions

Detailed architecture can be found in [ARCHITECTURE.md](ARCHITECTURE.md). ## 🛠️ Tech Stack - **Python 3.10+** with UV package manager - **LangChain** for RAG pipeline - **ChromaDB** for vector storage - **rank-bm25** for lexical search - **Ollama** for local LLM serving - **Streamlit** for web interface - **sentence-transformers** for embeddings - **tiktoken** for token counting - **Ruff** for linting/formatting ## 📦 Installation ### Prerequisites 1. **Python 3.10+** 2. **UV Package Manager**: Install via: ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` 3. **Ollama**: Install from [ollama.ai](https://ollama.ai) ### Setup 1. **Clone the repository**: ```bash git clone cd profillybot ``` 2. **Install dependencies with UV**: ```bash uv venv source .venv/bin/activate # On Windows: .venv\Scripts\activate uv pip install -r requirements.txt ``` 3. **Set up environment variables**: ```bash cp .env.template .env # Edit .env with your settings ``` 4. **Configure the chatbot**: - Edit `config.yaml` to set your name, title, and preferences - Adjust model settings, chunking parameters, and system prompt 5. **Add your documents**: ```bash # Place your PDFs, Word docs, HTML files in: mkdir -p data/documents # Copy your resume, project reports, LinkedIn profile, etc. ``` 6. **Create your Main Profile Document** (📌 **Recommended**): The Main Document feature ensures critical information is **always** included in responses, never missed by vector search. ```bash # Create main profile document (any format: .md, .txt, .pdf, .docx, .html) nano data/documents/main_profile.md ``` **Include essential information:** - Full name, title, contact info - Current role and key responsibilities - Core skills and expertise - Major projects with metrics - Education and certifications **Example structure:** ```markdown # Your Name **Title**: Your Professional Title **Email**: your.email@example.com ## Professional Summary Brief summary of your experience... ## Core Skills - Skill 1, Skill 2, Skill 3 ## Current Role ### Company - Role (Dates) - Achievement 1 with metrics - Achievement 2 with metrics ## Education - Degree, University, Year ``` **Why use this?** - ✅ **Guaranteed Context**: Critical info never missed by vector similarity search - ✅ **Priority Positioning**: Appears BEFORE retrieved chunks (higher LLM attention) - ✅ **Auto-Format Detection**: Supports MD, TXT, PDF, DOCX, HTML automatically - ✅ **Smart Token Management**: LLM-based summarization if content exceeds 10k tokens The feature is enabled by default in `config.yaml`. See [docs/MAIN_DOCUMENT_GUIDE.md](docs/MAIN_DOCUMENT_GUIDE.md) for advanced configuration. 7. **Pull Ollama model**: ```bash ollama pull llama3.2:3b # Or other small models: phi3:mini, gemma2:2b ``` ## 🚀 Usage ### Local Development 1. **Build the retrieval index** (default: hybrid BM25 + Vector): ```bash # Build with default strategy from config (bm25_vector) python -m src.build_vectorstore # Or specify a strategy explicitly python -m src.build_vectorstore --strategy bm25_vector # Hybrid (recommended) python -m src.build_vectorstore --strategy vector # Vector-only python -m src.build_vectorstore --strategy bm25 # BM25-only ``` 2. **Run the Streamlit app**: ```bash streamlit run app.py ``` 3. **Open browser**: Navigate to `http://localhost:8501` **Screenshot Example:**

ProfillyBot UI Example

Streamlit chatbot interface in action

### 🐳 Docker Compose Run the complete stack with Docker Compose (includes Ollama): ```bash # Copy environment template cp .env.template .env # Start all services (app + Ollama) docker compose up -d # View logs docker compose logs -f # Stop services docker compose down ``` Access the app at `http://localhost:7860` ### 📋 Makefile Commands Use the Makefile for common development tasks: ```bash make help # Show all available commands ``` | Category | Command | Description | |----------|---------|-------------| | **Setup** | `make setup` | Initial project setup (copy env, install deps) | | | `make install` | Install dependencies | | | `make install-dev` | Install with dev dependencies | | **Development** | `make run` | Run Streamlit app locally | | | `make index-build` | Build vector and BM25 indexes | | | `make index-rebuild` | Rebuild indexes from scratch | | **Quality** | `make test` | Run tests | | | `make test-cov` | Run tests with coverage | | | `make lint` | Run linter (ruff) | | | `make format` | Format code | | | `make check` | Run all checks | | **Docker** | `make docker-up` | Start all services | | | `make docker-down` | Stop all services | | | `make docker-rebuild` | Rebuild and restart | | | `make docker-logs` | Show logs (follow mode) | | | `make docker-clean` | Stop and remove volumes | | **Ollama** | `make ollama-pull` | Pull configured model | | | `make ollama-list` | List available models | | **Cleanup** | `make clean` | Remove build artifacts | | | `make clean-all` | Clean everything + indexes | ## 🌐 Deployment to Hugging Face Spaces (ZeroGPU) ProfillyBot on Spaces uses **Gradio + ZeroGPU** with `Qwen/Qwen3.5-4B` (`gradio_app.py`). Streamlit (`app.py`) remains available for local/Ollama/Groq. ### Live Space - [huggingface.co/spaces/MinhDS/ProfillyBot](https://huggingface.co/spaces/MinhDS/ProfillyBot) ### Deploy / update 1. Create a Gradio Space with **ZeroGPU** hardware (Pro account required to *create* ZeroGPU Spaces). 2. Ensure this repo has: - `data/documents/MY_CV_2_0.pdf` (or your profile docs) - Pre-built `chroma_db/` + `bm25_index/` (recommended) — or let the app build on first start 3. Push to the Space: ```bash git remote add hf https://huggingface.co/spaces/MinhDS/ProfillyBot git push hf main ``` 4. Confirm Space Settings: SDK **Gradio**, hardware **ZeroGPU**, app file `gradio_app.py`. ### Important Notes for HF Spaces - **ZeroGPU is Gradio-only** — do not use the Docker/Ollama `Dockerfile` on ZeroGPU - **SLM**: `Qwen/Qwen3.5-4B` via the `transformers` provider - **Embeddings**: `all-MiniLM-L6-v2` on CPU (does not consume GPU quota) - **Vector DB**: Pre-build indexes locally when possible for faster cold start - **Secrets**: optional `HF_TOKEN` only if you switch to a gated model ## ⚙️ Configuration ### `config.yaml` - Main Settings ```yaml profile: name: "Your Name" # ← Change this! title: "Your Title" llm: model: "llama3.2:3b" # Choose your model temperature: 0.7 # Retrieval Strategy Configuration retrieval: strategy: "bm25_vector" # Options: vector, bm25, bm25_vector final_k: 4 # Number of documents to return vector: search_type: "similarity" k: 10 # Docs to retrieve before fusion bm25: k: 10 # Docs to retrieve before fusion tokenizer: "simple" fusion: algorithm: "rrf" # Reciprocal Rank Fusion weights: vector: 0.7 # 70% weight on semantic search bm25: 0.3 # 30% weight on keyword search document_processing: chunk_size: 1000 chunk_overlap: 200 ``` ### `.env` - Environment Variables ```bash OLLAMA_BASE_URL=http://localhost:11434 CHROMA_PERSIST_DIR=./chroma_db ``` ## 📚 Project Structure ``` profillybot/ ├── app.py # Streamlit app entry point ├── pyproject.toml # UV/pip dependencies & config ├── config.yaml # RAG & LLM settings ├── .env.template # Environment variables template ├── Makefile # Development commands ├── docker-compose.yml # Docker Compose configuration ├── Dockerfile # Docker image (standalone with Ollama) ├── Dockerfile.app # Docker image (for Compose) ├── README.md ├── data/ │ └── documents/ # Your profile documents ├── src/ │ ├── __init__.py │ ├── document_processor.py # Load & chunk documents │ ├── vectorstore.py # ChromaDB operations │ ├── llm_handler.py # Ollama/LLM interface │ ├── rag_pipeline.py # RAG chain logic │ ├── main_document_loader.py # Main document management │ ├── response_enhancer.py # Response post-processing │ ├── config_loader.py # Load config.yaml & .env │ ├── build_vectorstore.py # CLI to build indexes │ └── retrieval/ # Extensible retrieval system │ ├── __init__.py │ ├── base.py # BaseRetrieverStrategy ABC │ ├── factory.py # RetrieverFactory │ ├── fusion.py # RRF and fusion algorithms │ ├── stores/ │ │ └── bm25_store.py # BM25 index storage │ └── strategies/ │ ├── vector.py # Vector-only strategy │ ├── bm25.py # BM25-only strategy │ └── bm25_vector.py # Hybrid BM25+Vector strategy ├── chroma_db/ # Vector database (auto-generated) ├── bm25_index/ # BM25 index (auto-generated) └── tests/ # Unit tests ``` ## 🎯 Recommended Models for HF Spaces | Model | Size | Speed | Quality | HF Spaces Tier | |-------|------|-------|---------|----------------| | `llama3.2:3b` | 3B | Fast | Good | Free ✅ | | `phi3:mini` | 3.8B | Fast | Good | Free ✅ | | `gemma2:2b` | 2B | Very Fast | Decent | Free ✅ | | `llama3.1:8b` | 8B | Medium | Great | Upgraded GPU | ## 🔍 LLM Tracing with LangSmith Monitor prompts, responses, and latency using [LangSmith](https://smith.langchain.com). ### Setup 1. **Get API Key**: Sign up at [smith.langchain.com](https://smith.langchain.com) (free tier: 5,000 traces/month) 2. **Configure `.env`**: ```bash LANGCHAIN_TRACING_V2=true LANGCHAIN_API_KEY=your_api_key_here LANGCHAIN_PROJECT=profillybot ``` 3. **Restart the app**: ```bash streamlit run app.py ``` ### What You Can See - Full prompts sent to LLM (system prompt + context + question) - Complete LLM responses - Latency breakdown per step - Token usage (when available) ## 🔧 Troubleshooting ### Ollama Connection Issues ```bash # Check Ollama is running ollama list # Start Ollama service ollama serve ``` ### ChromaDB / Index Errors ```bash # Rebuild all indexes from scratch rm -rf chroma_db/ bm25_index/ python -m src.build_vectorstore --force-rebuild ``` ### HuggingFace Spaces Issues - Check logs in the Space's "Logs" tab - Ensure `requirements.txt` is generated: `uv pip compile pyproject.toml -o requirements.txt` - Verify GPU/CPU settings match your model size ## 📄 License MIT License - see LICENSE file --- **Note**: Remember to update `config.yaml` with your personal information before deploying!