| # Chat with GitHub Repo - GenAI Project |
|
|
| ## Project Overview |
|
|
| A GenAI application that allows developers to paste a GitHub repository URL and ask natural language questions about the codebase. The system clones the repo, processes and embeds the code files, then uses RAG (Retrieval Augmented Generation) to answer questions about the code. |
|
|
| ## Architecture |
|
|
| ``` |
| ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ |
| │ Frontend │ │ Backend API │ │ Vector DB │ |
| │ (React/Next) │───▶│ (FastAPI) │───▶│ (Pinecone/ │ |
| │ │ │ │ │ Chroma) │ |
| └─────────────────┘ └─────────────────┘ └─────────────────┘ |
| │ |
| ▼ |
| ┌─────────────────┐ |
| │ GitHub API │ |
| │ + Git Clone │ |
| └─────────────────┘ |
| ``` |
|
|
| ## Tech Stack |
|
|
| ### Backend |
| - **FastAPI** - High-performance Python web framework |
| - **LangChain** - LLM orchestration and RAG implementation |
| - **Free LLM Options** - Ollama (local), Groq (fast inference), or Hugging Face |
| - **Free Embeddings** - Sentence Transformers (local) or Hugging Face |
| - **Chroma** - Vector database for embeddings |
| - **GitPython** - For cloning and processing repositories |
| - **Celery + Redis** - For background processing |
|
|
| ### Frontend |
| - **Next.js** - React framework |
| - **TypeScript** - Type safety |
| - **Tailwind CSS** - Styling |
| - **Socket.io** - Real-time updates |
|
|
| ## Backend Implementation |
|
|
| ### 1. Project Structure |
|
|
| ``` |
| backend/ |
| ├── app/ |
| │ ├── __init__.py |
| │ ├── main.py |
| │ ├── models/ |
| │ │ ├── __init__.py |
| │ │ └── schemas.py |
| │ ├── services/ |
| │ │ ├── __init__.py |
| │ │ ├── github_service.py |
| │ │ ├── embedding_service.py |
| │ │ └── chat_service.py |
| │ ├── utils/ |
| │ │ ├── __init__.py |
| │ │ └── file_processor.py |
| │ └── config.py |
| ├── requirements.txt |
| ├── docker-compose.yml |
| └── Dockerfile |
| ``` |
|
|
| ### 2. Core Dependencies (requirements.txt) |
|
|
| ```txt |
| fastapi==0.104.1 |
| uvicorn[standard]==0.24.0 |
| python-multipart==0.0.6 |
| pydantic==2.5.0 |
| langchain==0.1.0 |
| langchain-community==0.0.10 |
| chromadb==0.4.18 |
| GitPython==3.1.40 |
| python-dotenv==1.0.0 |
| celery==5.3.4 |
| redis==5.0.1 |
| socketio==5.10.0 |
| python-socketio==5.10.0 |
| aiofiles==23.2.1 |
|
|
| # Free LLM & Embedding Options |
| sentence-transformers==2.2.2 |
| transformers==4.36.0 |
| torch==2.1.0 |
| ollama==0.1.7 |
| groq==0.4.1 |
| huggingface-hub==0.19.4 |
| ``` |
|
|
| ### 3. Configuration (config.py) |
|
|
| ```python |
| import os |
| from pydantic_settings import BaseSettings |
| from enum import Enum |
|
|
| class LLMProvider(str, Enum): |
| OLLAMA = "ollama" |
| GROQ = "groq" |
| HUGGINGFACE = "huggingface" |
|
|
| class EmbeddingProvider(str, Enum): |
| SENTENCE_TRANSFORMERS = "sentence_transformers" |
| HUGGINGFACE = "huggingface" |
|
|
| class Settings(BaseSettings): |
| # LLM Configuration |
| llm_provider: LLMProvider = LLMProvider.OLLAMA |
| ollama_base_url: str = "http://localhost:11434" |
| ollama_model: str = "llama2" # or codellama, mistral, etc. |
| groq_api_key: str = "" |
| groq_model: str = "mixtral-8x7b-32768" |
| huggingface_api_key: str = "" |
| huggingface_model: str = "microsoft/DialoGPT-medium" |
| |
| # Embedding Configuration |
| embedding_provider: EmbeddingProvider = EmbeddingProvider.SENTENCE_TRANSFORMERS |
| sentence_transformer_model: str = "all-MiniLM-L6-v2" # Fast and good |
| # Alternative: "all-mpnet-base-v2" (better quality, slower) |
| |
| # Other settings |
| github_token: str = "" |
| redis_url: str = "redis://localhost:6379" |
| vector_db_path: str = "./chroma_db" |
| max_file_size: int = 1024 * 1024 # 1MB |
| supported_extensions: list = [ |
| ".py", ".js", ".ts", ".jsx", ".tsx", ".java", ".cpp", ".c", |
| ".cs", ".go", ".rs", ".php", ".rb", ".swift", ".kt", ".scala", |
| ".md", ".txt", ".json", ".yaml", ".yml", ".toml" |
| ] |
| |
| class Config: |
| env_file = ".env" |
|
|
| settings = Settings() |
| ``` |
|
|
| ### 4. Data Models (models/schemas.py) |
|
|
| ```python |
| from pydantic import BaseModel, HttpUrl |
| from typing import List, Optional |
| from enum import Enum |
|
|
| class ProcessingStatus(str, Enum): |
| PENDING = "pending" |
| PROCESSING = "processing" |
| COMPLETED = "completed" |
| FAILED = "failed" |
|
|
| class RepoProcessRequest(BaseModel): |
| repo_url: HttpUrl |
| branch: Optional[str] = "main" |
|
|
| class ChatMessage(BaseModel): |
| message: str |
| repo_id: str |
|
|
| class ChatResponse(BaseModel): |
| response: str |
| sources: List[dict] |
| repo_id: str |
|
|
| class RepoStatus(BaseModel): |
| repo_id: str |
| status: ProcessingStatus |
| progress: int |
| message: str |
| total_files: Optional[int] = None |
| processed_files: Optional[int] = None |
| ``` |
|
|
| ### 5. GitHub Service (services/github_service.py) |
|
|
| ```python |
| import os |
| import tempfile |
| import shutil |
| from git import Repo |
| from typing import List, Tuple |
| import hashlib |
| from urllib.parse import urlparse |
|
|
| class GitHubService: |
| def __init__(self, github_token: str = ""): |
| self.github_token = github_token |
| |
| def generate_repo_id(self, repo_url: str) -> str: |
| """Generate a unique ID for the repository""" |
| return hashlib.md5(repo_url.encode()).hexdigest() |
| |
| def parse_github_url(self, url: str) -> Tuple[str, str]: |
| """Extract owner and repo name from GitHub URL""" |
| parsed = urlparse(url) |
| path_parts = parsed.path.strip('/').split('/') |
| if len(path_parts) >= 2: |
| return path_parts[0], path_parts[1] |
| raise ValueError("Invalid GitHub URL format") |
| |
| async def clone_repository(self, repo_url: str, branch: str = "main") -> str: |
| """Clone repository to temporary directory""" |
| temp_dir = tempfile.mkdtemp() |
| try: |
| if self.github_token: |
| # Use token for private repos or higher rate limits |
| auth_url = repo_url.replace("https://", f"https://{self.github_token}@") |
| Repo.clone_from(auth_url, temp_dir, branch=branch, depth=1) |
| else: |
| Repo.clone_from(repo_url, temp_dir, branch=branch, depth=1) |
| return temp_dir |
| except Exception as e: |
| shutil.rmtree(temp_dir, ignore_errors=True) |
| raise Exception(f"Failed to clone repository: {str(e)}") |
| |
| def cleanup_repo(self, repo_path: str): |
| """Clean up cloned repository""" |
| if os.path.exists(repo_path): |
| shutil.rmtree(repo_path, ignore_errors=True) |
| ``` |
|
|
| ### 6. File Processor (utils/file_processor.py) |
|
|
| ```python |
| import os |
| from typing import List, Dict, Generator |
| import mimetypes |
| from pathlib import Path |
|
|
| class FileProcessor: |
| def __init__(self, supported_extensions: List[str], max_file_size: int): |
| self.supported_extensions = supported_extensions |
| self.max_file_size = max_file_size |
| self.ignore_dirs = { |
| '.git', '__pycache__', 'node_modules', '.pytest_cache', |
| 'venv', 'env', '.venv', 'build', 'dist', '.next', |
| 'coverage', '.coverage', 'logs', 'log' |
| } |
| self.ignore_files = { |
| '.gitignore', '.env', '.env.local', '.DS_Store', |
| 'package-lock.json', 'yarn.lock', 'poetry.lock' |
| } |
| |
| def should_process_file(self, file_path: str) -> bool: |
| """Check if file should be processed""" |
| path = Path(file_path) |
| |
| # Check if any parent directory is in ignore list |
| for parent in path.parents: |
| if parent.name in self.ignore_dirs: |
| return False |
| |
| # Check file name |
| if path.name in self.ignore_files: |
| return False |
| |
| # Check extension |
| if path.suffix.lower() not in self.supported_extensions: |
| return False |
| |
| # Check file size |
| try: |
| if os.path.getsize(file_path) > self.max_file_size: |
| return False |
| except OSError: |
| return False |
| |
| return True |
| |
| def extract_files(self, repo_path: str) -> Generator[Dict, None, None]: |
| """Extract and yield file information""" |
| for root, dirs, files in os.walk(repo_path): |
| # Filter out ignored directories |
| dirs[:] = [d for d in dirs if d not in self.ignore_dirs] |
| |
| for file in files: |
| file_path = os.path.join(root, file) |
| relative_path = os.path.relpath(file_path, repo_path) |
| |
| if not self.should_process_file(file_path): |
| continue |
| |
| try: |
| with open(file_path, 'r', encoding='utf-8', errors='ignore') as f: |
| content = f.read() |
| |
| yield { |
| 'path': relative_path, |
| 'content': content, |
| 'extension': Path(file_path).suffix.lower(), |
| 'size': len(content) |
| } |
| except Exception as e: |
| print(f"Error reading file {relative_path}: {e}") |
| continue |
| ``` |
|
|
| ### 7. Free Embedding Service (services/embedding_service.py) |
|
|
| ```python |
| from sentence_transformers import SentenceTransformer |
| from langchain.text_splitter import RecursiveCharacterTextSplitter |
| from langchain.schema import Document |
| from langchain_community.vectorstores import Chroma |
| from langchain_community.embeddings import HuggingFaceEmbeddings, SentenceTransformerEmbeddings |
| import chromadb |
| from typing import List, Dict |
| import os |
|
|
| class FreeEmbeddingService: |
| def __init__(self, embedding_provider: str, vector_db_path: str, model_name: str = "all-MiniLM-L6-v2"): |
| self.vector_db_path = vector_db_path |
| self.embedding_provider = embedding_provider |
| |
| # Initialize embedding function based on provider |
| if embedding_provider == "sentence_transformers": |
| self.embeddings = SentenceTransformerEmbeddings( |
| model_name=model_name, |
| cache_folder="./models" # Cache models locally |
| ) |
| elif embedding_provider == "huggingface": |
| self.embeddings = HuggingFaceEmbeddings( |
| model_name=model_name, |
| cache_folder="./models" |
| ) |
| else: |
| raise ValueError(f"Unsupported embedding provider: {embedding_provider}") |
| |
| self.text_splitter = RecursiveCharacterTextSplitter( |
| chunk_size=1000, |
| chunk_overlap=200, |
| separators=["\n\n", "\n", " ", ""] |
| ) |
| |
| def create_documents(self, files: List[Dict], repo_id: str) -> List[Document]: |
| """Create documents from file contents""" |
| documents = [] |
| |
| for file_info in files: |
| # Create document with metadata |
| doc = Document( |
| page_content=file_info['content'], |
| metadata={ |
| 'path': file_info['path'], |
| 'extension': file_info['extension'], |
| 'repo_id': repo_id, |
| 'size': file_info['size'] |
| } |
| ) |
| documents.append(doc) |
| |
| return documents |
| |
| def split_documents(self, documents: List[Document]) -> List[Document]: |
| """Split documents into chunks""" |
| return self.text_splitter.split_documents(documents) |
| |
| async def create_embeddings(self, files: List[Dict], repo_id: str): |
| """Create and store embeddings for repository files""" |
| # Create documents |
| documents = self.create_documents(files, repo_id) |
| |
| # Split into chunks |
| chunks = self.split_documents(documents) |
| |
| # Create vector store |
| collection_name = f"repo_{repo_id}" |
| vectorstore = Chroma( |
| collection_name=collection_name, |
| embedding_function=self.embeddings, |
| persist_directory=self.vector_db_path |
| ) |
| |
| # Add documents to vector store in batches |
| batch_size = 100 |
| for i in range(0, len(chunks), batch_size): |
| batch = chunks[i:i + batch_size] |
| vectorstore.add_documents(batch) |
| |
| return vectorstore |
| |
| def get_vectorstore(self, repo_id: str): |
| """Get existing vector store for repository""" |
| collection_name = f"repo_{repo_id}" |
| return Chroma( |
| collection_name=collection_name, |
| embedding_function=self.embeddings, |
| persist_directory=self.vector_db_path |
| ) |
|
|
|
|
| # Alternative: Direct SentenceTransformers implementation for more control |
| class DirectEmbeddingService: |
| def __init__(self, model_name: str = "all-MiniLM-L6-v2"): |
| self.model = SentenceTransformer(model_name, cache_folder="./models") |
| self.text_splitter = RecursiveCharacterTextSplitter( |
| chunk_size=1000, |
| chunk_overlap=200, |
| separators=["\n\n", "\n", " ", ""] |
| ) |
| |
| def embed_texts(self, texts: List[str]) -> List[List[float]]: |
| """Generate embeddings for texts""" |
| return self.model.encode(texts, convert_to_numpy=True).tolist() |
| |
| def embed_query(self, query: str) -> List[float]: |
| """Generate embedding for a single query""" |
| return self.model.encode([query], convert_to_numpy=True)[0].tolist() |
| ``` |
|
|
| ### 8. Free Chat Service (services/chat_service.py) |
|
|
| ```python |
| from langchain.chains import RetrievalQA |
| from langchain.prompts import PromptTemplate |
| from langchain_community.llms import Ollama |
| from typing import Dict, List |
| import json |
| import requests |
| import os |
|
|
| # For Groq (free tier available) |
| class GroqLLM: |
| def __init__(self, api_key: str, model: str = "mixtral-8x7b-32768"): |
| self.api_key = api_key |
| self.model = model |
| self.base_url = "https://api.groq.com/openai/v1" |
| |
| def __call__(self, prompt: str) -> str: |
| headers = { |
| "Authorization": f"Bearer {self.api_key}", |
| "Content-Type": "application/json" |
| } |
| |
| data = { |
| "model": self.model, |
| "messages": [{"role": "user", "content": prompt}], |
| "temperature": 0.1, |
| "max_tokens": 1024 |
| } |
| |
| response = requests.post( |
| f"{self.base_url}/chat/completions", |
| headers=headers, |
| json=data |
| ) |
| |
| if response.status_code == 200: |
| return response.json()["choices"][0]["message"]["content"] |
| else: |
| raise Exception(f"Groq API error: {response.text}") |
|
|
| # For Hugging Face Inference API |
| class HuggingFaceLLM: |
| def __init__(self, api_key: str, model: str = "microsoft/DialoGPT-medium"): |
| self.api_key = api_key |
| self.model = model |
| self.base_url = f"https://api-inference.huggingface.co/models/{model}" |
| |
| def __call__(self, prompt: str) -> str: |
| headers = {"Authorization": f"Bearer {self.api_key}"} |
| data = {"inputs": prompt, "parameters": {"max_length": 1000, "temperature": 0.1}} |
| |
| response = requests.post(self.base_url, headers=headers, json=data) |
| |
| if response.status_code == 200: |
| result = response.json() |
| if isinstance(result, list) and len(result) > 0: |
| return result[0].get("generated_text", "").replace(prompt, "").strip() |
| return str(result) |
| else: |
| raise Exception(f"HuggingFace API error: {response.text}") |
|
|
| class FreeChatService: |
| def __init__(self, llm_provider: str, **kwargs): |
| self.llm_provider = llm_provider |
| |
| if llm_provider == "ollama": |
| self.llm = Ollama( |
| model=kwargs.get("model", "llama2"), |
| base_url=kwargs.get("base_url", "http://localhost:11434"), |
| temperature=0.1 |
| ) |
| elif llm_provider == "groq": |
| self.llm = GroqLLM( |
| api_key=kwargs.get("api_key"), |
| model=kwargs.get("model", "mixtral-8x7b-32768") |
| ) |
| elif llm_provider == "huggingface": |
| self.llm = HuggingFaceLLM( |
| api_key=kwargs.get("api_key"), |
| model=kwargs.get("model", "microsoft/DialoGPT-medium") |
| ) |
| else: |
| raise ValueError(f"Unsupported LLM provider: {llm_provider}") |
| |
| self.prompt_template = PromptTemplate( |
| input_variables=["context", "question"], |
| template=""" |
| You are a helpful AI assistant that analyzes code repositories. Use the following code snippets to answer the user's question about the repository. |
|
|
| Context from repository: |
| {context} |
|
|
| Question: {question} |
|
|
| Please provide a detailed answer based on the code context provided. If you reference specific files or functions, mention their file paths. If the question cannot be fully answered from the provided context, say so clearly. |
|
|
| Answer:""" |
| ) |
| |
| async def answer_question(self, question: str, vectorstore, repo_id: str) -> Dict: |
| """Answer question using RAG with free LLM""" |
| try: |
| if self.llm_provider == "ollama": |
| # Use LangChain's RetrievalQA for Ollama |
| qa_chain = RetrievalQA.from_chain_type( |
| llm=self.llm, |
| chain_type="stuff", |
| retriever=vectorstore.as_retriever(search_kwargs={"k": 5}), |
| chain_type_kwargs={"prompt": self.prompt_template}, |
| return_source_documents=True |
| ) |
| |
| result = qa_chain({"query": question}) |
| answer = result["result"] |
| source_docs = result.get("source_documents", []) |
| |
| else: |
| # Manual RAG for other providers |
| docs = vectorstore.similarity_search(question, k=5) |
| context = "\n\n".join([doc.page_content for doc in docs]) |
| |
| prompt = self.prompt_template.format( |
| context=context, |
| question=question |
| ) |
| |
| answer = self.llm(prompt) |
| source_docs = docs |
| |
| # Format sources |
| sources = [] |
| for doc in source_docs: |
| sources.append({ |
| "path": doc.metadata.get("path", "Unknown"), |
| "content_preview": doc.page_content[:200] + "..." if len(doc.page_content) > 200 else doc.page_content |
| }) |
| |
| return { |
| "response": answer, |
| "sources": sources, |
| "repo_id": repo_id |
| } |
| |
| except Exception as e: |
| return { |
| "response": f"Error processing question: {str(e)}", |
| "sources": [], |
| "repo_id": repo_id |
| } |
| ``` |
|
|
| ### 9. Updated Main FastAPI Application (main.py) |
|
|
| ```python |
| from fastapi import FastAPI, HTTPException, BackgroundTasks |
| from fastapi.middleware.cors import CORSMiddleware |
| import uvicorn |
| from typing import Dict |
| import asyncio |
| from datetime import datetime |
|
|
| from models.schemas import RepoProcessRequest, ChatMessage, ChatResponse, RepoStatus, ProcessingStatus |
| from services.github_service import GitHubService |
| from services.embedding_service import FreeEmbeddingService |
| from services.chat_service import FreeChatService |
| from utils.file_processor import FileProcessor |
| from config import settings |
|
|
| app = FastAPI(title="Chat with GitHub Repo (Free Version)", version="1.0.0") |
|
|
| # CORS middleware |
| app.add_middleware( |
| CORSMiddleware, |
| allow_origins=["*"], # Configure properly for production |
| allow_credentials=True, |
| allow_methods=["*"], |
| allow_headers=["*"], |
| ) |
|
|
| # Initialize services based on configuration |
| github_service = GitHubService(settings.github_token) |
|
|
| embedding_service = FreeEmbeddingService( |
| embedding_provider=settings.embedding_provider.value, |
| vector_db_path=settings.vector_db_path, |
| model_name=settings.sentence_transformer_model |
| ) |
|
|
| # Initialize chat service based on provider |
| chat_kwargs = {} |
| if settings.llm_provider.value == "ollama": |
| chat_kwargs = { |
| "model": settings.ollama_model, |
| "base_url": settings.ollama_base_url |
| } |
| elif settings.llm_provider.value == "groq": |
| chat_kwargs = { |
| "api_key": settings.groq_api_key, |
| "model": settings.groq_model |
| } |
| elif settings.llm_provider.value == "huggingface": |
| chat_kwargs = { |
| "api_key": settings.huggingface_api_key, |
| "model": settings.huggingface_model |
| } |
|
|
| chat_service = FreeChatService( |
| llm_provider=settings.llm_provider.value, |
| **chat_kwargs |
| ) |
|
|
| file_processor = FileProcessor(settings.supported_extensions, settings.max_file_size) |
|
|
| # In-memory status tracking (use Redis in production) |
| repo_status: Dict[str, RepoStatus] = {} |
|
|
| async def process_repository(repo_url: str, branch: str, repo_id: str): |
| """Background task to process repository""" |
| try: |
| repo_status[repo_id] = RepoStatus( |
| repo_id=repo_id, |
| status=ProcessingStatus.PROCESSING, |
| progress=10, |
| message="Cloning repository..." |
| ) |
| |
| # Clone repository |
| repo_path = await github_service.clone_repository(repo_url, branch) |
| |
| repo_status[repo_id].progress = 30 |
| repo_status[repo_id].message = "Processing files..." |
| |
| # Extract files |
| files = list(file_processor.extract_files(repo_path)) |
| |
| repo_status[repo_id].total_files = len(files) |
| repo_status[repo_id].progress = 50 |
| repo_status[repo_id].message = "Creating embeddings (this may take a while for large repos)..." |
| |
| # Create embeddings |
| await embedding_service.create_embeddings(files, repo_id) |
| |
| # Cleanup |
| github_service.cleanup_repo(repo_path) |
| |
| repo_status[repo_id].status = ProcessingStatus.COMPLETED |
| repo_status[repo_id].progress = 100 |
| repo_status[repo_id].message = f"Repository processed successfully! Using {settings.llm_provider.value} for chat." |
| |
| except Exception as e: |
| repo_status[repo_id].status = ProcessingStatus.FAILED |
| repo_status[repo_id].message = f"Error: {str(e)}" |
|
|
| @app.post("/api/process-repo") |
| async def process_repo(request: RepoProcessRequest, background_tasks: BackgroundTasks): |
| """Process a GitHub repository""" |
| repo_id = github_service.generate_repo_id(str(request.repo_url)) |
| |
| # Check if already processed |
| if repo_id in repo_status and repo_status[repo_id].status == ProcessingStatus.COMPLETED: |
| return {"repo_id": repo_id, "message": "Repository already processed"} |
| |
| # Start processing |
| repo_status[repo_id] = RepoStatus( |
| repo_id=repo_id, |
| status=ProcessingStatus.PENDING, |
| progress=0, |
| message="Starting processing..." |
| ) |
| |
| background_tasks.add_task(process_repository, str(request.repo_url), request.branch, repo_id) |
| |
| return {"repo_id": repo_id, "message": "Processing started"} |
|
|
| @app.get("/api/status/{repo_id}", response_model=RepoStatus) |
| async def get_repo_status(repo_id: str): |
| """Get repository processing status""" |
| if repo_id not in repo_status: |
| raise HTTPException(status_code=404, detail="Repository not found") |
| |
| return repo_status[repo_id] |
|
|
| @app.post("/api/chat", response_model=ChatResponse) |
| async def chat_with_repo(message: ChatMessage): |
| """Chat with repository""" |
| repo_id = message.repo_id |
| |
| # Check if repo is processed |
| if repo_id not in repo_status or repo_status[repo_id].status != ProcessingStatus.COMPLETED: |
| raise HTTPException(status_code=400, detail="Repository not processed") |
| |
| try: |
| # Get vector store |
| vectorstore = embedding_service.get_vectorstore(repo_id) |
| |
| # Get answer |
| result = await chat_service.answer_question(message.message, vectorstore, repo_id) |
| |
| return ChatResponse(**result) |
| |
| except Exception as e: |
| raise HTTPException(status_code=500, detail=str(e)) |
|
|
| @app.get("/api/health") |
| async def health_check(): |
| return { |
| "status": "healthy", |
| "timestamp": datetime.utcnow(), |
| "llm_provider": settings.llm_provider.value, |
| "embedding_provider": settings.embedding_provider.value |
| } |
|
|
| @app.get("/api/config") |
| async def get_config(): |
| """Get current configuration""" |
| return { |
| "llm_provider": settings.llm_provider.value, |
| "embedding_provider": settings.embedding_provider.value, |
| "embedding_model": settings.sentence_transformer_model, |
| "supported_extensions": settings.supported_extensions |
| } |
|
|
| if __name__ == "__main__": |
| uvicorn.run(app, host="0.0.0.0", port=8000, reload=True) |
| ```_service.create_embeddings(files, repo_id) |
| |
| # Cleanup |
| github_service.cleanup_repo(repo_path) |
| |
| repo_status[repo_id].status = ProcessingStatus.COMPLETED |
| repo_status[repo_id].progress = 100 |
| repo_status[repo_id].message = "Repository processed successfully!" |
| |
| except Exception as e: |
| repo_status[repo_id].status = ProcessingStatus.FAILED |
| repo_status[repo_id].message = f"Error: {str(e)}" |
|
|
| @app.post("/api/process-repo") |
| async def process_repo(request: RepoProcessRequest, background_tasks: BackgroundTasks): |
| """Process a GitHub repository""" |
| repo_id = github_service.generate_repo_id(str(request.repo_url)) |
| |
| # Check if already processed |
| if repo_id in repo_status and repo_status[repo_id].status == ProcessingStatus.COMPLETED: |
| return {"repo_id": repo_id, "message": "Repository already processed"} |
| |
| # Start processing |
| repo_status[repo_id] = RepoStatus( |
| repo_id=repo_id, |
| status=ProcessingStatus.PENDING, |
| progress=0, |
| message="Starting processing..." |
| ) |
| |
| background_tasks.add_task(process_repository, str(request.repo_url), request.branch, repo_id) |
| |
| return {"repo_id": repo_id, "message": "Processing started"} |
|
|
| @app.get("/api/status/{repo_id}", response_model=RepoStatus) |
| async def get_repo_status(repo_id: str): |
| """Get repository processing status""" |
| if repo_id not in repo_status: |
| raise HTTPException(status_code=404, detail="Repository not found") |
| |
| return repo_status[repo_id] |
|
|
| @app.post("/api/chat", response_model=ChatResponse) |
| async def chat_with_repo(message: ChatMessage): |
| """Chat with repository""" |
| repo_id = message.repo_id |
| |
| # Check if repo is processed |
| if repo_id not in repo_status or repo_status[repo_id].status != ProcessingStatus.COMPLETED: |
| raise HTTPException(status_code=400, detail="Repository not processed") |
| |
| try: |
| # Get vector store |
| vectorstore = embedding_service.get_vectorstore(repo_id) |
| |
| # Get answer |
| result = await chat_service.answer_question(message.message, vectorstore, repo_id) |
| |
| return ChatResponse(**result) |
| |
| except Exception as e: |
| raise HTTPException(status_code=500, detail=str(e)) |
|
|
| @app.get("/api/health") |
| async def health_check(): |
| return {"status": "healthy", "timestamp": datetime.utcnow()} |
|
|
| if __name__ == "__main__": |
| uvicorn.run(app, host="0.0.0.0", port=8000, reload=True) |
| ``` |
|
|
| ### 10. Environment Configuration (.env) |
|
|
| ```env |
| # LLM Provider Configuration |
| LLM_PROVIDER=ollama # Options: ollama, groq, huggingface |
| EMBEDDING_PROVIDER=sentence_transformers # Options: sentence_transformers, huggingface |
|
|
| # Ollama Configuration (Local LLM - Free) |
| OLLAMA_BASE_URL=http://localhost:11434 |
| OLLAMA_MODEL=llama2 # Options: llama2, codellama, mistral, phi, etc. |
|
|
| # Groq Configuration (Fast inference - Free tier available) |
| GROQ_API_KEY=your_groq_api_key_here |
| GROQ_MODEL=mixtral-8x7b-32768 # Options: mixtral-8x7b-32768, llama2-70b-4096 |
|
|
| # Hugging Face Configuration (Free inference API) |
| HUGGINGFACE_API_KEY=your_hf_api_key_here |
| HUGGINGFACE_MODEL=microsoft/DialoGPT-medium |
|
|
| # Embedding Model Configuration |
| SENTENCE_TRANSFORMER_MODEL=all-MiniLM-L6-v2 # Fast and good quality |
|
|
| # Other Configuration |
| GITHUB_TOKEN=your_github_token_here |
| REDIS_URL=redis://localhost:6379 |
| VECTOR_DB_PATH=./chroma_db |
| ``` |
|
|
| ### 11. Updated Docker Configuration |
|
|
| **Dockerfile:** |
| ```dockerfile |
| FROM python:3.11-slim |
|
|
| WORKDIR /app |
|
|
| # Install system dependencies |
| RUN apt-get update && apt-get install -y \ |
| git \ |
| build-essential \ |
| && rm -rf /var/lib/apt/lists/* |
|
|
| COPY requirements.txt . |
| RUN pip install --no-cache-dir -r requirements.txt |
|
|
| # Pre-download embedding models |
| RUN python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2', cache_folder='./models')" |
|
|
| COPY . . |
|
|
| EXPOSE 8000 |
|
|
| CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] |
| ``` |
|
|
| **docker-compose.yml:** |
| ```yaml |
| version: '3.8' |
|
|
| services: |
| api: |
| build: . |
| ports: |
| - "8000:8000" |
| environment: |
| - LLM_PROVIDER=${LLM_PROVIDER} |
| - EMBEDDING_PROVIDER=${EMBEDDING_PROVIDER} |
| - OLLAMA_BASE_URL=http://ollama:11434 |
| - OLLAMA_MODEL=${OLLAMA_MODEL} |
| - GROQ_API_KEY=${GROQ_API_KEY} |
| - GROQ_MODEL=${GROQ_MODEL} |
| - HUGGINGFACE_API_KEY=${HUGGINGFACE_API_KEY} |
| - HUGGINGFACE_MODEL=${HUGGINGFACE_MODEL} |
| - SENTENCE_TRANSFORMER_MODEL=${SENTENCE_TRANSFORMER_MODEL} |
| - GITHUB_TOKEN=${GITHUB_TOKEN} |
| - REDIS_URL=redis://redis:6379 |
| volumes: |
| - ./chroma_db:/app/chroma_db |
| - ./models:/app/models # Cache for models |
| depends_on: |
| - redis |
| - ollama |
| |
| ollama: |
| image: ollama/ollama:latest |
| ports: |
| - "11434:11434" |
| volumes: |
| - ollama_data:/root/.ollama |
| environment: |
| - OLLAMA_KEEP_ALIVE=24h |
| |
| redis: |
| image: redis:7-alpine |
| ports: |
| - "6379:6379" |
|
|
| volumes: |
| ollama_data: |
| ``` |
|
|
| **Setup script for Ollama models (setup_ollama.sh):** |
| ```bash |
| #!/bin/bash |
| # Pull required models |
| docker exec chat-with-github-repo-ollama-1 ollama pull llama2 |
| docker exec chat-with-github-repo-ollama-1 ollama pull codellama |
| docker exec chat-with-github-repo-ollama-1 ollama pull mistral |
| ``` |
|
|
| ## Performance Comparison |
|
|
| | Provider | Speed | Quality | Cost | Setup Difficulty | |
| |----------|-------|---------|------|------------------| |
| | Ollama + Llama2 | Medium | Good | Free | Easy | |
| | Ollama + CodeLlama | Medium | Excellent (Code) | Free | Easy | |
| | Groq + Mixtral | Very Fast | Excellent | Free Tier | Very Easy | |
| | HuggingFace | Slow | Variable | Free | Very Easy | |
| | Local Transformers | Slow-Medium | Good | Free | Medium | |
|
|
| ## Recommended Configurations |
|
|
| ### For Development/Testing: |
| ```env |
| LLM_PROVIDER=groq |
| GROQ_MODEL=mixtral-8x7b-32768 |
| EMBEDDING_PROVIDER=sentence_transformers |
| SENTENCE_TRANSFORMER_MODEL=all-MiniLM-L6-v2 |
| ``` |
|
|
| ### For Production (Local): |
| ```env |
| LLM_PROVIDER=ollama |
| OLLAMA_MODEL=codellama |
| EMBEDDING_PROVIDER=sentence_transformers |
| SENTENCE_TRANSFORMER_MODEL=all-mpnet-base-v2 |
| ``` |
|
|
| ### For Minimal Resources: |
| ```env |
| LLM_PROVIDER=ollama |
| OLLAMA_MODEL=phi |
| EMBEDDING_PROVIDER=sentence_transformers |
| SENTENCE_TRANSFORMER_MODEL=all-MiniLM-L6-v2 |
| ``` |
|
|
| ## Troubleshooting |
|
|
| ### Ollama Issues: |
| ```bash |
| # Check if Ollama is running |
| curl http://localhost:11434/api/version |
|
|
| # List available models |
| ollama list |
|
|
| # Check logs |
| ollama logs |
| ``` |
|
|
| ### Memory Issues: |
| - Use smaller models (`phi` instead of `llama2`) |
| - Reduce batch size in embedding service |
| - Use quantized models |
| - Process repos in smaller chunks |
|
|
| ### Performance Optimization: |
| ```python |
| # For faster embeddings |
| embedding_service = FreeEmbeddingService( |
| embedding_provider="sentence_transformers", |
| vector_db_path="./chroma_db", |
| model_name="all-MiniLM-L6-v2" # Faster than all-mpnet-base-v2 |
| ) |
|
|
| # Batch processing |
| async def create_embeddings_batch(self, files: List[Dict], repo_id: str, batch_size: int = 50): |
| for i in range(0, len(files), batch_size): |
| batch = files[i:i + batch_size] |
| # Process batch... |
| ``` |
|
|
| ## Deployment & Scaling |
|
|
| ### Project Structure |
| ``` |
| frontend/ |
| ├── src/ |
| │ ├── app/ |
| │ │ ├── page.tsx |
| │ │ ├── layout.tsx |
| │ │ └── globals.css |
| │ ├── components/ |
| │ │ ├── RepoInput.tsx |
| │ │ ├── ChatInterface.tsx |
| │ │ ├── ProcessingStatus.tsx |
| │ │ └── SourceDisplay.tsx |
| │ └── lib/ |
| │ └── api.ts |
| ├── package.json |
| ├── tailwind.config.js |
| └── next.config.js |
| ``` |
|
|
| ### Key Features |
| - Repository URL input with validation |
| - Real-time processing status updates |
| - Chat interface with message history |
| - Source code display with syntax highlighting |
| - Responsive design |
|
|
| ## Free Alternatives to OpenAI |
|
|
| ### 1. **Ollama (Recommended for Local Deployment)** |
| **Pros:** |
| - 100% free and private |
| - Runs locally, no API calls |
| - Supports many models (Llama2, CodeLlama, Mistral, Phi, etc.) |
| - Good performance on decent hardware |
|
|
| **Setup:** |
| ```bash |
| # Install Ollama |
| curl -fsSL https://ollama.ai/install.sh | sh |
|
|
| # Pull models |
| ollama pull llama2 # General purpose |
| ollama pull codellama # Better for code |
| ollama pull mistral # Good balance |
| ollama pull phi # Lightweight |
|
|
| # Start Ollama server |
| ollama serve |
| ``` |
|
|
| **Requirements:** 8GB+ RAM, preferably with GPU |
|
|
| ### 2. **Groq (Fast Inference - Free Tier)** |
| **Pros:** |
| - Extremely fast inference |
| - Free tier: 100 requests/minute |
| - High-quality models (Mixtral, Llama2) |
| - Simple API |
|
|
| **Setup:** |
| 1. Sign up at [groq.com](https://groq.com) |
| 2. Get API key from console |
| 3. Set `GROQ_API_KEY` in environment |
|
|
| **Free Limits:** 100 requests/minute, 1000 requests/day |
|
|
| ### 3. **Hugging Face Inference API (Free)** |
| **Pros:** |
| - Completely free |
| - Access to thousands of models |
| - No setup required |
| - Good for experimentation |
|
|
| **Setup:** |
| 1. Sign up at [huggingface.co](https://huggingface.co) |
| 2. Get API token from settings |
| 3. Set `HUGGINGFACE_API_KEY` in environment |
|
|
| **Note:** Can be slower due to cold starts |
|
|
| ### 4. **Local Transformers (Completely Free)** |
| For maximum control, you can run models directly: |
|
|
| ```python |
| # services/local_llm_service.py |
| from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM |
| import torch |
|
|
| class LocalLLMService: |
| def __init__(self, model_name="microsoft/DialoGPT-small"): |
| self.tokenizer = AutoTokenizer.from_pretrained(model_name) |
| self.model = AutoModelForCausalLM.from_pretrained(model_name) |
| self.generator = pipeline( |
| "text-generation", |
| model=self.model, |
| tokenizer=self.tokenizer, |
| device=0 if torch.cuda.is_available() else -1 |
| ) |
| |
| def generate_response(self, prompt: str, max_length: int = 512): |
| response = self.generator( |
| prompt, |
| max_length=max_length, |
| temperature=0.7, |
| do_sample=True, |
| pad_token_id=self.tokenizer.eos_token_id |
| ) |
| return response[0]["generated_text"].replace(prompt, "").strip() |
| ``` |
|
|
| ## Embedding Models Comparison |
|
|
| ### 1. **Sentence Transformers (Recommended)** |
| ```python |
| # Best models for code: |
| "all-MiniLM-L6-v2" # Fast, good quality |
| "all-mpnet-base-v2" # Better quality, slower |
| "multi-qa-MiniLM-L6-cos-v1" # Good for Q&A |
| ``` |
|
|
| ### 2. **Hugging Face Embeddings** |
| ```python |
| # Popular models: |
| "sentence-transformers/all-MiniLM-L6-v2" |
| "sentence-transformers/paraphrase-MiniLM-L6-v2" |
| ``` |
|
|
| ## Quick Start Guide |
|
|
| ### Option 1: Ollama (Local) |
| ```bash |
| # 1. Install and start Ollama |
| curl -fsSL https://ollama.ai/install.sh | sh |
| ollama serve |
|
|
| # 2. Pull a model |
| ollama pull llama2 |
|
|
| # 3. Set environment variables |
| export LLM_PROVIDER=ollama |
| export OLLAMA_MODEL=llama2 |
| export EMBEDDING_PROVIDER=sentence_transformers |
|
|
| # 4. Start the application |
| python main.py |
| ``` |
|
|
| ### Option 2: Groq (Cloud) |
| ```bash |
| # 1. Get API key from groq.com |
| export GROQ_API_KEY=your_api_key_here |
| export LLM_PROVIDER=groq |
| export GROQ_MODEL=mixtral-8x7b-32768 |
| export EMBEDDING_PROVIDER=sentence_transformers |
|
|
| # 2. Start the application |
| python main.py |
| ``` |
|
|
| ### Option 3: Hugging Face (Cloud) |
| ```bash |
| # 1. Get token from huggingface.co |
| export HUGGINGFACE_API_KEY=your_token_here |
| export LLM_PROVIDER=huggingface |
| export HUGGINGFACE_MODEL=microsoft/DialoGPT-medium |
| export EMBEDDING_PROVIDER=sentence_transformers |
|
|
| # 2. Start the application |
| python main.py |
| ``` |
|
|
| ### Production Considerations |
| 1. **Vector Database**: Use Pinecone for better scalability |
| 2. **Background Processing**: Implement with Celery + Redis |
| 3. **Caching**: Add Redis caching for frequent queries |
| 4. **Rate Limiting**: Implement API rate limiting |
| 5. **Authentication**: Add user authentication |
| 6. **Monitoring**: Add logging and monitoring (Sentry, DataDog) |
|
|
| ### Performance Optimizations |
| 1. **Chunking Strategy**: Optimize chunk size and overlap |
| 2. **Embedding Model**: Consider using smaller models for faster processing |
| 3. **Retrieval**: Implement hybrid search (dense + sparse) |
| 4. **Caching**: Cache embeddings and frequently asked questions |
|
|
| ## Usage Examples |
|
|
| ### Sample Questions Users Can Ask: |
| - "How is authentication handled in this project?" |
| - "Explain the UserService class and its methods" |
| - "What testing framework is used and where are the tests?" |
| - "How does the database connection work?" |
| - "What are the main API endpoints?" |
| - "Show me how error handling is implemented" |
|
|
| ## Next Steps & Enhancements |
|
|
| 1. **Multi-language Support**: Better handling of different programming languages |
| 2. **Code Analysis**: Add static code analysis features |
| 3. **Visualization**: Generate architecture diagrams |
| 4. **Collaboration**: Multi-user support with shared repositories |
| 5. **Integration**: GitHub webhook integration for auto-updates |
| 6. **AI Features**: Code suggestions and improvements |
|
|
| This project provides a solid foundation for building a production-ready "Chat with GitHub Repo" application that developers will find incredibly useful for understanding and navigating codebases! |
|
|
