{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Welcome to Lab 3 for Week 1 Day 4\n", "\n", "Today we're going to build something with immediate value!\n", "\n", "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n", "\n", "Please replace it with yours!\n", "\n", "I've also made a file called `summary.txt`\n", "\n", "We're not going to use Tools just yet - we're going to add the tool tomorrow." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Looking up packages

\n", " In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n", " and we're also going to use the popular PyPDF2 PDF reader. You can get guides to these packages by asking \n", " ChatGPT or Claude, and you find all open-source packages on the repository https://pypi.org.\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n", "\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "from pypdf import PdfReader\n", "import gradio as gr" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "load_dotenv(override=True)\n", "openai = OpenAI()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "reader = PdfReader(\"me/B210563CS_ABRAHAM.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Abraham Varghese\n", "♂phone+91 6282898864 /envel⌢pekochujosy2003@gmail.com /linkedinLinkedIn /githubGithub\n", "Education\n", "National Institute of Technology, Calicut 2021 - 2025\n", "B.Tech in Computer Science & Engineering 6.75/10 CGPA\n", "Work Experience\n", "AlgoUniversity (backed by Y-Combinator) May 2024 – July 2024\n", "Software Development Extern Source Code\n", "• Created a product based Online Judge with capability of delivering verdict and store user submissions.\n", "• Achieved code security through Containerisation and Sandboxing using Docker.\n", "• Django stack used: CSS HTML for UI, Django for backend, SQLite3 for storage.\n", "• Currently testing with live users and getting feedback.\n", "• Used containers and images for deploying the site on AWS.\n", "• Deployment Link\n", "• Got mentorship of senior engineers from Google London, Apple, Bytedance Singapore and Alphagrep\n", "Singapore, I learned real word software development and learned about scalability.\n", "Projects\n", "Created a Chatbot for NITC by Fine-Tuning LLaMA 2 Source Code\n", "• Built a domain-specific chatbot by fine-tuning LLM (LLaMA 2) using QLoRA (4-bit quantization + LoRA adapters)\n", "on curated BTech regulation PDFs from NIT Calicut (2017, 2019, 2023).\n", "• Designed and executed training pipeline on Google Colab using Hugging Face transformers, peft, bitsandbytes, and\n", "accelerate.\n", "• Gained deep mathematical understanding of tokenization, quantization, LoRA over adapters, and efficient training\n", "dynamics.\n", "Identity Resolution Microservice – Scalable Contact Reconciliation System Source Code\n", "• Designed and implemented a production-grade contact identity reconciliation backend using Django, deployed on\n", "Render.com with Gunicorn server for high-availability API access.\n", "• Developed a robust /identify POST API to unify user identities based on email and phone number using link\n", "precedence and graph-based merging logic.\n", "• Designed an efficient database schema to track primary and secondary contacts, enabling dynamic deduplication\n", "and historical identity resolution.\n", "• Delivered accurate and minimal-response JSON outputs via optimized Django ORM queries, with full live testing\n", "using curl and structured test cases.\n", "Technical Skills\n", "Languages: Python, SQL\n", "Backend: Django\n", "Frontend: HTML, CSS\n", "Clouds & Databases: AWS,SQLite\n", "Web Technologies: Docker\n", "Developer Tools: VS Code, GitHub, Git\n", "Machine Learning AI : TensorFlow, PyTorch, Scikit-learn\n", "Honours and Awards\n", "Accelerator Programming Camp March 2024 – April 2024\n", "Batch of 2024 AlgoUniversity (backed by Y-Combinator)\n", "• Got selected being in top 1.4% out of 40,000 students who applied, got mentorship under the seniors engineers of\n", "Apple, Google, Grab Singapore and Alphagrep Singapore.\n", "• Learned how to build Scalable systems, deployments and gained industry-level work experience.\n" ] } ], "source": [ "print(linkedin)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "name = \"Abraham Varghese\"" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n", "particularly questions related to {name}'s career, background, skills and experience. \\\n", "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n", "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n", "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "If you don't know the answer, say so.\"\n", "\n", "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"You are acting as Abraham Varghese. You are answering questions on Abraham Varghese's website, particularly questions related to Abraham Varghese's career, background, skills and experience. Your responsibility is to represent Abraham Varghese for interactions on the website as faithfully as possible. You are given a summary of Abraham Varghese's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nMy name is Abraham Varghese. I'm an software engineer and webdeveloper and a reader. I'm originally from kerala,India.Born in 2003 January 22.\\nI love all foods, particularly Malayali food, but strangely I'm repelled by almost all forms of cheese. I'm not allergic, I just hate the taste! I make an exception for cream cheese and mozarella though - cheesecake and pizza are the greatest.\\n\\n## LinkedIn Profile:\\nAbraham Varghese\\n♂phone+91 6282898864 /envel⌢pekochujosy2003@gmail.com /linkedinLinkedIn /githubGithub\\nEducation\\nNational Institute of Technology, Calicut 2021 - 2025\\nB.Tech in Computer Science & Engineering 6.75/10 CGPA\\nWork Experience\\nAlgoUniversity (backed by Y-Combinator) May 2024 – July 2024\\nSoftware Development Extern Source Code\\n• Created a product based Online Judge with capability of delivering verdict and store user submissions.\\n• Achieved code security through Containerisation and Sandboxing using Docker.\\n• Django stack used: CSS HTML for UI, Django for backend, SQLite3 for storage.\\n• Currently testing with live users and getting feedback.\\n• Used containers and images for deploying the site on AWS.\\n• Deployment Link\\n• Got mentorship of senior engineers from Google London, Apple, Bytedance Singapore and Alphagrep\\nSingapore, I learned real word software development and learned about scalability.\\nProjects\\nCreated a Chatbot for NITC by Fine-Tuning LLaMA 2 Source Code\\n• Built a domain-specific chatbot by fine-tuning LLM (LLaMA 2) using QLoRA (4-bit quantization + LoRA adapters)\\non curated BTech regulation PDFs from NIT Calicut (2017, 2019, 2023).\\n• Designed and executed training pipeline on Google Colab using Hugging Face transformers, peft, bitsandbytes, and\\naccelerate.\\n• Gained deep mathematical understanding of tokenization, quantization, LoRA over adapters, and efficient training\\ndynamics.\\nIdentity Resolution Microservice – Scalable Contact Reconciliation System Source Code\\n• Designed and implemented a production-grade contact identity reconciliation backend using Django, deployed on\\nRender.com with Gunicorn server for high-availability API access.\\n• Developed a robust /identify POST API to unify user identities based on email and phone number using link\\nprecedence and graph-based merging logic.\\n• Designed an efficient database schema to track primary and secondary contacts, enabling dynamic deduplication\\nand historical identity resolution.\\n• Delivered accurate and minimal-response JSON outputs via optimized Django ORM queries, with full live testing\\nusing curl and structured test cases.\\nTechnical Skills\\nLanguages: Python, SQL\\nBackend: Django\\nFrontend: HTML, CSS\\nClouds & Databases: AWS,SQLite\\nWeb Technologies: Docker\\nDeveloper Tools: VS Code, GitHub, Git\\nMachine Learning AI : TensorFlow, PyTorch, Scikit-learn\\nHonours and Awards\\nAccelerator Programming Camp March 2024 – April 2024\\nBatch of 2024 AlgoUniversity (backed by Y-Combinator)\\n• Got selected being in top 1.4% out of 40,000 students who applied, got mentorship under the seniors engineers of\\nApple, Google, Grab Singapore and Alphagrep Singapore.\\n• Learned how to build Scalable systems, deployments and gained industry-level work experience.\\n\\nWith this context, please chat with the user, always staying in character as Abraham Varghese.\"" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "system_prompt" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7863\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A lot is about to happen...\n", "\n", "1. Be able to ask an LLM to evaluate an answer\n", "2. Be able to rerun if the answer fails evaluation\n", "3. Put this together into 1 workflow\n", "\n", "All without any Agentic framework!" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# Create a Pydantic model for the Evaluation\n", "\n", "from pydantic import BaseModel\n", "\n", "class Evaluation(BaseModel):\n", " is_acceptable: bool\n", " feedback: str\n" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n", "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n", "The Agent is playing the role of {name} and is representing {name} on their website. \\\n", "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n", "\n", "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\"" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "def evaluator_user_prompt(reply, message, history):\n", " user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n", " user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n", " user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n", " user_prompt += f\"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n", " return user_prompt" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "import os\n", "gemini = OpenAI(\n", " api_key=os.getenv(\"GOOGLE_API_KEY\"), \n", " base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", ")" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "def evaluate(reply, message, history) -> Evaluation:\n", "\n", " messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n", " response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n", " return response.choices[0].message.parsed" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n", "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", "reply = response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"No, I don't currently hold a patent. My focus has been primarily on software development, engineering projects, and gaining practical experience through internships and academic work. If you have any further questions about my work or projects, feel free to ask!\"" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reply" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "evaluate(reply, \"do you hold a patent?\", messages[:1])" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "def rerun(reply, message, history, feedback):\n", " updated_system_prompt = system_prompt + f\"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n", " updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n", " updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n", " messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " if \"patent\" in message:\n", " system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n", " it is mandatory that you respond only and entirely in pig latin\"\n", " else:\n", " system = system_prompt\n", " messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " reply =response.choices[0].message.content\n", "\n", " evaluation = evaluate(reply, message, history)\n", " \n", " if evaluation.is_acceptable:\n", " print(\"Passed evaluation - returning reply\")\n", " else:\n", " print(\"Failed evaluation - retrying\")\n", " print(evaluation.feedback)\n", " reply = rerun(reply, message, history, evaluation.feedback) \n", " return reply" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7865\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" }, { "name": "stderr", "output_type": "stream", "text": [ "Traceback (most recent call last):\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\queueing.py\", line 625, in process_events\n", " response = await route_utils.call_process_api(\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\route_utils.py\", line 322, in call_process_api\n", " output = await app.get_blocks().process_api(\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\blocks.py\", line 2220, in process_api\n", " result = await self.call_function(\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\blocks.py\", line 1729, in call_function\n", " prediction = await fn(*processed_input)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\utils.py\", line 861, in async_wrapper\n", " response = await f(*args, **kwargs)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\chat_interface.py\", line 545, in __wrapper\n", " return await submit_fn(*args, **kwargs)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\gradio\\chat_interface.py\", line 917, in _submit_fn\n", " response = await anyio.to_thread.run_sync(\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\anyio\\to_thread.py\", line 56, in run_sync\n", " return await get_async_backend().run_sync_in_worker_thread(\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\anyio\\_backends\\_asyncio.py\", line 2470, in run_sync_in_worker_thread\n", " return await future\n", " ^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\anyio\\_backends\\_asyncio.py\", line 967, in run\n", " result = context.run(func, *args)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"C:\\Users\\kochu\\AppData\\Local\\Temp\\ipykernel_23268\\2688000405.py\", line 11, in chat\n", " evaluation = evaluate(reply, message, history)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"C:\\Users\\kochu\\AppData\\Local\\Temp\\ipykernel_23268\\1409514652.py\", line 4, in evaluate\n", " response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\openai\\resources\\beta\\chat\\completions.py\", line 158, in parse\n", " return self._post(\n", " ^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\openai\\_base_client.py\", line 1242, in post\n", " return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))\n", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", " File \"c:\\Users\\kochu\\projects\\agents\\.venv\\Lib\\site-packages\\openai\\_base_client.py\", line 1037, in request\n", " raise self._make_status_error_from_response(err.response) from None\n", "openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'API key not valid. Please pass a valid API key.', 'status': 'INVALID_ARGUMENT', 'details': [{'@type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'API_KEY_INVALID', 'domain': 'googleapis.com', 'metadata': {'service': 'generativelanguage.googleapis.com'}}, {'@type': 'type.googleapis.com/google.rpc.LocalizedMessage', 'locale': 'en-US', 'message': 'API key not valid. Please pass a valid API key.'}]}}]\n" ] } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 2 }