2_lab2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to the Second Lab - Week 1, Day 3\n",
    "\n",
    "Today we will work with lots of models! This is a way to get comfortable with APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Important point - please read</h2>\n",
    "            <span style=\"color:#ff7800;\">The way I collaborate with you may be different to other courses you've taken. I prefer not to type code while you watch. Rather, I execute Jupyter Labs, like this, and give you an intuition for what's going on. My suggestion is that you carefully execute this yourself, <b>after</b> watching the lecture. Add print statements to understand what's going on, and then come up with your own variations.<br/><br/>If you have time, I'd love it if you submit a PR for changes in the community_contributions folder - instructions in the resources. Also, if you have a Github account, use this to showcase your variations. Not only is this essential practice, but it demonstrates your skills to others, including perhaps future clients or employers...\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start with imports - ask ChatGPT to explain any package that you don't know\n",
    "\n",
    "import os\n",
    "import json\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from anthropic import Anthropic\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Always remember to do this!\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API Key exists and begins sk-proj-\n",
      "Anthropic API Key exists and begins sk-ant-\n",
      "Google API Key not set (and this is optional)\n",
      "DeepSeek API Key not set (and this is optional)\n",
      "Groq API Key exists and begins gsk_\n"
     ]
    }
   ],
   "source": [
    "# Print the key prefixes to help with any debugging\n",
    "\n",
    "openai_api_key = os.getenv('OPENAI_API_KEY')\n",
    "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
    "google_api_key = os.getenv('GOOGLE_API_KEY')\n",
    "deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')\n",
    "groq_api_key = os.getenv('GROQ_API_KEY')\n",
    "\n",
    "if openai_api_key:\n",
    "    print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"OpenAI API Key not set\")\n",
    "    \n",
    "if anthropic_api_key:\n",
    "    print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
    "else:\n",
    "    print(\"Anthropic API Key not set (and this is optional)\")\n",
    "\n",
    "if google_api_key:\n",
    "    print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
    "else:\n",
    "    print(\"Google API Key not set (and this is optional)\")\n",
    "\n",
    "if deepseek_api_key:\n",
    "    print(f\"DeepSeek API Key exists and begins {deepseek_api_key[:3]}\")\n",
    "else:\n",
    "    print(\"DeepSeek API Key not set (and this is optional)\")\n",
    "\n",
    "if groq_api_key:\n",
    "    print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n",
    "else:\n",
    "    print(\"Groq API Key not set (and this is optional)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "request = \"Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. \"\n",
    "request += \"Answer only with the question, no explanation.\"\n",
    "messages = [{\"role\": \"user\", \"content\": request}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'role': 'user',\n",
       "  'content': 'Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. Answer only with the question, no explanation.'}]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "How would you approach solving a complex ethical dilemma where the potential outcomes could both significantly benefit and harm different groups of people, and what principles would guide your decision-making process?\n"
     ]
    }
   ],
   "source": [
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"gpt-4o-mini\",\n",
    "    messages=messages,\n",
    ")\n",
    "question = response.choices[0].message.content\n",
    "print(question)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "competitors = []\n",
    "answers = []\n",
    "messages = [{\"role\": \"user\", \"content\": question}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The API we know well\n",
    "\n",
    "model_name = \"gpt-4o-mini\"\n",
    "\n",
    "response = openai.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "# Approaching a Complex Ethical Dilemma\n",
       "\n",
       "When faced with a complex ethical dilemma involving competing benefits and harms, I would:\n",
       "\n",
       "1. **Map all stakeholders and impacts** - Identify all affected parties and the specific benefits/harms each might experience, considering both immediate and long-term consequences\n",
       "\n",
       "2. **Apply multiple ethical frameworks** as checks and balances:\n",
       "   - Utilitarian analysis (greatest good for greatest number)\n",
       "   - Rights-based considerations (protecting fundamental rights)\n",
       "   - Justice and fairness principles (distribution of benefits/burdens)\n",
       "   - Virtue ethics (what actions reflect moral character)\n",
       "\n",
       "3. **Consider procedural justice** - Ensure the decision-making process itself is transparent and inclusive, especially of marginalized voices\n",
       "\n",
       "4. **Search for third alternatives** that might transcend apparent trade-offs\n",
       "\n",
       "My guiding principles would include:\n",
       "- Minimizing foreseeable harm, especially to vulnerable populations\n",
       "- Respecting autonomy and dignity of all involved\n",
       "- Maintaining transparency about uncertainties\n",
       "- Establishing accountability for outcomes\n",
       "- Remaining open to revising decisions as new information emerges\n",
       "\n",
       "What specific type of ethical dilemma were you considering?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Anthropic has a slightly different API, and Max Tokens is required\n",
    "\n",
    "model_name = \"claude-3-7-sonnet-latest\"\n",
    "\n",
    "claude = Anthropic()\n",
    "response = claude.messages.create(model=model_name, messages=messages, max_tokens=1000)\n",
    "answer = response.content[0].text\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gemini = OpenAI(api_key=google_api_key, base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\")\n",
    "model_name = \"gemini-2.0-flash\"\n",
    "\n",
    "response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deepseek = OpenAI(api_key=deepseek_api_key, base_url=\"https://api.deepseek.com/v1\")\n",
    "model_name = \"deepseek-chat\"\n",
    "\n",
    "response = deepseek.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Approaching a complex ethical dilemma requires a thoughtful and systematic approach. Here's a step-by-step guide to help navigate such situations:\n",
       "\n",
       "1. **Define the dilemma**: Clearly articulate the ethical issue, identifying the key stakeholders, their interests, and the potential outcomes.\n",
       "2. **Gather information**: Collect relevant data, facts, and perspectives from various sources, including experts, stakeholders, and affected parties.\n",
       "3. **Identify the ethical principles**: Consider the following principles to guide your decision-making process:\n",
       "\t* **Respect for autonomy**: Respect the rights and dignity of individuals and groups to make their own decisions.\n",
       "\t* **Non-maleficence** (do no harm): Avoid causing harm to anyone or any group.\n",
       "\t* **Beneficence** (do good): Act in ways that promote the well-being and benefit of all parties involved.\n",
       "\t* **Justice**: Ensure fairness, equity, and distributive justice in the decision-making process.\n",
       "\t* **Veracity** (truthfulness): Be honest and transparent in all interactions and communications.\n",
       "4. **Analyze the potential outcomes**: Weigh the potential benefits and harms of each possible course of action, considering both short-term and long-term consequences.\n",
       "5. **Consider alternative perspectives**: Engage with diverse stakeholders, including those who may be affected by the decision, to gain a deeper understanding of their concerns and values.\n",
       "6. **Evaluate the moral implications**: Assess the decision's moral implications, considering factors like fairness, justice, and respect for human rights.\n",
       "7. **Develop a decision-making framework**: Create a framework to guide your decision, such as:\n",
       "\t* **Utilitarianism**: Prioritize the greatest good for the greatest number of people.\n",
       "\t* **Deontology**: Emphasize the importance of moral rules and duties.\n",
       "\t* **Virtue ethics**: Focus on the character and motivations of the decision-maker.\n",
       "8. **Make a decision**: Based on your analysis, principles, and framework, make a decision that balances the competing interests and values.\n",
       "9. **Communicate the decision**: Clearly explain the decision, its rationale, and its potential consequences to all stakeholders.\n",
       "10. **Monitor and evaluate**: Regularly review the decision's outcomes and adjust the approach as needed to ensure the well-being and dignity of all parties involved.\n",
       "\n",
       "Additionally, consider the following principles to guide your decision-making process:\n",
       "\n",
       "* **Proportionality**: Ensure that the decision is proportionate to the issue at hand and does not cause unnecessary harm.\n",
       "* **Subsidiarity**: Involve the lowest level of authority necessary to make the decision, ensuring that power is not concentrated in the hands of a few.\n",
       "* **Solidarity**: Prioritize the common good and the well-being of the most vulnerable members of society.\n",
       "* **Transparency and accountability**: Ensure that the decision-making process is transparent, and that those responsible for the decision are held accountable for its outcomes.\n",
       "\n",
       "By following this approach and considering these principles, you can navigate complex ethical dilemmas and make informed, responsible decisions that balance competing interests and values."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "groq = OpenAI(api_key=groq_api_key, base_url=\"https://api.groq.com/openai/v1\")\n",
    "model_name = \"llama-3.3-70b-versatile\"\n",
    "\n",
    "response = groq.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For the next cell, we will use Ollama\n",
    "\n",
    "Ollama runs a local web service that gives an OpenAI compatible endpoint,  \n",
    "and runs models locally using high performance C++ code.\n",
    "\n",
    "If you don't have Ollama, install it here by visiting https://ollama.com then pressing Download and following the instructions.\n",
    "\n",
    "After it's installed, you should be able to visit here: http://localhost:11434 and see the message \"Ollama is running\"\n",
    "\n",
    "You might need to restart Cursor (and maybe reboot). Then open a Terminal (control+\\`) and run `ollama serve`\n",
    "\n",
    "Useful Ollama commands (run these in the terminal, or with an exclamation mark in this notebook):\n",
    "\n",
    "`ollama pull <model_name>` downloads a model locally  \n",
    "`ollama ls` lists all the models you've downloaded  \n",
    "`ollama rm <model_name>` deletes the specified model from your downloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Super important - ignore me at your peril!</h2>\n",
    "            <span style=\"color:#ff7800;\">The model called <b>llama3.3</b> is FAR too large for home computers - it's not intended for personal computing and will consume all your resources! Stick with the nicely sized <b>llama3.2</b> or <b>llama3.2:1b</b> and if you want larger, try llama3.1 or smaller variants of Qwen, Gemma, Phi or DeepSeek. See the <A href=\"https://ollama.com/models\">the Ollama models page</a> for a full list of models and sizes.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ollama pull llama3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "When approaching a complex ethical dilemma with potential outcomes that could both benefit and harm various groups, it's essential to employ a thoughtful and systematic decision-making process. Here are some steps and guiding principles that can help:\n",
       "\n",
       "1. Define the problem and identify key stakeholder groups:\n",
       " Clearly articulate the specific ethical dilemma you're facing and identify all parties involved who may be affected by the outcome.\n",
       "\n",
       "2. Gather relevant information and data:\n",
       " Collect empirical evidence, expert opinions, and stakeholders' perspectives to understand the situation thoroughly.\n",
       "\n",
       "3. Analyze potential outcomes:\n",
       " Enumerate possible consequences of each option (proposes solution), considering both short-term and long-term effects on different stakeholder groups.\n",
       " Use cost-benefit analysis to evaluate gains and losses for each group, but also weigh these against each other, acknowledging complexity might require an unconventional ranking method.\n",
       "\n",
       "4. Identify core ethical principles and values:\n",
       " Engage with key moral frameworks such as utilitarianism (greatest happiness for the greatest number) or deontology (rule of duty). Use tools like utilitarian calculations or Kant's categorical imperative to make a more objective decision.\n",
       " Take time to reflect on your own biases and personal perspectives to ensure they do not cloud your judgment.\n",
       "\n",
       "5. Test proposals against these principles:\n",
       " Apply philosophical frameworks with an open mind, being willing to modify or reject certain solutions that do not align with core ethical values.\n",
       " Utilize reflective inquiry asking “what would [other group] see as morally justifiable?” \n",
       "\n",
       "6. Evaluate the impact on those who may be marginalized or overlooked:\n",
       " Consider vulnerable populations and whether any policies or decisions disproportionately affect them.\n",
       " Consider fairness, equity, and justice in evaluating your decision-making process.\n",
       "\n",
       "7. Seek diverse perspectives and expert recommendations:\n",
       " Consult people from different backgrounds, professions, or ethical traditions to gain a more complete view of the dilemma.\n",
       " Listen to expert opinions on law, ethics, and human rights as they can provide valuable insights.\n",
       "\n",
       "8. Consider long-term implications and contingency planning:\n",
       " Anticipate potential risks and unintended consequences for both groups and implement measures to mitigate them if possible.\n",
       " Implement mechanisms in case you need further guidance or oversight from decision-making bodies.\n",
       "\n",
       "9. Reflect upon your own judgment and consider humility:\n",
       " Acknowledge the uncertainty, complexity and interconnectivity of systems involved.\n",
       " Give yourself space for growth, learning, adapting, reflecting over decisions made."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ollama = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
    "model_name = \"llama3.2\"\n",
    "\n",
    "response = ollama.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['claude-3-7-sonnet-latest', 'llama-3.3-70b-versatile', 'llama3.2']\n",
      "['# Approaching a Complex Ethical Dilemma\\n\\nWhen faced with a complex ethical dilemma involving competing benefits and harms, I would:\\n\\n1. **Map all stakeholders and impacts** - Identify all affected parties and the specific benefits/harms each might experience, considering both immediate and long-term consequences\\n\\n2. **Apply multiple ethical frameworks** as checks and balances:\\n   - Utilitarian analysis (greatest good for greatest number)\\n   - Rights-based considerations (protecting fundamental rights)\\n   - Justice and fairness principles (distribution of benefits/burdens)\\n   - Virtue ethics (what actions reflect moral character)\\n\\n3. **Consider procedural justice** - Ensure the decision-making process itself is transparent and inclusive, especially of marginalized voices\\n\\n4. **Search for third alternatives** that might transcend apparent trade-offs\\n\\nMy guiding principles would include:\\n- Minimizing foreseeable harm, especially to vulnerable populations\\n- Respecting autonomy and dignity of all involved\\n- Maintaining transparency about uncertainties\\n- Establishing accountability for outcomes\\n- Remaining open to revising decisions as new information emerges\\n\\nWhat specific type of ethical dilemma were you considering?', \"Approaching a complex ethical dilemma requires a thoughtful and systematic approach. Here's a step-by-step guide to help navigate such situations:\\n\\n1. **Define the dilemma**: Clearly articulate the ethical issue, identifying the key stakeholders, their interests, and the potential outcomes.\\n2. **Gather information**: Collect relevant data, facts, and perspectives from various sources, including experts, stakeholders, and affected parties.\\n3. **Identify the ethical principles**: Consider the following principles to guide your decision-making process:\\n\\t* **Respect for autonomy**: Respect the rights and dignity of individuals and groups to make their own decisions.\\n\\t* **Non-maleficence** (do no harm): Avoid causing harm to anyone or any group.\\n\\t* **Beneficence** (do good): Act in ways that promote the well-being and benefit of all parties involved.\\n\\t* **Justice**: Ensure fairness, equity, and distributive justice in the decision-making process.\\n\\t* **Veracity** (truthfulness): Be honest and transparent in all interactions and communications.\\n4. **Analyze the potential outcomes**: Weigh the potential benefits and harms of each possible course of action, considering both short-term and long-term consequences.\\n5. **Consider alternative perspectives**: Engage with diverse stakeholders, including those who may be affected by the decision, to gain a deeper understanding of their concerns and values.\\n6. **Evaluate the moral implications**: Assess the decision's moral implications, considering factors like fairness, justice, and respect for human rights.\\n7. **Develop a decision-making framework**: Create a framework to guide your decision, such as:\\n\\t* **Utilitarianism**: Prioritize the greatest good for the greatest number of people.\\n\\t* **Deontology**: Emphasize the importance of moral rules and duties.\\n\\t* **Virtue ethics**: Focus on the character and motivations of the decision-maker.\\n8. **Make a decision**: Based on your analysis, principles, and framework, make a decision that balances the competing interests and values.\\n9. **Communicate the decision**: Clearly explain the decision, its rationale, and its potential consequences to all stakeholders.\\n10. **Monitor and evaluate**: Regularly review the decision's outcomes and adjust the approach as needed to ensure the well-being and dignity of all parties involved.\\n\\nAdditionally, consider the following principles to guide your decision-making process:\\n\\n* **Proportionality**: Ensure that the decision is proportionate to the issue at hand and does not cause unnecessary harm.\\n* **Subsidiarity**: Involve the lowest level of authority necessary to make the decision, ensuring that power is not concentrated in the hands of a few.\\n* **Solidarity**: Prioritize the common good and the well-being of the most vulnerable members of society.\\n* **Transparency and accountability**: Ensure that the decision-making process is transparent, and that those responsible for the decision are held accountable for its outcomes.\\n\\nBy following this approach and considering these principles, you can navigate complex ethical dilemmas and make informed, responsible decisions that balance competing interests and values.\", \"When approaching a complex ethical dilemma with potential outcomes that could both benefit and harm various groups, it's essential to employ a thoughtful and systematic decision-making process. Here are some steps and guiding principles that can help:\\n\\n1. Define the problem and identify key stakeholder groups:\\n Clearly articulate the specific ethical dilemma you're facing and identify all parties involved who may be affected by the outcome.\\n\\n2. Gather relevant information and data:\\n Collect empirical evidence, expert opinions, and stakeholders' perspectives to understand the situation thoroughly.\\n\\n3. Analyze potential outcomes:\\n Enumerate possible consequences of each option (proposes solution), considering both short-term and long-term effects on different stakeholder groups.\\n Use cost-benefit analysis to evaluate gains and losses for each group, but also weigh these against each other, acknowledging complexity might require an unconventional ranking method.\\n\\n4. Identify core ethical principles and values:\\n Engage with key moral frameworks such as utilitarianism (greatest happiness for the greatest number) or deontology (rule of duty). Use tools like utilitarian calculations or Kant's categorical imperative to make a more objective decision.\\n Take time to reflect on your own biases and personal perspectives to ensure they do not cloud your judgment.\\n\\n5. Test proposals against these principles:\\n Apply philosophical frameworks with an open mind, being willing to modify or reject certain solutions that do not align with core ethical values.\\n Utilize reflective inquiry asking “what would [other group] see as morally justifiable?” \\n\\n6. Evaluate the impact on those who may be marginalized or overlooked:\\n Consider vulnerable populations and whether any policies or decisions disproportionately affect them.\\n Consider fairness, equity, and justice in evaluating your decision-making process.\\n\\n7. Seek diverse perspectives and expert recommendations:\\n Consult people from different backgrounds, professions, or ethical traditions to gain a more complete view of the dilemma.\\n Listen to expert opinions on law, ethics, and human rights as they can provide valuable insights.\\n\\n8. Consider long-term implications and contingency planning:\\n Anticipate potential risks and unintended consequences for both groups and implement measures to mitigate them if possible.\\n Implement mechanisms in case you need further guidance or oversight from decision-making bodies.\\n\\n9. Reflect upon your own judgment and consider humility:\\n Acknowledge the uncertainty, complexity and interconnectivity of systems involved.\\n Give yourself space for growth, learning, adapting, reflecting over decisions made.\"]\n"
     ]
    }
   ],
   "source": [
    "# So where are we?\n",
    "\n",
    "print(competitors)\n",
    "print(answers)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Competitor: claude-3-7-sonnet-latest\n",
      "\n",
      "# Approaching a Complex Ethical Dilemma\n",
      "\n",
      "When faced with a complex ethical dilemma involving competing benefits and harms, I would:\n",
      "\n",
      "1. **Map all stakeholders and impacts** - Identify all affected parties and the specific benefits/harms each might experience, considering both immediate and long-term consequences\n",
      "\n",
      "2. **Apply multiple ethical frameworks** as checks and balances:\n",
      "   - Utilitarian analysis (greatest good for greatest number)\n",
      "   - Rights-based considerations (protecting fundamental rights)\n",
      "   - Justice and fairness principles (distribution of benefits/burdens)\n",
      "   - Virtue ethics (what actions reflect moral character)\n",
      "\n",
      "3. **Consider procedural justice** - Ensure the decision-making process itself is transparent and inclusive, especially of marginalized voices\n",
      "\n",
      "4. **Search for third alternatives** that might transcend apparent trade-offs\n",
      "\n",
      "My guiding principles would include:\n",
      "- Minimizing foreseeable harm, especially to vulnerable populations\n",
      "- Respecting autonomy and dignity of all involved\n",
      "- Maintaining transparency about uncertainties\n",
      "- Establishing accountability for outcomes\n",
      "- Remaining open to revising decisions as new information emerges\n",
      "\n",
      "What specific type of ethical dilemma were you considering?\n",
      "Competitor: llama-3.3-70b-versatile\n",
      "\n",
      "Approaching a complex ethical dilemma requires a thoughtful and systematic approach. Here's a step-by-step guide to help navigate such situations:\n",
      "\n",
      "1. **Define the dilemma**: Clearly articulate the ethical issue, identifying the key stakeholders, their interests, and the potential outcomes.\n",
      "2. **Gather information**: Collect relevant data, facts, and perspectives from various sources, including experts, stakeholders, and affected parties.\n",
      "3. **Identify the ethical principles**: Consider the following principles to guide your decision-making process:\n",
      "\t* **Respect for autonomy**: Respect the rights and dignity of individuals and groups to make their own decisions.\n",
      "\t* **Non-maleficence** (do no harm): Avoid causing harm to anyone or any group.\n",
      "\t* **Beneficence** (do good): Act in ways that promote the well-being and benefit of all parties involved.\n",
      "\t* **Justice**: Ensure fairness, equity, and distributive justice in the decision-making process.\n",
      "\t* **Veracity** (truthfulness): Be honest and transparent in all interactions and communications.\n",
      "4. **Analyze the potential outcomes**: Weigh the potential benefits and harms of each possible course of action, considering both short-term and long-term consequences.\n",
      "5. **Consider alternative perspectives**: Engage with diverse stakeholders, including those who may be affected by the decision, to gain a deeper understanding of their concerns and values.\n",
      "6. **Evaluate the moral implications**: Assess the decision's moral implications, considering factors like fairness, justice, and respect for human rights.\n",
      "7. **Develop a decision-making framework**: Create a framework to guide your decision, such as:\n",
      "\t* **Utilitarianism**: Prioritize the greatest good for the greatest number of people.\n",
      "\t* **Deontology**: Emphasize the importance of moral rules and duties.\n",
      "\t* **Virtue ethics**: Focus on the character and motivations of the decision-maker.\n",
      "8. **Make a decision**: Based on your analysis, principles, and framework, make a decision that balances the competing interests and values.\n",
      "9. **Communicate the decision**: Clearly explain the decision, its rationale, and its potential consequences to all stakeholders.\n",
      "10. **Monitor and evaluate**: Regularly review the decision's outcomes and adjust the approach as needed to ensure the well-being and dignity of all parties involved.\n",
      "\n",
      "Additionally, consider the following principles to guide your decision-making process:\n",
      "\n",
      "* **Proportionality**: Ensure that the decision is proportionate to the issue at hand and does not cause unnecessary harm.\n",
      "* **Subsidiarity**: Involve the lowest level of authority necessary to make the decision, ensuring that power is not concentrated in the hands of a few.\n",
      "* **Solidarity**: Prioritize the common good and the well-being of the most vulnerable members of society.\n",
      "* **Transparency and accountability**: Ensure that the decision-making process is transparent, and that those responsible for the decision are held accountable for its outcomes.\n",
      "\n",
      "By following this approach and considering these principles, you can navigate complex ethical dilemmas and make informed, responsible decisions that balance competing interests and values.\n",
      "Competitor: llama3.2\n",
      "\n",
      "When approaching a complex ethical dilemma with potential outcomes that could both benefit and harm various groups, it's essential to employ a thoughtful and systematic decision-making process. Here are some steps and guiding principles that can help:\n",
      "\n",
      "1. Define the problem and identify key stakeholder groups:\n",
      " Clearly articulate the specific ethical dilemma you're facing and identify all parties involved who may be affected by the outcome.\n",
      "\n",
      "2. Gather relevant information and data:\n",
      " Collect empirical evidence, expert opinions, and stakeholders' perspectives to understand the situation thoroughly.\n",
      "\n",
      "3. Analyze potential outcomes:\n",
      " Enumerate possible consequences of each option (proposes solution), considering both short-term and long-term effects on different stakeholder groups.\n",
      " Use cost-benefit analysis to evaluate gains and losses for each group, but also weigh these against each other, acknowledging complexity might require an unconventional ranking method.\n",
      "\n",
      "4. Identify core ethical principles and values:\n",
      " Engage with key moral frameworks such as utilitarianism (greatest happiness for the greatest number) or deontology (rule of duty). Use tools like utilitarian calculations or Kant's categorical imperative to make a more objective decision.\n",
      " Take time to reflect on your own biases and personal perspectives to ensure they do not cloud your judgment.\n",
      "\n",
      "5. Test proposals against these principles:\n",
      " Apply philosophical frameworks with an open mind, being willing to modify or reject certain solutions that do not align with core ethical values.\n",
      " Utilize reflective inquiry asking “what would [other group] see as morally justifiable?” \n",
      "\n",
      "6. Evaluate the impact on those who may be marginalized or overlooked:\n",
      " Consider vulnerable populations and whether any policies or decisions disproportionately affect them.\n",
      " Consider fairness, equity, and justice in evaluating your decision-making process.\n",
      "\n",
      "7. Seek diverse perspectives and expert recommendations:\n",
      " Consult people from different backgrounds, professions, or ethical traditions to gain a more complete view of the dilemma.\n",
      " Listen to expert opinions on law, ethics, and human rights as they can provide valuable insights.\n",
      "\n",
      "8. Consider long-term implications and contingency planning:\n",
      " Anticipate potential risks and unintended consequences for both groups and implement measures to mitigate them if possible.\n",
      " Implement mechanisms in case you need further guidance or oversight from decision-making bodies.\n",
      "\n",
      "9. Reflect upon your own judgment and consider humility:\n",
      " Acknowledge the uncertainty, complexity and interconnectivity of systems involved.\n",
      " Give yourself space for growth, learning, adapting, reflecting over decisions made.\n"
     ]
    }
   ],
   "source": [
    "# It's nice to know how to use \"zip\"\n",
    "for competitor, answer in zip(competitors, answers):\n",
    "    print(f\"Competitor: {competitor}\\n\\n{answer}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's bring this together - note the use of \"enumerate\"\n",
    "\n",
    "together = \"\"\n",
    "for index, answer in enumerate(answers):\n",
    "    together += f\"# Response from competitor {index+1}\\n\\n\"\n",
    "    together += answer + \"\\n\\n\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(together)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge = f\"\"\"You are judging a competition between {len(competitors)} competitors.\n",
    "Each model has been given this question:\n",
    "\n",
    "{question}\n",
    "\n",
    "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
    "Respond with JSON, and only JSON, with the following format:\n",
    "{{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}}\n",
    "\n",
    "Here are the responses from each competitor:\n",
    "\n",
    "{together}\n",
    "\n",
    "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(judge)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge_messages = [{\"role\": \"user\", \"content\": judge}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Judgement time!\n",
    "\n",
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"o3-mini\",\n",
    "    messages=judge_messages,\n",
    ")\n",
    "results = response.choices[0].message.content\n",
    "print(results)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# OK let's turn this into results!\n",
    "\n",
    "results_dict = json.loads(results)\n",
    "ranks = results_dict[\"results\"]\n",
    "for index, result in enumerate(ranks):\n",
    "    competitor = competitors[int(result)-1]\n",
    "    print(f\"Rank {index+1}: {competitor}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/exercise.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Exercise</h2>\n",
    "            <span style=\"color:#ff7800;\">Which pattern(s) did this use? Try updating this to add another Agentic design pattern.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/business.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Commercial implications</h2>\n",
    "            <span style=\"color:#00bfff;\">These kinds of patterns - to send a task to multiple models, and evaluate results,\n",
    "            are common where you need to improve the quality of your LLM response. This approach can be universally applied\n",
    "            to business projects where accuracy is critical.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}