2_lab2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to the Second Lab - Week 1, Day 3\n",
    "\n",
    "Today we will work with lots of models! This is a way to get comfortable with APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Important point - please read</h2>\n",
    "            <span style=\"color:#ff7800;\">The way I collaborate with you may be different to other courses you've taken. I prefer not to type code while you watch. Rather, I execute Jupyter Labs, like this, and give you an intuition for what's going on. My suggestion is that you carefully execute this yourself, <b>after</b> watching the lecture. Add print statements to understand what's going on, and then come up with your own variations.<br/><br/>If you have time, I'd love it if you submit a PR for changes in the community_contributions folder - instructions in the resources. Also, if you have a Github account, use this to showcase your variations. Not only is this essential practice, but it demonstrates your skills to others, including perhaps future clients or employers...\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start with imports - ask ChatGPT to explain any package that you don't know\n",
    "\n",
    "import os\n",
    "import json\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from anthropic import Anthropic\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Always remember to do this!\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API Key exists and begins sk-proj-\n",
      "Anthropic API Key not set (and this is optional)\n",
      "Google API Key not set (and this is optional)\n",
      "DeepSeek API Key not set (and this is optional)\n",
      "Groq API Key not set (and this is optional)\n"
     ]
    }
   ],
   "source": [
    "# Print the key prefixes to help with any debugging\n",
    "\n",
    "openai_api_key = os.getenv('OPENAI_API_KEY')\n",
    "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
    "google_api_key = os.getenv('GOOGLE_API_KEY')\n",
    "deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')\n",
    "groq_api_key = os.getenv('GROQ_API_KEY')\n",
    "\n",
    "if openai_api_key:\n",
    "    print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"OpenAI API Key not set\")\n",
    "    \n",
    "if anthropic_api_key:\n",
    "    print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
    "else:\n",
    "    print(\"Anthropic API Key not set (and this is optional)\")\n",
    "\n",
    "if google_api_key:\n",
    "    print(f\"Google API Key exists and begins {google_api_key[:2]}\")\n",
    "else:\n",
    "    print(\"Google API Key not set (and this is optional)\")\n",
    "\n",
    "if deepseek_api_key:\n",
    "    print(f\"DeepSeek API Key exists and begins {deepseek_api_key[:3]}\")\n",
    "else:\n",
    "    print(\"DeepSeek API Key not set (and this is optional)\")\n",
    "\n",
    "if groq_api_key:\n",
    "    print(f\"Groq API Key exists and begins {groq_api_key[:4]}\")\n",
    "else:\n",
    "    print(\"Groq API Key not set (and this is optional)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "request = \"Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. \"\n",
    "request += \"Answer only with the question, no explanation.\"\n",
    "messages = [{\"role\": \"user\", \"content\": request}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'role': 'user',\n",
       "  'content': 'Please come up with a challenging, nuanced question that I can ask a number of LLMs to evaluate their intelligence. Answer only with the question, no explanation.'}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "If you could rewrite a historical event with three significant changes and predict the outcomes of those changes, what event would you choose, what changes would you make, and how would those changes alter the course of history?\n"
     ]
    }
   ],
   "source": [
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"gpt-4o-mini\",\n",
    "    messages=messages,\n",
    ")\n",
    "question = response.choices[0].message.content\n",
    "print(question)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "competitors = []\n",
    "answers = []\n",
    "messages = [{\"role\": \"user\", \"content\": question}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "One intriguing historical event to consider rewriting is the American Revolutionary War (1775-1783). Here are three significant changes I would make, along with predictions on how they might alter the course of history:\n",
       "\n",
       "### 1. Successful Negotiation for Greater Autonomy:\n",
       "**Change:** Instead of escalating tensions leading to war, King George III and the British Parliament agree to grant the American colonies a significant degree of autonomy and a form of self-governance while still remaining under British rule. This could resemble a federal system similar to the modern United Kingdom.\n",
       "\n",
       "**Outcome:** By addressing colonial grievances through political reform rather than military conflict, tensions would diffuse, and the colonies might remain loyal to Britain while developing their own legislative bodies. This new system could lead to an earlier expansion of democratic practices and a more gradual integration of ideas about individual rights, potentially influencing Britain toward reforming its own governance sooner.\n",
       "\n",
       "### 2. The Enlightenment Influences Military Strategy:\n",
       "**Change:** Influential Enlightenment thinkers, such as Thomas Paine and John Locke, become key military strategists, emphasizing psychological warfare, diplomacy, and strategic alliances with foreign powers, such as France and Spain, from the outset of colonial resistance.\n",
       "\n",
       "**Outcome:** By focusing on partnerships and leveraging global sentiments against British imperialism, the colonies could form a coalition with other countries sooner, leading to increased military support and resources. This could not only prolong the war but also shift the nature of colonial independence movements, inspiring other nations to pursue their own paths of liberty sooner.\n",
       "\n",
       "### 3. Conditional Treaty with Britain:\n",
       "**Change:** At the end of the Revolutionary War, instead of a complete separation, the Treaty of Paris allows for an \"interim period\" where the United States remains under British protection but gains significant self-governing privileges, including rules around trade and local governance. \n",
       "\n",
       "**Outcome:** This compromise could lead to a gradual cultural and political evolution where the U.S. functions similarly to Canada today, with a strong British influence on its legal systems and governance. It might delay the full realization of independence for the U.S. but create a smoother path for eventual nationhood without the baggage of a violent revolution, potentially averting future conflicts such as the War of 1812 or Civil War.\n",
       "\n",
       "### Overall Impact on World History:\n",
       "These changes could result in a more unified British Empire, maintaining stronger ties with the colonies and delaying or shaping the struggle for independence in other territories. The ideals of democracy, liberty, and national identity could spread more organically, leading to a more collaborative international community and possibly altering the timeline of global decolonization movements throughout the 19th and 20th centuries. The Industrial Revolution might also unfold differently as economic relations between the colonies and Britain could foster earlier industrial development in North America, changing the economic trajectories of both the U.S. and Britain."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# The API we know well\n",
    "\n",
    "model_name = \"gpt-4o-mini\"\n",
    "\n",
    "response = openai.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Anthropic has a slightly different API, and Max Tokens is required\n",
    "\n",
    "model_name = \"claude-3-7-sonnet-latest\"\n",
    "\n",
    "claude = Anthropic()\n",
    "response = claude.messages.create(model=model_name, messages=messages, max_tokens=1000)\n",
    "answer = response.content[0].text\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gemini = OpenAI(api_key=google_api_key, base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\")\n",
    "model_name = \"gemini-2.0-flash\"\n",
    "\n",
    "response = gemini.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deepseek = OpenAI(api_key=deepseek_api_key, base_url=\"https://api.deepseek.com/v1\")\n",
    "model_name = \"deepseek-chat\"\n",
    "\n",
    "response = deepseek.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groq = OpenAI(api_key=groq_api_key, base_url=\"https://api.groq.com/openai/v1\")\n",
    "model_name = \"llama-3.3-70b-versatile\"\n",
    "\n",
    "response = groq.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## For the next cell, we will use Ollama\n",
    "\n",
    "Ollama runs a local web service that gives an OpenAI compatible endpoint,  \n",
    "and runs models locally using high performance C++ code.\n",
    "\n",
    "If you don't have Ollama, install it here by visiting https://ollama.com then pressing Download and following the instructions.\n",
    "\n",
    "After it's installed, you should be able to visit here: http://localhost:11434 and see the message \"Ollama is running\"\n",
    "\n",
    "You might need to restart Cursor (and maybe reboot). Then open a Terminal (control+\\`) and run `ollama serve`\n",
    "\n",
    "Useful Ollama commands (run these in the terminal, or with an exclamation mark in this notebook):\n",
    "\n",
    "`ollama pull <model_name>` downloads a model locally  \n",
    "`ollama ls` lists all the models you've downloaded  \n",
    "`ollama rm <model_name>` deletes the specified model from your downloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/stop.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Super important - ignore me at your peril!</h2>\n",
    "            <span style=\"color:#ff7800;\">The model called <b>llama3.3</b> is FAR too large for home computers - it's not intended for personal computing and will consume all your resources! Stick with the nicely sized <b>llama3.2</b> or <b>llama3.2:1b</b> and if you want larger, try llama3.1 or smaller variants of Qwen, Gemma, Phi or DeepSeek. See the <A href=\"https://ollama.com/models\">the Ollama models page</a> for a full list of models and sizes.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ollama pull llama3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ollama = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
    "model_name = \"llama3.2\"\n",
    "\n",
    "response = ollama.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['gpt-4o-mini']\n",
      "['One intriguing historical event to consider rewriting is the American Revolutionary War (1775-1783). Here are three significant changes I would make, along with predictions on how they might alter the course of history:\\n\\n### 1. Successful Negotiation for Greater Autonomy:\\n**Change:** Instead of escalating tensions leading to war, King George III and the British Parliament agree to grant the American colonies a significant degree of autonomy and a form of self-governance while still remaining under British rule. This could resemble a federal system similar to the modern United Kingdom.\\n\\n**Outcome:** By addressing colonial grievances through political reform rather than military conflict, tensions would diffuse, and the colonies might remain loyal to Britain while developing their own legislative bodies. This new system could lead to an earlier expansion of democratic practices and a more gradual integration of ideas about individual rights, potentially influencing Britain toward reforming its own governance sooner.\\n\\n### 2. The Enlightenment Influences Military Strategy:\\n**Change:** Influential Enlightenment thinkers, such as Thomas Paine and John Locke, become key military strategists, emphasizing psychological warfare, diplomacy, and strategic alliances with foreign powers, such as France and Spain, from the outset of colonial resistance.\\n\\n**Outcome:** By focusing on partnerships and leveraging global sentiments against British imperialism, the colonies could form a coalition with other countries sooner, leading to increased military support and resources. This could not only prolong the war but also shift the nature of colonial independence movements, inspiring other nations to pursue their own paths of liberty sooner.\\n\\n### 3. Conditional Treaty with Britain:\\n**Change:** At the end of the Revolutionary War, instead of a complete separation, the Treaty of Paris allows for an \"interim period\" where the United States remains under British protection but gains significant self-governing privileges, including rules around trade and local governance. \\n\\n**Outcome:** This compromise could lead to a gradual cultural and political evolution where the U.S. functions similarly to Canada today, with a strong British influence on its legal systems and governance. It might delay the full realization of independence for the U.S. but create a smoother path for eventual nationhood without the baggage of a violent revolution, potentially averting future conflicts such as the War of 1812 or Civil War.\\n\\n### Overall Impact on World History:\\nThese changes could result in a more unified British Empire, maintaining stronger ties with the colonies and delaying or shaping the struggle for independence in other territories. The ideals of democracy, liberty, and national identity could spread more organically, leading to a more collaborative international community and possibly altering the timeline of global decolonization movements throughout the 19th and 20th centuries. The Industrial Revolution might also unfold differently as economic relations between the colonies and Britain could foster earlier industrial development in North America, changing the economic trajectories of both the U.S. and Britain.']\n"
     ]
    }
   ],
   "source": [
    "# So where are we?\n",
    "\n",
    "print(competitors)\n",
    "print(answers)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Competitor: gpt-4o-mini\n",
      "\n",
      "One intriguing historical event to consider rewriting is the American Revolutionary War (1775-1783). Here are three significant changes I would make, along with predictions on how they might alter the course of history:\n",
      "\n",
      "### 1. Successful Negotiation for Greater Autonomy:\n",
      "**Change:** Instead of escalating tensions leading to war, King George III and the British Parliament agree to grant the American colonies a significant degree of autonomy and a form of self-governance while still remaining under British rule. This could resemble a federal system similar to the modern United Kingdom.\n",
      "\n",
      "**Outcome:** By addressing colonial grievances through political reform rather than military conflict, tensions would diffuse, and the colonies might remain loyal to Britain while developing their own legislative bodies. This new system could lead to an earlier expansion of democratic practices and a more gradual integration of ideas about individual rights, potentially influencing Britain toward reforming its own governance sooner.\n",
      "\n",
      "### 2. The Enlightenment Influences Military Strategy:\n",
      "**Change:** Influential Enlightenment thinkers, such as Thomas Paine and John Locke, become key military strategists, emphasizing psychological warfare, diplomacy, and strategic alliances with foreign powers, such as France and Spain, from the outset of colonial resistance.\n",
      "\n",
      "**Outcome:** By focusing on partnerships and leveraging global sentiments against British imperialism, the colonies could form a coalition with other countries sooner, leading to increased military support and resources. This could not only prolong the war but also shift the nature of colonial independence movements, inspiring other nations to pursue their own paths of liberty sooner.\n",
      "\n",
      "### 3. Conditional Treaty with Britain:\n",
      "**Change:** At the end of the Revolutionary War, instead of a complete separation, the Treaty of Paris allows for an \"interim period\" where the United States remains under British protection but gains significant self-governing privileges, including rules around trade and local governance. \n",
      "\n",
      "**Outcome:** This compromise could lead to a gradual cultural and political evolution where the U.S. functions similarly to Canada today, with a strong British influence on its legal systems and governance. It might delay the full realization of independence for the U.S. but create a smoother path for eventual nationhood without the baggage of a violent revolution, potentially averting future conflicts such as the War of 1812 or Civil War.\n",
      "\n",
      "### Overall Impact on World History:\n",
      "These changes could result in a more unified British Empire, maintaining stronger ties with the colonies and delaying or shaping the struggle for independence in other territories. The ideals of democracy, liberty, and national identity could spread more organically, leading to a more collaborative international community and possibly altering the timeline of global decolonization movements throughout the 19th and 20th centuries. The Industrial Revolution might also unfold differently as economic relations between the colonies and Britain could foster earlier industrial development in North America, changing the economic trajectories of both the U.S. and Britain.\n"
     ]
    }
   ],
   "source": [
    "# It's nice to know how to use \"zip\"\n",
    "for competitor, answer in zip(competitors, answers):\n",
    "    print(f\"Competitor: {competitor}\\n\\n{answer}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's bring this together - note the use of \"enumerate\"\n",
    "\n",
    "together = \"\"\n",
    "for index, answer in enumerate(answers):\n",
    "    together += f\"# Response from competitor {index+1}\\n\\n\"\n",
    "    together += answer + \"\\n\\n\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Response from competitor 1\n",
      "\n",
      "One intriguing historical event to consider rewriting is the American Revolutionary War (1775-1783). Here are three significant changes I would make, along with predictions on how they might alter the course of history:\n",
      "\n",
      "### 1. Successful Negotiation for Greater Autonomy:\n",
      "**Change:** Instead of escalating tensions leading to war, King George III and the British Parliament agree to grant the American colonies a significant degree of autonomy and a form of self-governance while still remaining under British rule. This could resemble a federal system similar to the modern United Kingdom.\n",
      "\n",
      "**Outcome:** By addressing colonial grievances through political reform rather than military conflict, tensions would diffuse, and the colonies might remain loyal to Britain while developing their own legislative bodies. This new system could lead to an earlier expansion of democratic practices and a more gradual integration of ideas about individual rights, potentially influencing Britain toward reforming its own governance sooner.\n",
      "\n",
      "### 2. The Enlightenment Influences Military Strategy:\n",
      "**Change:** Influential Enlightenment thinkers, such as Thomas Paine and John Locke, become key military strategists, emphasizing psychological warfare, diplomacy, and strategic alliances with foreign powers, such as France and Spain, from the outset of colonial resistance.\n",
      "\n",
      "**Outcome:** By focusing on partnerships and leveraging global sentiments against British imperialism, the colonies could form a coalition with other countries sooner, leading to increased military support and resources. This could not only prolong the war but also shift the nature of colonial independence movements, inspiring other nations to pursue their own paths of liberty sooner.\n",
      "\n",
      "### 3. Conditional Treaty with Britain:\n",
      "**Change:** At the end of the Revolutionary War, instead of a complete separation, the Treaty of Paris allows for an \"interim period\" where the United States remains under British protection but gains significant self-governing privileges, including rules around trade and local governance. \n",
      "\n",
      "**Outcome:** This compromise could lead to a gradual cultural and political evolution where the U.S. functions similarly to Canada today, with a strong British influence on its legal systems and governance. It might delay the full realization of independence for the U.S. but create a smoother path for eventual nationhood without the baggage of a violent revolution, potentially averting future conflicts such as the War of 1812 or Civil War.\n",
      "\n",
      "### Overall Impact on World History:\n",
      "These changes could result in a more unified British Empire, maintaining stronger ties with the colonies and delaying or shaping the struggle for independence in other territories. The ideals of democracy, liberty, and national identity could spread more organically, leading to a more collaborative international community and possibly altering the timeline of global decolonization movements throughout the 19th and 20th centuries. The Industrial Revolution might also unfold differently as economic relations between the colonies and Britain could foster earlier industrial development in North America, changing the economic trajectories of both the U.S. and Britain.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(together)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge = f\"\"\"You are judging a competition between {len(competitors)} competitors.\n",
    "Each model has been given this question:\n",
    "\n",
    "{question}\n",
    "\n",
    "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
    "Respond with JSON, and only JSON, with the following format:\n",
    "{{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}}\n",
    "\n",
    "Here are the responses from each competitor:\n",
    "\n",
    "{together}\n",
    "\n",
    "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are judging a competition between 1 competitors.\n",
      "Each model has been given this question:\n",
      "\n",
      "If you could rewrite a historical event with three significant changes and predict the outcomes of those changes, what event would you choose, what changes would you make, and how would those changes alter the course of history?\n",
      "\n",
      "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
      "Respond with JSON, and only JSON, with the following format:\n",
      "{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}\n",
      "\n",
      "Here are the responses from each competitor:\n",
      "\n",
      "# Response from competitor 1\n",
      "\n",
      "One intriguing historical event to consider rewriting is the American Revolutionary War (1775-1783). Here are three significant changes I would make, along with predictions on how they might alter the course of history:\n",
      "\n",
      "### 1. Successful Negotiation for Greater Autonomy:\n",
      "**Change:** Instead of escalating tensions leading to war, King George III and the British Parliament agree to grant the American colonies a significant degree of autonomy and a form of self-governance while still remaining under British rule. This could resemble a federal system similar to the modern United Kingdom.\n",
      "\n",
      "**Outcome:** By addressing colonial grievances through political reform rather than military conflict, tensions would diffuse, and the colonies might remain loyal to Britain while developing their own legislative bodies. This new system could lead to an earlier expansion of democratic practices and a more gradual integration of ideas about individual rights, potentially influencing Britain toward reforming its own governance sooner.\n",
      "\n",
      "### 2. The Enlightenment Influences Military Strategy:\n",
      "**Change:** Influential Enlightenment thinkers, such as Thomas Paine and John Locke, become key military strategists, emphasizing psychological warfare, diplomacy, and strategic alliances with foreign powers, such as France and Spain, from the outset of colonial resistance.\n",
      "\n",
      "**Outcome:** By focusing on partnerships and leveraging global sentiments against British imperialism, the colonies could form a coalition with other countries sooner, leading to increased military support and resources. This could not only prolong the war but also shift the nature of colonial independence movements, inspiring other nations to pursue their own paths of liberty sooner.\n",
      "\n",
      "### 3. Conditional Treaty with Britain:\n",
      "**Change:** At the end of the Revolutionary War, instead of a complete separation, the Treaty of Paris allows for an \"interim period\" where the United States remains under British protection but gains significant self-governing privileges, including rules around trade and local governance. \n",
      "\n",
      "**Outcome:** This compromise could lead to a gradual cultural and political evolution where the U.S. functions similarly to Canada today, with a strong British influence on its legal systems and governance. It might delay the full realization of independence for the U.S. but create a smoother path for eventual nationhood without the baggage of a violent revolution, potentially averting future conflicts such as the War of 1812 or Civil War.\n",
      "\n",
      "### Overall Impact on World History:\n",
      "These changes could result in a more unified British Empire, maintaining stronger ties with the colonies and delaying or shaping the struggle for independence in other territories. The ideals of democracy, liberty, and national identity could spread more organically, leading to a more collaborative international community and possibly altering the timeline of global decolonization movements throughout the 19th and 20th centuries. The Industrial Revolution might also unfold differently as economic relations between the colonies and Britain could foster earlier industrial development in North America, changing the economic trajectories of both the U.S. and Britain.\n",
      "\n",
      "\n",
      "\n",
      "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\n"
     ]
    }
   ],
   "source": [
    "print(judge)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "judge_messages = [{\"role\": \"user\", \"content\": judge}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"results\": [\"1\"]}\n"
     ]
    }
   ],
   "source": [
    "# Judgement time!\n",
    "\n",
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"o3-mini\",\n",
    "    messages=judge_messages,\n",
    ")\n",
    "results = response.choices[0].message.content\n",
    "print(results)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# OK let's turn this into results!\n",
    "\n",
    "results_dict = json.loads(results)\n",
    "ranks = results_dict[\"results\"]\n",
    "for index, result in enumerate(ranks):\n",
    "    competitor = competitors[int(result)-1]\n",
    "    print(f\"Rank {index+1}: {competitor}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/exercise.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#ff7800;\">Exercise</h2>\n",
    "            <span style=\"color:#ff7800;\">Which pattern(s) did this use? Try updating this to add another Agentic design pattern.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/business.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Commercial implications</h2>\n",
    "            <span style=\"color:#00bfff;\">These kinds of patterns - to send a task to multiple models, and evaluate results,\n",
    "            are common where you need to improve the quality of your LLM response. This approach can be universally applied\n",
    "            to business projects where accuracy is critical.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}