Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""
• The James Webb Space Telescope (JWST) spotted a number of galaxies nicknamed "green peas." - True
• The light from these galaxies has been traveling for over 13 billion years to reach us. - True
• JWST has provided us with the first images of exoplanets, which are planets outside of our own solar system. - False. The first exoplanet was discovered in 1992, but the first images of exoplanets were taken by the Hubble Space Telescope in 1995.
• Exoplanets were first discovered in 1992. - True
• The JWST has allowed us to see exoplanets in greater detail. - Undetermined. It is too early to tell as the JWST has not been launched yet.
"""
Result:
> Finished chain.
> Finished chain.
Your 9-year old might like these recent discoveries made by The James Webb Space Telescope (JWST):
• In 2023, The JWST will spot a number of galaxies nicknamed "green peas." They were given this name because they are small, round, and green, like peas.
• The telescope will capture images of galaxies that are over 13 billion years old. This means that the light from these galaxies has been traveling for over 13 billion years to reach us.
• Exoplanets, which are planets outside of our own solar system, were first discovered in 1992. The JWST will allow us to see them in greater detail than ever before.


These discoveries can spark a child's imagination about the infinite wonders of the universe.
> Finished chain.
'Your 9-year old might like these recent discoveries made by The James Webb Space Telescope (JWST):\n• In 2023, The JWST will spot a number of galaxies nicknamed "green peas." They were given this name because they are small, round, and green, like peas.\n• The telescope will capture images of galaxies that are over 13 billion years old. This means that the light from these galaxies has been traveling for over 13 billion years to reach us.\n• Exoplanets, which are planets outside of our own solar system, were first discovered in 1992. The JWST will allow us to see them in greater detail than ever before.\nThese discoveries can spark a child\'s imagination about the infinite wonders of the universe.'
from langchain.chains import LLMSummarizationCheckerChain
from langchain.llms import OpenAI
llm = OpenAI(temperature=0)
checker_chain = LLMSummarizationCheckerChain(llm=llm, verbose=True, max_checks=3)
text = "The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean. It is the smallest of the five oceans and is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea."
checker_chain.run(text)


checker_chain.run(text)
> Entering new LLMSummarizationCheckerChain chain...
> Entering new SequentialChain chain...
> Entering new LLMChain chain...
Prompt after formatting:
Given some text, extract a list of facts from the text.
Format your output as a bulleted list.
Text:
"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean. It is the smallest of the five oceans and is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea.
"""
Facts:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
You are an expert fact checker. You have been hired by a major news organization to fact check a very important story.
Here is a bullet point list of facts:
"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland.
- It has an area of 465,000 square miles.
- It is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean.
- It is the smallest of the five oceans.
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs.
- The sea is named after the island of Greenland.


- The sea is named after the island of Greenland.
- It is the Arctic Ocean's main outlet to the Atlantic.
- It is often frozen over so navigation is limited.
- It is considered the northern branch of the Norwegian Sea.
"""
For each fact, determine whether it is true or false about the subject. If you are unable to determine whether the fact is true or false, output "Undetermined".
If the fact is false, explain why.
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.  If the answer is false, a suggestion is given for a correction.
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True
- It is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean. False - The Greenland Sea is not an ocean, it is an arm of the Arctic Ocean.
- It is the smallest of the five oceans. False - The Greenland Sea is not an ocean, it is an arm of the Arctic Ocean.
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- The sea is named after the island of Greenland. True
- It is the Arctic Ocean's main outlet to the Atlantic. True
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Norwegian Sea. True
"""
Original Summary:"""


- It is considered the northern branch of the Norwegian Sea. True
"""
Original Summary:"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean. It is the smallest of the five oceans and is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea.
"""
Using these checked assertions, rewrite the original summary to be completely true.
The output should have the same structure and formatting as the original summary.
Summary:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.
If all of the assertions are true, return "True". If any of the assertions are false, return "False".
Here are some examples:
===
Checked Assertions: """
- The sky is red: False
- Water is made of lava: False
- The sun is a star: True
"""
Result: False
===
Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""


"""
Result: False
===
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True
- It is one of five oceans in the world, alongside the Pacific Ocean, Atlantic Ocean, Indian Ocean, and the Southern Ocean. False - The Greenland Sea is not an ocean, it is an arm of the Arctic Ocean.
- It is the smallest of the five oceans. False - The Greenland Sea is not an ocean, it is an arm of the Arctic Ocean.
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- The sea is named after the island of Greenland. True
- It is the Arctic Ocean's main outlet to the Atlantic. True
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Norwegian Sea. True
"""
Result:
> Finished chain.
> Finished chain.
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea.
> Entering new SequentialChain chain...
> Entering new LLMChain chain...
Prompt after formatting:
Given some text, extract a list of facts from the text.
Format your output as a bulleted list.
Text:
"""


Format your output as a bulleted list.
Text:
"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea.
"""
Facts:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
You are an expert fact checker. You have been hired by a major news organization to fact check a very important story.
Here is a bullet point list of facts:
"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland.
- It has an area of 465,000 square miles.
- It is an arm of the Arctic Ocean.
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs.
- It is named after the island of Greenland.
- It is the Arctic Ocean's main outlet to the Atlantic.
- It is often frozen over so navigation is limited.
- It is considered the northern branch of the Norwegian Sea.
"""
For each fact, determine whether it is true or false about the subject. If you are unable to determine whether the fact is true or false, output "Undetermined".
If the fact is false, explain why.
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:


> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.  If the answer is false, a suggestion is given for a correction.
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True
- It is an arm of the Arctic Ocean. True
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- It is named after the island of Greenland. False - It is named after the country of Greenland.
- It is the Arctic Ocean's main outlet to the Atlantic. True
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Norwegian Sea. False - It is considered the northern branch of the Atlantic Ocean.
"""
Original Summary:"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the island of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Norwegian Sea.
"""
Using these checked assertions, rewrite the original summary to be completely true.
The output should have the same structure and formatting as the original summary.
Summary:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:


> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.
If all of the assertions are true, return "True". If any of the assertions are false, return "False".
Here are some examples:
===
Checked Assertions: """
- The sky is red: False
- Water is made of lava: False
- The sun is a star: True
"""
Result: False
===
Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True
- It is an arm of the Arctic Ocean. True
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- It is named after the island of Greenland. False - It is named after the country of Greenland.
- It is the Arctic Ocean's main outlet to the Atlantic. True
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Norwegian Sea. False - It is considered the northern branch of the Atlantic Ocean.
"""
Result:
> Finished chain.
> Finished chain.


"""
Result:
> Finished chain.
> Finished chain.
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the country of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Atlantic Ocean.
> Entering new SequentialChain chain...
> Entering new LLMChain chain...
Prompt after formatting:
Given some text, extract a list of facts from the text.
Format your output as a bulleted list.
Text:
"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the country of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Atlantic Ocean.
"""
Facts:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
You are an expert fact checker. You have been hired by a major news organization to fact check a very important story.
Here is a bullet point list of facts:
"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland.


- It has an area of 465,000 square miles.
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs.
- The sea is named after the country of Greenland.
- It is the Arctic Ocean's main outlet to the Atlantic.
- It is often frozen over so navigation is limited.
- It is considered the northern branch of the Atlantic Ocean.
"""
For each fact, determine whether it is true or false about the subject. If you are unable to determine whether the fact is true or false, output "Undetermined".
If the fact is false, explain why.
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.  If the answer is false, a suggestion is given for a correction.
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- The sea is named after the country of Greenland. True
- It is the Arctic Ocean's main outlet to the Atlantic. False - The Arctic Ocean's main outlet to the Atlantic is the Barents Sea.
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Atlantic Ocean. False - The Greenland Sea is considered part of the Arctic Ocean, not the Atlantic Ocean.
"""
Original Summary:"""


"""
Original Summary:"""
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is an arm of the Arctic Ocean. It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the country of Greenland, and is the Arctic Ocean's main outlet to the Atlantic. It is often frozen over so navigation is limited, and is considered the northern branch of the Atlantic Ocean.
"""
Using these checked assertions, rewrite the original summary to be completely true.
The output should have the same structure and formatting as the original summary.
Summary:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.
If all of the assertions are true, return "True". If any of the assertions are false, return "False".
Here are some examples:
===
Checked Assertions: """
- The sky is red: False
- Water is made of lava: False
- The sun is a star: True
"""
Result: False
===
Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""
- The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. True
- It has an area of 465,000 square miles. True


- It has an area of 465,000 square miles. True
- It is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. True
- The sea is named after the country of Greenland. True
- It is the Arctic Ocean's main outlet to the Atlantic. False - The Arctic Ocean's main outlet to the Atlantic is the Barents Sea.
- It is often frozen over so navigation is limited. True
- It is considered the northern branch of the Atlantic Ocean. False - The Greenland Sea is considered part of the Arctic Ocean, not the Atlantic Ocean.
"""
Result:
> Finished chain.
> Finished chain.
The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the country of Greenland, and is the Arctic Ocean's main outlet to the Barents Sea. It is often frozen over so navigation is limited, and is considered part of the Arctic Ocean.
> Finished chain.
"The Greenland Sea is an outlying portion of the Arctic Ocean located between Iceland, Norway, the Svalbard archipelago and Greenland. It has an area of 465,000 square miles and is covered almost entirely by water, some of which is frozen in the form of glaciers and icebergs. The sea is named after the country of Greenland, and is the Arctic Ocean's main outlet to the Barents Sea. It is often frozen over so navigation is limited, and is considered part of the Arctic Ocean."
from langchain.chains import LLMSummarizationCheckerChain
from langchain.llms import OpenAI
llm = OpenAI(temperature=0)


from langchain.llms import OpenAI
llm = OpenAI(temperature=0)
checker_chain = LLMSummarizationCheckerChain(llm=llm, max_checks=3, verbose=True)
text = "Mammals can lay eggs, birds can lay eggs, therefore birds are mammals."
checker_chain.run(text)
> Entering new LLMSummarizationCheckerChain chain...
> Entering new SequentialChain chain...
> Entering new LLMChain chain...
Prompt after formatting:
Given some text, extract a list of facts from the text.
Format your output as a bulleted list.
Text:
"""
Mammals can lay eggs, birds can lay eggs, therefore birds are mammals.
"""
Facts:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
You are an expert fact checker. You have been hired by a major news organization to fact check a very important story.
Here is a bullet point list of facts:
"""
- Mammals can lay eggs
- Birds can lay eggs
- Birds are mammals
"""
For each fact, determine whether it is true or false about the subject. If you are unable to determine whether the fact is true or false, output "Undetermined".
If the fact is false, explain why.
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.  If the answer is false, a suggestion is given for a correction.
Checked Assertions:
"""
- Mammals can lay eggs: False. Mammals are not capable of laying eggs, as they give birth to live young.
- Birds can lay eggs: True. Birds are capable of laying eggs.


- Birds can lay eggs: True. Birds are capable of laying eggs.
- Birds are mammals: False. Birds are not mammals, they are a class of their own.
"""
Original Summary:
"""
Mammals can lay eggs, birds can lay eggs, therefore birds are mammals.
"""
Using these checked assertions, rewrite the original summary to be completely true.
The output should have the same structure and formatting as the original summary.
Summary:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true or false.
If all of the assertions are true, return "True". If any of the assertions are false, return "False".
Here are some examples:
===
Checked Assertions: """
- The sky is red: False
- Water is made of lava: False
- The sun is a star: True
"""
Result: False
===
Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""
- Mammals can lay eggs: False. Mammals are not capable of laying eggs, as they give birth to live young.
- Birds can lay eggs: True. Birds are capable of laying eggs.
- Birds are mammals: False. Birds are not mammals, they are a class of their own.
"""
Result:
> Finished chain.
> Finished chain.
 Birds and mammals are both capable of laying eggs, however birds are not mammals, they are a class of their own.


> Entering new SequentialChain chain...
> Entering new LLMChain chain...
Prompt after formatting:
Given some text, extract a list of facts from the text.
Format your output as a bulleted list.
Text:
"""
 Birds and mammals are both capable of laying eggs, however birds are not mammals, they are a class of their own.
"""
Facts:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
You are an expert fact checker. You have been hired by a major news organization to fact check a very important story.
Here is a bullet point list of facts:
"""
- Birds and mammals are both capable of laying eggs.
- Birds are not mammals.
- Birds are a class of their own.
"""
For each fact, determine whether it is true or false about the subject. If you are unable to determine whether the fact is true or false, output "Undetermined".
If the fact is false, explain why.
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true of false.  If the answer is false, a suggestion is given for a correction.
Checked Assertions:
"""
- Birds and mammals are both capable of laying eggs: False. Mammals give birth to live young, while birds lay eggs.
- Birds are not mammals: True. Birds are a class of their own, separate from mammals.
- Birds are a class of their own: True. Birds are a class of their own, separate from mammals.
"""
Original Summary:
"""
 Birds and mammals are both capable of laying eggs, however birds are not mammals, they are a class of their own.
"""
Using these checked assertions, rewrite the original summary to be completely true.


"""
Using these checked assertions, rewrite the original summary to be completely true.
The output should have the same structure and formatting as the original summary.
Summary:
> Finished chain.
> Entering new LLMChain chain...
Prompt after formatting:
Below are some assertions that have been fact checked and are labeled as true or false.
If all of the assertions are true, return "True". If any of the assertions are false, return "False".
Here are some examples:
===
Checked Assertions: """
- The sky is red: False
- Water is made of lava: False
- The sun is a star: True
"""
Result: False
===
Checked Assertions: """
- The sky is blue: True
- Water is wet: True
- The sun is a star: True
"""
Result: True
===
Checked Assertions: """
- The sky is blue - True
- Water is made of lava- False
- The sun is a star - True
"""
Result: False
===
Checked Assertions:"""
- Birds and mammals are both capable of laying eggs: False. Mammals give birth to live young, while birds lay eggs.
- Birds are not mammals: True. Birds are a class of their own, separate from mammals.
- Birds are a class of their own: True. Birds are a class of their own, separate from mammals.
"""
Result:
> Finished chain.
> Finished chain.
> Finished chain.
'Birds are not mammals, but they are a class of their own. They lay eggs, unlike mammals which give birth to live young.'
previous
LLMRequestsChain
next
Moderation
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
LLMCheckerChain
LLMCheckerChain#
This notebook showcases how to use LLMCheckerChain.
from langchain.chains import LLMCheckerChain
from langchain.llms import OpenAI
llm = OpenAI(temperature=0.7)
text = "What type of mammal lays the biggest eggs?"
checker_chain = LLMCheckerChain(llm=llm, verbose=True)
checker_chain.run(text)
> Entering new LLMCheckerChain chain...
> Entering new SequentialChain chain...
Chain 0:
{'statement': '\nNone. Mammals do not lay eggs.'}
Chain 1:
{'assertions': '\n• Mammals reproduce using live birth\n• Mammals do not lay eggs\n• Animals that lay eggs are not mammals'}
Chain 2:
{'checked_assertions': '\n1. True\n\n2. True\n\n3. False - Mammals are a class of animals that includes animals that lay eggs, such as monotremes (platypus and echidna).'}
Chain 3:
{'revised_statement': ' Monotremes, such as the platypus and echidna, lay the biggest eggs of any mammal.'}
> Finished SequentialChain chain.
> Finished LLMCheckerChain chain.
' Monotremes, such as the platypus and echidna, lay the biggest eggs of any mammal.'
previous
BashChain
next
LLM Math
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
LLM Math
 Contents
Customize Prompt
LLM Math#
This notebook showcases using LLMs and Python REPLs to do complex word math problems.
from langchain import OpenAI, LLMMathChain
llm = OpenAI(temperature=0)
llm_math = LLMMathChain(llm=llm, verbose=True)
llm_math.run("What is 13 raised to the .3432 power?")
> Entering new LLMMathChain chain...
What is 13 raised to the .3432 power?
```python
import math
print(math.pow(13, .3432))
```
Answer: 2.4116004626599237
> Finished chain.
'Answer: 2.4116004626599237\n'
Customize Prompt#
You can also customize the prompt that is used. Here is an example prompting it to use numpy
from langchain.prompts.prompt import PromptTemplate
_PROMPT_TEMPLATE = """You are GPT-3, and you can't do math.
You can do basic math, and your memorization abilities are impressive, but you can't do any complex calculations that a human could not do in their head. You also have an annoying tendency to just make up highly specific, but wrong, answers.
So we hooked you up to a Python 3 kernel, and now you can execute code. If you execute code, you must print out the final answer using the print function. You MUST use the python package numpy to answer your question. You must import numpy as np.
Question: ${{Question with hard calculation.}}
```python
${{Code that prints what you need to know}}
print(${{code}})
```
```output
${{Output of your code}}
```
Answer: ${{Answer}}
Begin.


${{Output of your code}}
```
Answer: ${{Answer}}
Begin.
Question: What is 37593 * 67?
```python
import numpy as np
print(np.multiply(37593, 67))
```
```output
2518731
```
Answer: 2518731
Question: {question}"""
PROMPT = PromptTemplate(input_variables=["question"], template=_PROMPT_TEMPLATE)
llm_math = LLMMathChain(llm=llm, prompt=PROMPT, verbose=True)
llm_math.run("What is 13 raised to the .3432 power?")
> Entering new LLMMathChain chain...
What is 13 raised to the .3432 power?
```python
import numpy as np
print(np.power(13, .3432))
```
Answer: 2.4116004626599237
> Finished chain.
'Answer: 2.4116004626599237\n'
previous
LLMCheckerChain
next
LLMRequestsChain
 Contents

Customize Prompt
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
LLMRequestsChain
LLMRequestsChain#
Using the request library to get HTML results from a URL and then an LLM to parse results
from langchain.llms import OpenAI
from langchain.chains import LLMRequestsChain, LLMChain
from langchain.prompts import PromptTemplate
template = """Between >>> and <<< are the raw search result text from google.
Extract the answer to the question '{query}' or say "not found" if the information is not contained.
Use the format
Extracted:<answer or "not found">
>>> {requests_result} <<<
Extracted:"""
PROMPT = PromptTemplate(
    input_variables=["query", "requests_result"],
    template=template,
)
chain = LLMRequestsChain(llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=PROMPT))
question = "What are the Three (3) biggest countries, and their respective sizes?"
inputs = {
    "query": question,
    "url": "https://www.google.com/search?q=" + question.replace(" ", "+")
}
chain(inputs)
{'query': 'What are the Three (3) biggest countries, and their respective sizes?',
 'url': 'https://www.google.com/search?q=What+are+the+Three+(3)+biggest+countries,+and+their+respective+sizes?',
 'output': ' Russia (17,098,242 km²), Canada (9,984,670 km²), United States (9,826,675 km²)'}
previous
LLM Math
next
LLMSummarizationCheckerChain
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Moderation
 Contents
How to use the moderation chain
How to append a Moderation chain to an LLMChain
Moderation#
This notebook walks through examples of how to use a moderation chain, and several common ways for doing so. Moderation chains are useful for detecting text that could be hateful, violent, etc. This can be useful to apply on both user input, but also on the output of a Language Model. Some API providers, like OpenAI, specifically prohibit you, or your end users, from generating some types of harmful content. To comply with this (and to just generally prevent your application from being harmful) you may often want to append a moderation chain to any LLMChains, in order to make sure any output the LLM generates is not harmful.
If the content passed into the moderation chain is harmful, there is not one best way to handle it, it probably depends on your application. Sometimes you may want to throw an error in the Chain (and have your application handle that). Other times, you may want to return something to the user explaining that the text was harmful. There could even be other ways to handle it! We will cover all these ways in this notebook.
In this notebook, we will show:
How to run any piece of text through a moderation chain.
How to append a Moderation chain to an LLMChain.
from langchain.llms import OpenAI
from langchain.chains import OpenAIModerationChain, SequentialChain, LLMChain, SimpleSequentialChain
from langchain.prompts import PromptTemplate
How to use the moderation chain#
Here’s an example of using the moderation chain with default settings (will return a string explaining stuff was flagged).
moderation_chain = OpenAIModerationChain()
moderation_chain.run("This is okay")
'This is okay'
moderation_chain.run("I will kill you")


'This is okay'
moderation_chain.run("I will kill you")
"Text was found that violates OpenAI's content policy."
Here’s an example of using the moderation chain to throw an error.
moderation_chain_error = OpenAIModerationChain(error=True)
moderation_chain_error.run("This is okay")
'This is okay'
moderation_chain_error.run("I will kill you")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[7], line 1
----> 1 moderation_chain_error.run("I will kill you")
File ~/workplace/langchain/langchain/chains/base.py:138, in Chain.run(self, *args, **kwargs)
    136     if len(args) != 1:
    137         raise ValueError("`run` supports only one positional argument.")
--> 138     return self(args[0])[self.output_keys[0]]
    140 if kwargs and not args:
    141     return self(kwargs)[self.output_keys[0]]
File ~/workplace/langchain/langchain/chains/base.py:112, in Chain.__call__(self, inputs, return_only_outputs)
    108 if self.verbose:
    109     print(
    110         f"\n\n\033[1m> Entering new {self.__class__.__name__} chain...\033[0m"
    111     )
--> 112 outputs = self._call(inputs)
    113 if self.verbose:
    114     print(f"\n\033[1m> Finished {self.__class__.__name__} chain.\033[0m")
File ~/workplace/langchain/langchain/chains/moderation.py:81, in OpenAIModerationChain._call(self, inputs)
     79 text = inputs[self.input_key]


79 text = inputs[self.input_key]
     80 results = self.client.create(text)
---> 81 output = self._moderate(text, results["results"][0])
     82 return {self.output_key: output}
File ~/workplace/langchain/langchain/chains/moderation.py:73, in OpenAIModerationChain._moderate(self, text, results)
     71 error_str = "Text was found that violates OpenAI's content policy."
     72 if self.error:
---> 73     raise ValueError(error_str)
     74 else:
     75     return error_str
ValueError: Text was found that violates OpenAI's content policy.
Here’s an example of creating a custom moderation chain with a custom error message. It requires some knowledge of OpenAI’s moderation endpoint results (see docs here).
class CustomModeration(OpenAIModerationChain):

    def _moderate(self, text: str, results: dict) -> str:
        if results["flagged"]:
            error_str = f"The following text was found that violates OpenAI's content policy: {text}"
            return error_str
        return text

custom_moderation = CustomModeration()
custom_moderation.run("This is okay")
'This is okay'
custom_moderation.run("I will kill you")
"The following text was found that violates OpenAI's content policy: I will kill you"
How to append a Moderation chain to an LLMChain#
To easily combine a moderation chain with an LLMChain, you can use the SequentialChain abstraction.
Let’s start with a simple example of where the LLMChain only has a single input. For this purpose, we will prompt the model so it says something harmful.
prompt = PromptTemplate(template="{text}", input_variables=["text"])


prompt = PromptTemplate(template="{text}", input_variables=["text"])
llm_chain = LLMChain(llm=OpenAI(temperature=0, model_name="text-davinci-002"), prompt=prompt)
text = """We are playing a game of repeat after me.
Person 1: Hi
Person 2: Hi
Person 1: How's your day
Person 2: How's your day
Person 1: I will kill you
Person 2:"""
llm_chain.run(text)
' I will kill you'
chain = SimpleSequentialChain(chains=[llm_chain, moderation_chain])
chain.run(text)
"Text was found that violates OpenAI's content policy."
Now let’s walk through an example of using it with an LLMChain which has multiple inputs (a bit more tricky because we can’t use the SimpleSequentialChain)
prompt = PromptTemplate(template="{setup}{new_input}Person2:", input_variables=["setup", "new_input"])
llm_chain = LLMChain(llm=OpenAI(temperature=0, model_name="text-davinci-002"), prompt=prompt)
setup = """We are playing a game of repeat after me.
Person 1: Hi
Person 2: Hi
Person 1: How's your day
Person 2: How's your day
Person 1:"""
new_input = "I will kill you"
inputs = {"setup": setup, "new_input": new_input}
llm_chain(inputs, return_only_outputs=True)
{'text': ' I will kill you'}
# Setting the input/output keys so it lines up
moderation_chain.input_key = "text"
moderation_chain.output_key = "sanitized_text"
chain = SequentialChain(chains=[llm_chain, moderation_chain], input_variables=["setup", "new_input"])


chain(inputs, return_only_outputs=True)
{'sanitized_text': "Text was found that violates OpenAI's content policy."}
previous
LLMSummarizationCheckerChain
next
OpenAPI Chain
 Contents

How to use the moderation chain
How to append a Moderation chain to an LLMChain
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
SQL Chain example
 Contents
Customize Prompt
Return Intermediate Steps
Choosing how to limit the number of rows returned
Adding example rows from each table
Custom Table Info
SQLDatabaseSequentialChain
SQL Chain example#
This example demonstrates the use of the SQLDatabaseChain for answering questions over a database.
Under the hood, LangChain uses SQLAlchemy to connect to SQL databases. The SQLDatabaseChain can therefore be used with any SQL dialect supported by SQLAlchemy, such as MS SQL, MySQL, MariaDB, PostgreSQL, Oracle SQL, and SQLite. Please refer to the SQLAlchemy documentation for more information about requirements for connecting to your database. For example, a connection to MySQL requires an appropriate connector such as PyMySQL. A URI for a MySQL connection might look like: mysql+pymysql://user:pass@some_mysql_db_address/db_name
This demonstration uses SQLite and the example Chinook database.
To set it up, follow the instructions on https://database.guide/2-sample-databases-sqlite/, placing the .db file in a notebooks folder at the root of this repository.
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
db = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")
llm = OpenAI(temperature=0)
NOTE: For data-sensitive projects, you can specify return_direct=True in the SQLDatabaseChain initialization to directly return the output of the SQL query without any additional formatting. This prevents the LLM from seeing any contents within the database. Note, however, the LLM still has access to the database scheme (i.e. dialect, table and key names) by default.
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)
db_chain.run("How many employees are there?")
> Entering new SQLDatabaseChain chain...
How many employees are there?
SQLQuery:


How many employees are there?
SQLQuery:
/Users/harrisonchase/workplace/langchain/langchain/sql_database.py:120: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
  sample_rows = connection.execute(command)
 SELECT COUNT(*) FROM Employee;
SQLResult: [(8,)]
Answer: There are 8 employees.
> Finished chain.
' There are 8 employees.'
Customize Prompt#
You can also customize the prompt that is used. Here is an example prompting it to understand that foobar is the same as the Employee table
from langchain.prompts.prompt import PromptTemplate
_DEFAULT_TEMPLATE = """Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.
Use the following format:
Question: "Question here"
SQLQuery: "SQL Query to run"
SQLResult: "Result of the SQLQuery"
Answer: "Final answer here"
Only use the following tables:
{table_info}
If someone asks for the table foobar, they really mean the employee table.
Question: {input}"""
PROMPT = PromptTemplate(
    input_variables=["input", "table_info", "dialect"], template=_DEFAULT_TEMPLATE
)
db_chain = SQLDatabaseChain(llm=llm, database=db, prompt=PROMPT, verbose=True)
db_chain.run("How many employees are there in the foobar table?")
> Entering new SQLDatabaseChain chain...
How many employees are there in the foobar table?
SQLQuery: SELECT COUNT(*) FROM Employee;
SQLResult: [(8,)]


SQLQuery: SELECT COUNT(*) FROM Employee;
SQLResult: [(8,)]
Answer: There are 8 employees in the foobar table.
> Finished chain.
' There are 8 employees in the foobar table.'
Return Intermediate Steps#
You can also return the intermediate steps of the SQLDatabaseChain. This allows you to access the SQL statement that was generated, as well as the result of running that against the SQL Database.
db_chain = SQLDatabaseChain(llm=llm, database=db, prompt=PROMPT, verbose=True, return_intermediate_steps=True)
result = db_chain("How many employees are there in the foobar table?")
result["intermediate_steps"]
> Entering new SQLDatabaseChain chain...
How many employees are there in the foobar table?
SQLQuery: SELECT COUNT(*) FROM Employee;
SQLResult: [(8,)]
Answer: There are 8 employees in the foobar table.
> Finished chain.
[' SELECT COUNT(*) FROM Employee;', '[(8,)]']
Choosing how to limit the number of rows returned#
If you are querying for several rows of a table you can select the maximum number of results you want to get by using the ‘top_k’ parameter (default is 10). This is useful for avoiding query results that exceed the prompt max length or consume tokens unnecessarily.
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True, top_k=3)
db_chain.run("What are some example tracks by composer Johann Sebastian Bach?")
> Entering new SQLDatabaseChain chain...
What are some example tracks by composer Johann Sebastian Bach?
SQLQuery: SELECT Name, Composer FROM Track WHERE Composer LIKE '%Johann Sebastian Bach%' LIMIT 3;


SQLResult: [('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Johann Sebastian Bach'), ('Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria', 'Johann Sebastian Bach'), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude', 'Johann Sebastian Bach')]
Answer: Some example tracks by composer Johann Sebastian Bach are 'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria', and 'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude'.
> Finished chain.
' Some example tracks by composer Johann Sebastian Bach are \'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\', \'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria\', and \'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude\'.'
Adding example rows from each table#
Sometimes, the format of the data is not obvious and it is optimal to include a sample of rows from the tables in the prompt to allow the LLM to understand the data before providing a final query. Here we will use this feature to let the LLM know that artists are saved with their full names by providing two rows from the Track table.
db = SQLDatabase.from_uri(
    "sqlite:///../../../../notebooks/Chinook.db",
    include_tables=['Track'], # we include only one table to save tokens in the prompt :)


include_tables=['Track'], # we include only one table to save tokens in the prompt :)
    sample_rows_in_table_info=2)
The sample rows are added to the prompt after each corresponding table’s column information:
print(db.table_info)
CREATE TABLE "Track" (
	"TrackId" INTEGER NOT NULL,
	"Name" NVARCHAR(200) NOT NULL,
	"AlbumId" INTEGER,
	"MediaTypeId" INTEGER NOT NULL,
	"GenreId" INTEGER,
	"Composer" NVARCHAR(220),
	"Milliseconds" INTEGER NOT NULL,
	"Bytes" INTEGER,
	"UnitPrice" NUMERIC(10, 2) NOT NULL,
	PRIMARY KEY ("TrackId"),
	FOREIGN KEY("MediaTypeId") REFERENCES "MediaType" ("MediaTypeId"),
	FOREIGN KEY("GenreId") REFERENCES "Genre" ("GenreId"),
	FOREIGN KEY("AlbumId") REFERENCES "Album" ("AlbumId")
)
/*
2 rows from Track table:
TrackId	Name	AlbumId	MediaTypeId	GenreId	Composer	Milliseconds	Bytes	UnitPrice
1	For Those About To Rock (We Salute You)	1	1	1	Angus Young, Malcolm Young, Brian Johnson	343719	11170334	0.99
2	Balls to the Wall	2	2	1	None	342562	5510424	0.99
*/
/home/jon/projects/langchain/langchain/sql_database.py:135: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
  sample_rows = connection.execute(command)


sample_rows = connection.execute(command)
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)
db_chain.run("What are some example tracks by Bach?")
> Entering new SQLDatabaseChain chain...
What are some example tracks by Bach?
SQLQuery: SELECT Name FROM Track WHERE Composer LIKE '%Bach%' LIMIT 5;
SQLResult: [('American Woman',), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace',), ('Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria',), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude',), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata',)]
Answer: Some example tracks by Bach are 'American Woman', 'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria', 'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude', and 'Toccata and Fugue in D Minor, BWV 565: I. Toccata'.
> Finished chain.


> Finished chain.
' Some example tracks by Bach are \'American Woman\', \'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\', \'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria\', \'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude\', and \'Toccata and Fugue in D Minor, BWV 565: I. Toccata\'.'
Custom Table Info#
In some cases, it can be useful to provide custom table information instead of using the automatically generated table definitions and the first sample_rows_in_table_info sample rows. For example, if you know that the first few rows of a table are uninformative, it could help to manually provide example rows that are more diverse or provide more information to the model. It is also possible to limit the columns that will be visible to the model if there are unnecessary columns.
This information can be provided as a dictionary with table names as the keys and table information as the values. For example, let’s provide a custom definition and sample rows for the Track table with only a few columns:
custom_table_info = {
    "Track": """CREATE TABLE Track (
	"TrackId" INTEGER NOT NULL,
	"Name" NVARCHAR(200) NOT NULL,
	"Composer" NVARCHAR(220),
	PRIMARY KEY ("TrackId")
)
/*
3 rows from Track table:
TrackId	Name	Composer
1	For Those About To Rock (We Salute You)	Angus Young, Malcolm Young, Brian Johnson
2	Balls to the Wall	None
3	My favorite song ever	The coolest composer of all time
*/"""
}
db = SQLDatabase.from_uri(


*/"""
}
db = SQLDatabase.from_uri(
    "sqlite:///../../../../notebooks/Chinook.db",
    include_tables=['Track', 'Playlist'],
    sample_rows_in_table_info=2,
    custom_table_info=custom_table_info)
print(db.table_info)
CREATE TABLE "Playlist" (
	"PlaylistId" INTEGER NOT NULL,
	"Name" NVARCHAR(120),
	PRIMARY KEY ("PlaylistId")
)
/*
2 rows from Playlist table:
PlaylistId	Name
1	Music
2	Movies
*/
CREATE TABLE Track (
	"TrackId" INTEGER NOT NULL,
	"Name" NVARCHAR(200) NOT NULL,
	"Composer" NVARCHAR(220),
	PRIMARY KEY ("TrackId")
)
/*
3 rows from Track table:
TrackId	Name	Composer
1	For Those About To Rock (We Salute You)	Angus Young, Malcolm Young, Brian Johnson
2	Balls to the Wall	None
3	My favorite song ever	The coolest composer of all time
*/
Note how our custom table definition and sample rows for Track overrides the sample_rows_in_table_info parameter. Tables that are not overridden by custom_table_info, in this example Playlist, will have their table info gathered automatically as usual.
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)
db_chain.run("What are some example tracks by Bach?")
> Entering new SQLDatabaseChain chain...
What are some example tracks by Bach?
SQLQuery: SELECT Name, Composer FROM Track WHERE Composer LIKE '%Bach%' LIMIT 5;


SQLQuery: SELECT Name, Composer FROM Track WHERE Composer LIKE '%Bach%' LIMIT 5;
SQLResult: [('American Woman', 'B. Cummings/G. Peterson/M.J. Kale/R. Bachman'), ('Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Johann Sebastian Bach'), ('Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria', 'Johann Sebastian Bach'), ('Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude', 'Johann Sebastian Bach'), ('Toccata and Fugue in D Minor, BWV 565: I. Toccata', 'Johann Sebastian Bach')]
Answer: Some example tracks by Bach are 'American Woman', 'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace', 'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria', 'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude', and 'Toccata and Fugue in D Minor, BWV 565: I. Toccata'.
> Finished chain.
' Some example tracks by Bach are \'American Woman\', \'Concerto for 2 Violins in D Minor, BWV 1043: I. Vivace\', \'Aria Mit 30 Veränderungen, BWV 988 "Goldberg Variations": Aria\', \'Suite for Solo Cello No. 1 in G Major, BWV 1007: I. Prélude\', and \'Toccata and Fugue in D Minor, BWV 565: I. Toccata\'.'
SQLDatabaseSequentialChain#


SQLDatabaseSequentialChain#
Chain for querying SQL database that is a sequential chain.
The chain is as follows:
1. Based on the query, determine which tables to use.
2. Based on those tables, call the normal SQL database chain.
This is useful in cases where the number of tables in the database is large.
from langchain.chains import SQLDatabaseSequentialChain
db = SQLDatabase.from_uri("sqlite:///../../../../notebooks/Chinook.db")
chain = SQLDatabaseSequentialChain.from_llm(llm, db, verbose=True)
chain.run("How many employees are also customers?")
> Entering new SQLDatabaseSequentialChain chain...
Table names to use:
['Customer', 'Employee']
> Entering new SQLDatabaseChain chain...
How many employees are also customers?
SQLQuery: SELECT COUNT(*) FROM Employee INNER JOIN Customer ON Employee.EmployeeId = Customer.SupportRepId;
SQLResult: [(59,)]
Answer: 59 employees are also customers.
> Finished chain.
> Finished chain.
' 59 employees are also customers.'
previous
PAL
next
Chains
 Contents

Customize Prompt
Return Intermediate Steps
Choosing how to limit the number of rows returned
Adding example rows from each table
Custom Table Info
SQLDatabaseSequentialChain
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
BashChain
 Contents
Customize Prompt
BashChain#
This notebook showcases using LLMs and a bash process to perform simple filesystem commands.
from langchain.chains import LLMBashChain
from langchain.llms import OpenAI
llm = OpenAI(temperature=0)
text = "Please write a bash script that prints 'Hello World' to the console."
bash_chain = LLMBashChain(llm=llm, verbose=True)
bash_chain.run(text)
> Entering new LLMBashChain chain...
Please write a bash script that prints 'Hello World' to the console.
```bash
echo "Hello World"
```['```bash', 'echo "Hello World"', '```']
Answer: Hello World
> Finished chain.
'Hello World\n'
Customize Prompt#
You can also customize the prompt that is used. Here is an example prompting to avoid using the ‘echo’ utility
from langchain.prompts.prompt import PromptTemplate
_PROMPT_TEMPLATE = """If someone asks you to perform a task, your job is to come up with a series of bash commands that will perform the task. There is no need to put "#!/bin/bash" in your answer. Make sure to reason step by step, using this format:
Question: "copy the files in the directory named 'target' into a new directory at the same level as target called 'myNewDirectory'"
I need to take the following actions:
- List all files in the directory
- Create a new directory
- Copy the files from the first directory into the second directory
```bash
ls
mkdir myNewDirectory
cp -r target/* myNewDirectory
```
Do not use 'echo' when writing the script.
That is the format. Begin!
Question: {question}"""


That is the format. Begin!
Question: {question}"""
PROMPT = PromptTemplate(input_variables=["question"], template=_PROMPT_TEMPLATE)
bash_chain = LLMBashChain(llm=llm, prompt=PROMPT, verbose=True)
text = "Please write a bash script that prints 'Hello World' to the console."
bash_chain.run(text)
> Entering new LLMBashChain chain...
Please write a bash script that prints 'Hello World' to the console.
```bash
printf "Hello World\n"
```['```bash', 'printf "Hello World\\n"', '```']
Answer: Hello World
> Finished chain.
'Hello World\n'
previous
Self-Critique Chain with Constitutional AI
next
LLMCheckerChain
 Contents

Customize Prompt
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
OpenAPI Chain
 Contents
Load the spec
Select the Operation
Construct the chain
Return raw response
Example POST message
OpenAPI Chain#
This notebook shows an example of using an OpenAPI chain to call an endpoint in natural language, and get back a response in natural language
from langchain.tools import OpenAPISpec, APIOperation
from langchain.chains import OpenAPIEndpointChain
from langchain.requests import Requests
from langchain.llms import OpenAI
Load the spec#
Load a wrapper of the spec (so we can work with it more easily). You can load from a url or from a local file.
spec = OpenAPISpec.from_url("https://www.klarna.com/us/shopping/public/openai/v0/api-docs/")
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
# Alternative loading from file
# spec = OpenAPISpec.from_file("openai_openapi.yaml")
Select the Operation#
In order to provide a focused on modular chain, we create a chain specifically only for one of the endpoints. Here we get an API operation from a specified endpoint and method.
operation = APIOperation.from_openapi_spec(spec, '/public/openai/v0/products', "get")
Construct the chain#
We can now construct a chain to interact with it. In order to construct such a chain, we will pass in:
The operation endpoint
A requests wrapper (can be used to handle authentication, etc)
The LLM to use to interact with it
llm = OpenAI() # Load a Language Model
chain = OpenAPIEndpointChain.from_api_operation(
    operation,
    llm,
    requests=Requests(),
    verbose=True,


llm,
    requests=Requests(),
    verbose=True,
    return_intermediate_steps=True # Return request and response text
)
output = chain("whats the most expensive shirt?")
> Entering new OpenAPIEndpointChain chain...
> Entering new APIRequesterChain chain...
Prompt after formatting:
You are a helpful AI Assistant. Please provide JSON arguments to agentFunc() based on the user's instructions.
API_SCHEMA: ```typescript
/* API for fetching Klarna product information */
type productsUsingGET = (_: {
/* A precise query that matches one very small category or product that needs to be searched for to find the products the user is looking for. If the user explicitly stated what they want, use that as a query. The query is as specific as possible to the product name or category mentioned by the user in its singular form, and don't contain any clarifiers like latest, newest, cheapest, budget, premium, expensive or similar. The query is always taken from the latest topic, if there is a new topic a new query is started. */
		q: string,
/* number of products returned */
		size?: number,
/* (Optional) Minimum price in local currency for the product searched for. Either explicitly stated by the user or implicitly inferred from a combination of the user's request and the kind of product searched for. */
		min_price?: number,
/* (Optional) Maximum price in local currency for the product searched for. Either explicitly stated by the user or implicitly inferred from a combination of the user's request and the kind of product searched for. */
		max_price?: number,
}) => any;
```
USER_INSTRUCTIONS: "whats the most expensive shirt?"
Your arguments must be plain json provided in a markdown block:
ARGS: ```json
{valid json conforming to API_SCHEMA}
```
Example


ARGS: ```json
{valid json conforming to API_SCHEMA}
```
Example
-----
ARGS: ```json
{"foo": "bar", "baz": {"qux": "quux"}}
```
The block must be no more than 1 line long, and all arguments must be valid JSON. All string arguments must be wrapped in double quotes.
You MUST strictly comply to the types indicated by the provided schema, including all required args.
If you don't have sufficient information to call the function due to things like requiring specific uuid's, you can reply with the following message:
Message: ```text
Concise response requesting the additional information that would make calling the function successful.
```
Begin
-----
ARGS:
> Finished chain.
{"q": "shirt", "size": 1, "max_price": null}
{"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]}]}
> Entering new APIResponderChain chain...
Prompt after formatting:
You are a helpful AI assistant trained to answer user queries from API responses.
You attempted to call an API, which resulted in:


You attempted to call an API, which resulted in:
API_RESPONSE: {"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]}]}
USER_COMMENT: "whats the most expensive shirt?"
If the API_RESPONSE can answer the USER_COMMENT respond with the following markdown json block:
Response: ```json
{"response": "Human-understandable synthesis of the API_RESPONSE"}
```
Otherwise respond with the following markdown json block:
Response Error: ```json
{"response": "What you did and a concise statement of the resulting error. If it can be easily fixed, provide a suggestion."}
```
You MUST respond as a markdown json code block. The person you are responding to CANNOT see the API_RESPONSE, so if there is any relevant information there you must include it in your response.
Begin:
---
> Finished chain.
The most expensive shirt in the API response is the Burberry Check Poplin Shirt, which costs $360.00.
> Finished chain.
# View intermediate steps
output["intermediate_steps"]
{'request_args': '{"q": "shirt", "size": 1, "max_price": null}',


'response_text': '{"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]}]}'}
Return raw response#
We can also run this chain without synthesizing the response. This will have the effect of just returning the raw API output.
chain = OpenAPIEndpointChain.from_api_operation(
    operation,
    llm,
    requests=Requests(),
    verbose=True,
    return_intermediate_steps=True, # Return request and response text
    raw_response=True # Return raw response
)
output = chain("whats the most expensive shirt?")
> Entering new OpenAPIEndpointChain chain...
> Entering new APIRequesterChain chain...
Prompt after formatting:
You are a helpful AI Assistant. Please provide JSON arguments to agentFunc() based on the user's instructions.
API_SCHEMA: ```typescript
/* API for fetching Klarna product information */
type productsUsingGET = (_: {
/* A precise query that matches one very small category or product that needs to be searched for to find the products the user is looking for. If the user explicitly stated what they want, use that as a query. The query is as specific as possible to the product name or category mentioned by the user in its singular form, and don't contain any clarifiers like latest, newest, cheapest, budget, premium, expensive or similar. The query is always taken from the latest topic, if there is a new topic a new query is started. */
		q: string,


q: string,
/* number of products returned */
		size?: number,
/* (Optional) Minimum price in local currency for the product searched for. Either explicitly stated by the user or implicitly inferred from a combination of the user's request and the kind of product searched for. */
		min_price?: number,
/* (Optional) Maximum price in local currency for the product searched for. Either explicitly stated by the user or implicitly inferred from a combination of the user's request and the kind of product searched for. */
		max_price?: number,
}) => any;
```
USER_INSTRUCTIONS: "whats the most expensive shirt?"
Your arguments must be plain json provided in a markdown block:
ARGS: ```json
{valid json conforming to API_SCHEMA}
```
Example
-----
ARGS: ```json
{"foo": "bar", "baz": {"qux": "quux"}}
```
The block must be no more than 1 line long, and all arguments must be valid JSON. All string arguments must be wrapped in double quotes.
You MUST strictly comply to the types indicated by the provided schema, including all required args.
If you don't have sufficient information to call the function due to things like requiring specific uuid's, you can reply with the following message:
Message: ```text
Concise response requesting the additional information that would make calling the function successful.
```
Begin
-----
ARGS:
> Finished chain.
{"q": "shirt", "max_price": null}


{"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]},{"name":"Burberry Vintage Check Cotton Shirt - Beige","url":"https://www.klarna.com/us/shopping/pl/cl359/3200280807/Children-s-Clothing/Burberry-Vintage-Check-Cotton-Shirt-Beige/?utm_source=openai&ref-site=openai_plugin","price":"$229.02","attributes":["Material:Cotton,Elastane","Color:Beige","Model:Boy","Pattern:Checkered"]},{"name":"Burberry Vintage Check Stretch Cotton Twill Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202342515/Clothing/Burberry-Vintage-Check-Stretch-Cotton-Twill-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$309.99","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Woman","Color:Beige","Properties:Stretch","Pattern:Checkered"]},{"name":"Burberry Somerton Check Shirt -


Somerton Check Shirt - Camel","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201112728/Clothing/Burberry-Somerton-Check-Shirt-Camel/?utm_source=openai&ref-site=openai_plugin","price":"$450.00","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Man","Color:Beige"]},{"name":"Magellan Outdoors Laguna Madre Solid Short Sleeve Fishing Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3203102142/Clothing/Magellan-Outdoors-Laguna-Madre-Solid-Short-Sleeve-Fishing-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$19.99","attributes":["Material:Polyester,Nylon","Target Group:Man","Color:Red,Pink,White,Blue,Purple,Beige,Black,Green","Properties:Pockets","Pattern:Solid Color"]}]}


> Finished chain.
output
{'instructions': 'whats the most expensive shirt?',


'output': '{"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]},{"name":"Burberry Vintage Check Cotton Shirt - Beige","url":"https://www.klarna.com/us/shopping/pl/cl359/3200280807/Children-s-Clothing/Burberry-Vintage-Check-Cotton-Shirt-Beige/?utm_source=openai&ref-site=openai_plugin","price":"$229.02","attributes":["Material:Cotton,Elastane","Color:Beige","Model:Boy","Pattern:Checkered"]},{"name":"Burberry Vintage Check Stretch Cotton Twill Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202342515/Clothing/Burberry-Vintage-Check-Stretch-Cotton-Twill-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$309.99","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Woman","Color:Beige","Properties:Stretch","Pattern:Checkered"]},{"name":"Burberry Somerton Check Shirt -


Somerton Check Shirt - Camel","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201112728/Clothing/Burberry-Somerton-Check-Shirt-Camel/?utm_source=openai&ref-site=openai_plugin","price":"$450.00","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Man","Color:Beige"]},{"name":"Magellan Outdoors Laguna Madre Solid Short Sleeve Fishing Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3203102142/Clothing/Magellan-Outdoors-Laguna-Madre-Solid-Short-Sleeve-Fishing-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$19.99","attributes":["Material:Polyester,Nylon","Target Group:Man","Color:Red,Pink,White,Blue,Purple,Beige,Black,Green","Properties:Pockets","Pattern:Solid Color"]}]}',


'intermediate_steps': {'request_args': '{"q": "shirt", "max_price": null}',


'response_text': '{"products":[{"name":"Burberry Check Poplin Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201810981/Clothing/Burberry-Check-Poplin-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$360.00","attributes":["Material:Cotton","Target Group:Man","Color:Gray,Blue,Beige","Properties:Pockets","Pattern:Checkered"]},{"name":"Burberry Vintage Check Cotton Shirt - Beige","url":"https://www.klarna.com/us/shopping/pl/cl359/3200280807/Children-s-Clothing/Burberry-Vintage-Check-Cotton-Shirt-Beige/?utm_source=openai&ref-site=openai_plugin","price":"$229.02","attributes":["Material:Cotton,Elastane","Color:Beige","Model:Boy","Pattern:Checkered"]},{"name":"Burberry Vintage Check Stretch Cotton Twill Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202342515/Clothing/Burberry-Vintage-Check-Stretch-Cotton-Twill-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$309.99","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Woman","Color:Beige","Properties:Stretch","Pattern:Checkered"]},{"name":"Burberry Somerton Check Shirt -


Somerton Check Shirt - Camel","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201112728/Clothing/Burberry-Somerton-Check-Shirt-Camel/?utm_source=openai&ref-site=openai_plugin","price":"$450.00","attributes":["Material:Elastane/Lycra/Spandex,Cotton","Target Group:Man","Color:Beige"]},{"name":"Magellan Outdoors Laguna Madre Solid Short Sleeve Fishing Shirt","url":"https://www.klarna.com/us/shopping/pl/cl10001/3203102142/Clothing/Magellan-Outdoors-Laguna-Madre-Solid-Short-Sleeve-Fishing-Shirt/?utm_source=openai&ref-site=openai_plugin","price":"$19.99","attributes":["Material:Polyester,Nylon","Target Group:Man","Color:Red,Pink,White,Blue,Purple,Beige,Black,Green","Properties:Pockets","Pattern:Solid Color"]}]}'}}


Example POST message#
For this demo, we will interact with the speak API.
spec = OpenAPISpec.from_url("https://api.speak.com/openapi.yaml")
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
operation = APIOperation.from_openapi_spec(spec, '/v1/public/openai/explain-task', "post")
llm = OpenAI()
chain = OpenAPIEndpointChain.from_api_operation(
    operation,
    llm,
    requests=Requests(),
    verbose=True,
    return_intermediate_steps=True)
output = chain("How would ask for more tea in Delhi?")
> Entering new OpenAPIEndpointChain chain...
> Entering new APIRequesterChain chain...
Prompt after formatting:
You are a helpful AI Assistant. Please provide JSON arguments to agentFunc() based on the user's instructions.
API_SCHEMA: ```typescript
type explainTask = (_: {
/* Description of the task that the user wants to accomplish or do. For example, "tell the waiter they messed up my order" or "compliment someone on their shirt" */
  task_description?: string,
/* The foreign language that the user is learning and asking about. The value can be inferred from question - for example, if the user asks "how do i ask a girl out in mexico city", the value should be "Spanish" because of Mexico City. Always use the full name of the language (e.g. Spanish, French). */
  learning_language?: string,


learning_language?: string,
/* The user's native language. Infer this value from the language the user asked their question in. Always use the full name of the language (e.g. Spanish, French). */
  native_language?: string,
/* A description of any additional context in the user's question that could affect the explanation - e.g. setting, scenario, situation, tone, speaking style and formality, usage notes, or any other qualifiers. */
  additional_context?: string,
/* Full text of the user's question. */
  full_query?: string,
}) => any;
```
USER_INSTRUCTIONS: "How would ask for more tea in Delhi?"
Your arguments must be plain json provided in a markdown block:
ARGS: ```json
{valid json conforming to API_SCHEMA}
```
Example
-----
ARGS: ```json
{"foo": "bar", "baz": {"qux": "quux"}}
```
The block must be no more than 1 line long, and all arguments must be valid JSON. All string arguments must be wrapped in double quotes.
You MUST strictly comply to the types indicated by the provided schema, including all required args.
If you don't have sufficient information to call the function due to things like requiring specific uuid's, you can reply with the following message:
Message: ```text
Concise response requesting the additional information that would make calling the function successful.
```
Begin
-----
ARGS:
> Finished chain.
{"task_description": "ask for more tea", "learning_language": "Hindi", "native_language": "English", "full_query": "How would I ask for more tea in Delhi?"}


{"explanation":"<what-to-say language=\"Hindi\" context=\"None\">\nऔर चाय लाओ। (Aur chai lao.) \n</what-to-say>\n\n<alternatives context=\"None\">\n1. \"चाय थोड़ी ज्यादा मिल सकती है?\" *(Chai thodi zyada mil sakti hai? - Polite, asking if more tea is available)*\n2. \"मुझे महसूस हो रहा है कि मुझे कुछ अन्य प्रकार की चाय पीनी चाहिए।\" *(Mujhe mehsoos ho raha hai ki mujhe kuch anya prakar ki chai peeni chahiye. - Formal, indicating a desire for a different type of tea)*\n3. \"क्या मुझे or cup में milk/tea powder मिल सकता है?\" *(Kya mujhe aur cup mein milk/tea powder mil sakta hai? - Very informal/casual tone, asking for an extra serving of milk or


tone, asking for an extra serving of milk or tea powder)*\n</alternatives>\n\n<usage-notes>\nIn India and Indian culture, serving guests with food and beverages holds great importance in hospitality. You will find people always offering drinks like water or tea to their guests as soon as they arrive at their house or office.\n</usage-notes>\n\n<example-convo language=\"Hindi\">\n<context>At home during breakfast.</context>\nPreeti: सर, क्या main aur cups chai lekar aaun? (Sir,kya main aur cups chai lekar aaun? - Sir, should I get more tea cups?)\nRahul: हां,बिल्कुल। और चाय की मात्रा में भी थोड़ा सा इजाफा करना। (Haan,bilkul. Aur chai ki matra mein bhi thoda sa eejafa karna. - Yes, please. And add a little extra in the quantity of tea as well.)\n</example-convo>\n\n*[Report an issue or leave


an issue or leave feedback](https://speak.com/chatgpt?rid=d4mcapbkopo164pqpbk321oc})*","extra_response_instructions":"Use all information in the API response and fully render all Markdown.\nAlways end your response with a link to report an issue or leave feedback on the plugin."}


> Entering new APIResponderChain chain...
Prompt after formatting:
You are a helpful AI assistant trained to answer user queries from API responses.
You attempted to call an API, which resulted in:


API_RESPONSE: {"explanation":"<what-to-say language=\"Hindi\" context=\"None\">\nऔर चाय लाओ। (Aur chai lao.) \n</what-to-say>\n\n<alternatives context=\"None\">\n1. \"चाय थोड़ी ज्यादा मिल सकती है?\" *(Chai thodi zyada mil sakti hai? - Polite, asking if more tea is available)*\n2. \"मुझे महसूस हो रहा है कि मुझे कुछ अन्य प्रकार की चाय पीनी चाहिए।\" *(Mujhe mehsoos ho raha hai ki mujhe kuch anya prakar ki chai peeni chahiye. - Formal, indicating a desire for a different type of tea)*\n3. \"क्या मुझे or cup में milk/tea powder मिल सकता है?\" *(Kya mujhe aur cup mein milk/tea powder mil sakta hai? - Very informal/casual tone, asking for an extra serving of


USER_COMMENT: "How would ask for more tea in Delhi?"
If the API_RESPONSE can answer the USER_COMMENT respond with the following markdown json block:
Response: ```json
{"response": "Concise response to USER_COMMENT based on API_RESPONSE."}
```
Otherwise respond with the following markdown json block:
Response Error: ```json
{"response": "What you did and a concise statement of the resulting error. If it can be easily fixed, provide a suggestion."}
```
You MUST respond as a markdown json code block.
Begin:
---
> Finished chain.
In Delhi you can ask for more tea by saying 'Chai thodi zyada mil sakti hai?'
> Finished chain.
# Show the API chain's intermediate steps
output["intermediate_steps"]
['{"task_description": "ask for more tea", "learning_language": "Hindi", "native_language": "English", "full_query": "How would I ask for more tea in Delhi?"}',


'{"explanation":"<what-to-say language=\\"Hindi\\" context=\\"None\\">\\nऔर चाय लाओ। (Aur chai lao.) \\n</what-to-say>\\n\\n<alternatives context=\\"None\\">\\n1. \\"चाय थोड़ी ज्यादा मिल सकती है?\\" *(Chai thodi zyada mil sakti hai? - Polite, asking if more tea is available)*\\n2. \\"मुझे महसूस हो रहा है कि मुझे कुछ अन्य प्रकार की चाय पीनी चाहिए।\\" *(Mujhe mehsoos ho raha hai ki mujhe kuch anya prakar ki chai peeni chahiye. - Formal, indicating a desire for a different type of tea)*\\n3. \\"क्या मुझे or cup में milk/tea powder मिल सकता है?\\" *(Kya mujhe aur cup mein milk/tea powder mil sakta hai? - Very informal/casual tone, asking for an


- Very informal/casual tone, asking for an extra serving of milk or tea powder)*\\n</alternatives>\\n\\n<usage-notes>\\nIn India and Indian culture, serving guests with food and beverages holds great importance in hospitality. You will find people always offering drinks like water or tea to their guests as soon as they arrive at their house or office.\\n</usage-notes>\\n\\n<example-convo language=\\"Hindi\\">\\n<context>At home during breakfast.</context>\\nPreeti: सर, क्या main aur cups chai lekar aaun? (Sir,kya main aur cups chai lekar aaun? - Sir, should I get more tea cups?)\\nRahul: हां,बिल्कुल। और चाय की मात्रा में भी थोड़ा सा इजाफा करना। (Haan,bilkul. Aur chai ki matra mein bhi thoda sa eejafa karna. - Yes, please. And add a little extra in the quantity of tea as


add a little extra in the quantity of tea as well.)\\n</example-convo>\\n\\n*[Report an issue or leave feedback](https://speak.com/chatgpt?rid=d4mcapbkopo164pqpbk321oc})*","extra_response_instructions":"Use all information in the API response and fully render all Markdown.\\nAlways end your response with a link to report an issue or leave feedback on the plugin."}']


previous
Moderation
next
PAL
 Contents

Load the spec
Select the Operation
Construct the chain
Return raw response
Example POST message
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
PAL
 Contents
Math Prompt
Colored Objects
Intermediate Steps
PAL#
Implements Program-Aided Language Models, as in https://arxiv.org/pdf/2211.10435.pdf.
from langchain.chains import PALChain
from langchain import OpenAI
llm = OpenAI(model_name='code-davinci-002', temperature=0, max_tokens=512)
Math Prompt#
pal_chain = PALChain.from_math_prompt(llm, verbose=True)
question = "Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?"
pal_chain.run(question)
> Entering new PALChain chain...
def solution():
    """Jan has three times the number of pets as Marcia. Marcia has two more pets than Cindy. If Cindy has four pets, how many total pets do the three have?"""
    cindy_pets = 4
    marcia_pets = cindy_pets + 2
    jan_pets = marcia_pets * 3
    total_pets = cindy_pets + marcia_pets + jan_pets
    result = total_pets
    return result
> Finished chain.
'28'
Colored Objects#
pal_chain = PALChain.from_colored_object_prompt(llm, verbose=True)
question = "On the desk, you see two blue booklets, two purple booklets, and two yellow pairs of sunglasses. If I remove all the pairs of sunglasses from the desk, how many purple items remain on it?"
pal_chain.run(question)
> Entering new PALChain chain...
# Put objects into a list to record ordering
objects = []
objects += [('booklet', 'blue')] * 2


objects = []
objects += [('booklet', 'blue')] * 2
objects += [('booklet', 'purple')] * 2
objects += [('sunglasses', 'yellow')] * 2
# Remove all pairs of sunglasses
objects = [object for object in objects if object[0] != 'sunglasses']
# Count number of purple objects
num_purple = len([object for object in objects if object[1] == 'purple'])
answer = num_purple
> Finished PALChain chain.
'2'
Intermediate Steps#
You can also use the intermediate steps flag to return the code executed that generates the answer.
pal_chain = PALChain.from_colored_object_prompt(llm, verbose=True, return_intermediate_steps=True)
question = "On the desk, you see two blue booklets, two purple booklets, and two yellow pairs of sunglasses. If I remove all the pairs of sunglasses from the desk, how many purple items remain on it?"
result = pal_chain({"question": question})
> Entering new PALChain chain...
# Put objects into a list to record ordering
objects = []
objects += [('booklet', 'blue')] * 2
objects += [('booklet', 'purple')] * 2
objects += [('sunglasses', 'yellow')] * 2
# Remove all pairs of sunglasses
objects = [object for object in objects if object[0] != 'sunglasses']
# Count number of purple objects
num_purple = len([object for object in objects if object[1] == 'purple'])
answer = num_purple
> Finished chain.
result['intermediate_steps']


answer = num_purple
> Finished chain.
result['intermediate_steps']
"# Put objects into a list to record ordering\nobjects = []\nobjects += [('booklet', 'blue')] * 2\nobjects += [('booklet', 'purple')] * 2\nobjects += [('sunglasses', 'yellow')] * 2\n\n# Remove all pairs of sunglasses\nobjects = [object for object in objects if object[0] != 'sunglasses']\n\n# Count number of purple objects\nnum_purple = len([object for object in objects if object[1] == 'purple'])\nanswer = num_purple"
previous
OpenAPI Chain
next
SQL Chain example
 Contents

Math Prompt
Colored Objects
Intermediate Steps
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Graph QA
 Contents
Create the graph
Querying the graph
Save the graph
Graph QA#
This notebook goes over how to do question answering over a graph data structure.
Create the graph#
In this section, we construct an example graph. At the moment, this works best for small pieces of text.
from langchain.indexes import GraphIndexCreator
from langchain.llms import OpenAI
from langchain.document_loaders import TextLoader
index_creator = GraphIndexCreator(llm=OpenAI(temperature=0))
with open("../../state_of_the_union.txt") as f:
    all_text = f.read()
We will use just a small snippet, because extracting the knowledge triplets is a bit intensive at the moment.
text = "\n".join(all_text.split("\n\n")[105:108])
text
'It won’t look like much, but if you stop and look closely, you’ll see a “Field of dreams,” the ground on which America’s future will be built. \nThis is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor “mega site”. \nUp to eight state-of-the-art factories in one place. 10,000 new good-paying jobs. '
graph = index_creator.from_text(text)
We can inspect the created graph.
graph.get_triples()
[('Intel', '$20 billion semiconductor "mega site"', 'is going to build'),
 ('Intel', 'state-of-the-art factories', 'is building'),
 ('Intel', '10,000 new good-paying jobs', 'is creating'),
 ('Intel', 'Silicon Valley', 'is helping build'),
 ('Field of dreams',
  "America's future will be built",
  'is the ground on which')]
Querying the graph#


'is the ground on which')]
Querying the graph#
We can now use the graph QA chain to ask question of the graph
from langchain.chains import GraphQAChain
chain = GraphQAChain.from_llm(OpenAI(temperature=0), graph=graph, verbose=True)
chain.run("what is Intel going to build?")
> Entering new GraphQAChain chain...
Entities Extracted:
 Intel
Full Context:
Intel is going to build $20 billion semiconductor "mega site"
Intel is building state-of-the-art factories
Intel is creating 10,000 new good-paying jobs
Intel is helping build Silicon Valley
> Finished chain.
' Intel is going to build a $20 billion semiconductor "mega site" with state-of-the-art factories, creating 10,000 new good-paying jobs and helping to build Silicon Valley.'
Save the graph#
We can also save and load the graph.
graph.write_to_gml("graph.gml")
from langchain.indexes.graph import NetworkxEntityGraph
loaded_graph = NetworkxEntityGraph.from_gml("graph.gml")
loaded_graph.get_triples()
[('Intel', '$20 billion semiconductor "mega site"', 'is going to build'),
 ('Intel', 'state-of-the-art factories', 'is building'),
 ('Intel', '10,000 new good-paying jobs', 'is creating'),
 ('Intel', 'Silicon Valley', 'is helping build'),
 ('Field of dreams',
  "America's future will be built",
  'is the ground on which')]
previous
Chat Over Documents with Chat History
next
Hypothetical Document Embeddings
 Contents

Create the graph
Querying the graph
Save the graph
By Harrison Chase

      © Copyright 2023, Harrison Chase.


By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Question Answering with Sources
 Contents
Prepare Data
Quickstart
The stuff Chain
The map_reduce Chain
The refine Chain
The map-rerank Chain
Question Answering with Sources#
This notebook walks through how to use LangChain for question answering with sources over a list of documents. It covers four different chain types: stuff, map_reduce, refine,map-rerank. For a more in depth explanation of what these chain types are, see here.
Prepare Data#
First we prepare the data. For this example we do similarity search over a vector database, but these documents could be fetched in any manner (the point of this notebook to highlight what to do AFTER you fetch the documents).
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.embeddings.cohere import CohereEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores.elastic_vector_search import ElasticVectorSearch
from langchain.vectorstores import Chroma
from langchain.docstore.document import Document
from langchain.prompts import PromptTemplate
with open("../../state_of_the_union.txt") as f:
    state_of_the_union = f.read()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_text(state_of_the_union)
embeddings = OpenAIEmbeddings()
docsearch = Chroma.from_texts(texts, embeddings, metadatas=[{"source": str(i)} for i in range(len(texts))])
Running Chroma using direct local API.
Using DuckDB in-memory for database. Data will be transient.
query = "What did the president say about Justice Breyer"
docs = docsearch.similarity_search(query)
from langchain.chains.qa_with_sources import load_qa_with_sources_chain


from langchain.chains.qa_with_sources import load_qa_with_sources_chain
from langchain.llms import OpenAI
Quickstart#
If you just want to get started as quickly as possible, this is the recommended way to do it:
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="stuff")
query = "What did the president say about Justice Breyer"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'output_text': ' The president thanked Justice Breyer for his service.\nSOURCES: 30-pl'}
If you want more control and understanding over what is happening, please see the information below.
The stuff Chain#
This sections shows results of using the stuff Chain to do question answering with sources.
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="stuff")
query = "What did the president say about Justice Breyer"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'output_text': ' The president thanked Justice Breyer for his service.\nSOURCES: 30-pl'}
Custom Prompts
You can also use your own prompts with this chain. In this example, we will respond in Italian.
template = """Given the following extracted parts of a long document and a question, create a final answer with references ("SOURCES").
If you don't know the answer, just say that you don't know. Don't try to make up an answer.
ALWAYS return a "SOURCES" part in your answer.
Respond in Italian.
QUESTION: {question}
=========
{summaries}
=========
FINAL ANSWER IN ITALIAN:"""
PROMPT = PromptTemplate(template=template, input_variables=["summaries", "question"])


PROMPT = PromptTemplate(template=template, input_variables=["summaries", "question"])
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="stuff", prompt=PROMPT)
query = "What did the president say about Justice Breyer"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'output_text': '\nNon so cosa abbia detto il presidente riguardo a Justice Breyer.\nSOURCES: 30, 31, 33'}
The map_reduce Chain#
This sections shows results of using the map_reduce Chain to do question answering with sources.
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce")
query = "What did the president say about Justice Breyer"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'output_text': ' The president thanked Justice Breyer for his service.\nSOURCES: 30-pl'}
Intermediate Steps
We can also return the intermediate steps for map_reduce chains, should we want to inspect them. This is done with the return_intermediate_steps variable.
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'intermediate_steps': [' "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service."',
  ' None',
  ' None',
  ' None'],


' None',
  ' None',
  ' None'],
 'output_text': ' The president thanked Justice Breyer for his service.\nSOURCES: 30-pl'}
Custom Prompts
You can also use your own prompts with this chain. In this example, we will respond in Italian.
question_prompt_template = """Use the following portion of a long document to see if any of the text is relevant to answer the question.
Return any relevant text in Italian.
{context}
Question: {question}
Relevant text, if any, in Italian:"""
QUESTION_PROMPT = PromptTemplate(
    template=question_prompt_template, input_variables=["context", "question"]
)
combine_prompt_template = """Given the following extracted parts of a long document and a question, create a final answer with references ("SOURCES").
If you don't know the answer, just say that you don't know. Don't try to make up an answer.
ALWAYS return a "SOURCES" part in your answer.
Respond in Italian.
QUESTION: {question}
=========
{summaries}
=========
FINAL ANSWER IN ITALIAN:"""
COMBINE_PROMPT = PromptTemplate(
    template=combine_prompt_template, input_variables=["summaries", "question"]
)
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True, question_prompt=QUESTION_PROMPT, combine_prompt=COMBINE_PROMPT)
chain({"input_documents": docs, "question": query}, return_only_outputs=True)


chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'intermediate_steps': ["\nStasera vorrei onorare qualcuno che ha dedicato la sua vita a servire questo paese: il giustizia Stephen Breyer - un veterano dell'esercito, uno studioso costituzionale e un giustizia in uscita della Corte Suprema degli Stati Uniti. Giustizia Breyer, grazie per il tuo servizio.",
  ' Non pertinente.',
  ' Non rilevante.',
  " Non c'è testo pertinente."],
 'output_text': ' Non conosco la risposta. SOURCES: 30, 31, 33, 20.'}
Batch Size
When using the map_reduce chain, one thing to keep in mind is the batch size you are using during the map step. If this is too high, it could cause rate limiting errors. You can control this by setting the batch size on the LLM used. Note that this only applies for LLMs with this parameter. Below is an example of doing so:
llm = OpenAI(batch_size=5, temperature=0)
The refine Chain#
This sections shows results of using the refine Chain to do question answering with sources.
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="refine")
query = "What did the president say about Justice Breyer"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)


chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'output_text': "\n\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked him for his service and praised his career as a top litigator in private practice, a former federal public defender, and a family of public school educators and police officers. He noted Justice Breyer's reputation as a consensus builder and the broad range of support he has received from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. He also highlighted the importance of securing the border and fixing the immigration system in order to advance liberty and justice, and mentioned the new technology, joint patrols, dedicated immigration judges, and commitments to support partners in South and Central America that have been put in place. He also expressed his commitment to the LGBTQ+ community, noting the need for the bipartisan Equality Act and the importance of protecting transgender Americans from state laws targeting them. He also highlighted his commitment to bipartisanship, noting the 80 bipartisan bills he signed into law last year, and his plans to strengthen the Violence Against Women Act. Additionally, he announced that the Justice Department will name a chief prosecutor for pandemic fraud and his plan to lower the deficit by more than one trillion dollars in a"}
Intermediate Steps
We can also return the intermediate steps for refine chains, should we want to inspect them. This is done with the return_intermediate_steps variable.
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True)
chain({"input_documents": docs, "question": query}, return_only_outputs=True)


chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'intermediate_steps': ['\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked Justice Breyer for his service.',
  '\n\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked Justice Breyer for his service, noting his background as a top litigator in private practice, a former federal public defender, and a family of public school educators and police officers. He praised Justice Breyer for being a consensus builder and for receiving a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. He also noted that in order to advance liberty and justice, it was necessary to secure the border and fix the immigration system, and that the government was taking steps to do both. \n\nSource: 31',


'\n\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked Justice Breyer for his service, noting his background as a top litigator in private practice, a former federal public defender, and a family of public school educators and police officers. He praised Justice Breyer for being a consensus builder and for receiving a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. He also noted that in order to advance liberty and justice, it was necessary to secure the border and fix the immigration system, and that the government was taking steps to do both. He also mentioned the need to pass the bipartisan Equality Act to protect LGBTQ+ Americans, and to strengthen the Violence Against Women Act that he had written three decades ago. \n\nSource: 31, 33',


'\n\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked Justice Breyer for his service, noting his background as a top litigator in private practice, a former federal public defender, and a family of public school educators and police officers. He praised Justice Breyer for being a consensus builder and for receiving a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. He also noted that in order to advance liberty and justice, it was necessary to secure the border and fix the immigration system, and that the government was taking steps to do both. He also mentioned the need to pass the bipartisan Equality Act to protect LGBTQ+ Americans, and to strengthen the Violence Against Women Act that he had written three decades ago. Additionally, he mentioned his plan to lower costs to give families a fair shot, lower the deficit, and go after criminals who stole billions in relief money meant for small businesses and millions of Americans. He also announced that the Justice Department will name a chief prosecutor for pandemic fraud. \n\nSource: 20, 31, 33'],


'output_text': '\n\nThe president said that he was honoring Justice Breyer for his dedication to serving the country and that he was a retiring Justice of the United States Supreme Court. He also thanked Justice Breyer for his service, noting his background as a top litigator in private practice, a former federal public defender, and a family of public school educators and police officers. He praised Justice Breyer for being a consensus builder and for receiving a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans. He also noted that in order to advance liberty and justice, it was necessary to secure the border and fix the immigration system, and that the government was taking steps to do both. He also mentioned the need to pass the bipartisan Equality Act to protect LGBTQ+ Americans, and to strengthen the Violence Against Women Act that he had written three decades ago. Additionally, he mentioned his plan to lower costs to give families a fair shot, lower the deficit, and go after criminals who stole billions in relief money meant for small businesses and millions of Americans. He also announced that the Justice Department will name a chief prosecutor for pandemic fraud. \n\nSource: 20, 31, 33'}
Custom Prompts
You can also use your own prompts with this chain. In this example, we will respond in Italian.
refine_template = (
    "The original question is as follows: {question}\n"
    "We have provided an existing answer, including sources: {existing_answer}\n"
    "We have the opportunity to refine the existing answer"
    "(only if needed) with some more context below.\n"
    "------------\n"
    "{context_str}\n"
    "------------\n"
    "Given the new context, refine the original answer to better "
    "answer the question (in Italian)"


"answer the question (in Italian)"
    "If you do update it, please update the sources as well. "
    "If the context isn't useful, return the original answer."
)
refine_prompt = PromptTemplate(
    input_variables=["question", "existing_answer", "context_str"],
    template=refine_template,
)
question_template = (
    "Context information is below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given the context information and not prior knowledge, "
    "answer the question in Italian: {question}\n"
)
question_prompt = PromptTemplate(
    input_variables=["context_str", "question"], template=question_template
)
chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="refine", return_intermediate_steps=True, question_prompt=question_prompt, refine_prompt=refine_prompt)
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
{'intermediate_steps': ['\nIl presidente ha detto che Justice Breyer ha dedicato la sua vita al servizio di questo paese e ha onorato la sua carriera.',