'link': 'https://www.reuters.com/business/autos-transportation/iss-recommends-tesla-investors-vote-against-re-election-robyn-denholm-2023-05-04/',
           'snippet': 'Proxy advisory firm ISS on Wednesday recommended Tesla '
                      'investors vote against re-election of board chair Robyn '
                      'Denholm, citing "concerns on...',
           'date': '5 mins ago',
           'source': 'Reuters',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcROdETe_GUyp1e8RHNhaRM8Z_vfxCvdfinZwzL1bT1ZGSYaGTeOojIdBoLevA&s',
           'position': 1},
          {'title': 'Global companies by market cap: Tesla fell most in April',
           'link': 'https://www.reuters.com/markets/global-companies-by-market-cap-tesla-fell-most-april-2023-05-02/',
           'snippet': 'Tesla Inc was the biggest loser among top companies by '
                      'market capitalisation in April, hit by disappointing '
                      'quarterly earnings after it...',
           'date': '1 day ago',
           'source': 'Reuters',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ4u4CP8aOdGyRFH6o4PkXi-_eZDeY96vLSag5gDjhKMYf98YBER2cZPbkStQ&s',
           'position': 2},
          {'title': 'Tesla Wanted an EV Price War. Ford Showed Up.',


'link': 'https://www.bloomberg.com/opinion/articles/2023-05-03/tesla-wanted-an-ev-price-war-ford-showed-up',
           'snippet': 'The legacy automaker is paring back the cost of its '
                      'Mustang Mach-E model after Tesla discounted its '
                      'competing EVs, portending tighter...',
           'date': '6 hours ago',
           'source': 'Bloomberg.com',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_3Eo4VI0H-nTeIbYc5DaQn5ep7YrWnmhx6pv8XddFgNF5zRC9gEpHfDq8yQ&s',
           'position': 3},
          {'title': 'Joby Aviation to get investment from Tesla shareholder '
                    'Baillie Gifford',
           'link': 'https://finance.yahoo.com/news/joby-aviation-investment-tesla-shareholder-204450712.html',
           'snippet': 'This comes days after Joby clinched a $55 million '
                      'contract extension to deliver up to nine air taxis to '
                      'the U.S. Air Force,...',
           'date': '4 hours ago',
           'source': 'Yahoo Finance',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQO0uVn297LI-xryrPNqJ-apUOulj4ohM-xkN4OfmvMOYh1CPdUEBbYx6hviw&s',
           'position': 4},
          {'title': 'Tesla resumes U.S. orders for a Model 3 version at lower '


'price, range',
           'link': 'https://finance.yahoo.com/news/tesla-resumes-us-orders-model-045736115.html',
           'snippet': '(Reuters) -Tesla Inc has resumed taking orders for its '
                      'Model 3 long-range vehicle in the United States, the '
                      "company's website showed late on...",
           'date': '19 hours ago',
           'source': 'Yahoo Finance',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTIZetJ62sQefPfbQ9KKDt6iH7Mc0ylT5t_hpgeeuUkHhJuAx2FOJ4ZTRVDFg&s',
           'position': 5},
          {'title': 'The Tesla Model 3 Long Range AWD Is Now Available in the '
                    'U.S. With 325 Miles of Range',
           'link': 'https://www.notateslaapp.com/news/1393/tesla-reopens-orders-for-model-3-long-range-after-months-of-unavailability',
           'snippet': 'Tesla has reopened orders for the Model 3 Long Range '
                      'RWD, which has been unavailable for months due to high '
                      'demand.',
           'date': '7 hours ago',
           'source': 'Not a Tesla App',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSecrgxZpRj18xIJY-nDHljyP-A4ejEkswa9eq77qhMNrScnVIqe34uql5U4w&s',
           'position': 6},
          {'title': 'Tesla Cybertruck alpha prototype spotted at the Fremont '


'factory in new pics and videos',
           'link': 'https://www.teslaoracle.com/2023/05/03/tesla-cybertruck-alpha-prototype-interior-and-exterior-spotted-at-the-fremont-factory-in-new-pics-and-videos/',
           'snippet': 'A Tesla Cybertruck alpha prototype goes to Fremont, '
                      'California for another round of testing before going to '
                      'production later this year (pics...',
           'date': '14 hours ago',
           'source': 'Tesla Oracle',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRO7M5ZLQE-Zo4-_5dv9hNAQZ3wSqfvYCuKqzxHG-M6CgLpwPMMG_ssebdcMg&s',
           'position': 7},
          {'title': 'Tesla putting facility in new part of country - Austin '
                    'Business Journal',
           'link': 'https://www.bizjournals.com/austin/news/2023/05/02/tesla-leases-building-seattle-area.html',
           'snippet': 'Check out what Puget Sound Business Journal has to '
                      "report about the Austin-based company's real estate "
                      'footprint in the Pacific Northwest.',
           'date': '22 hours ago',
           'source': 'The Business Journals',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR9kIEHWz1FcHKDUtGQBS0AjmkqtyuBkQvD8kyIY3kpaPrgYaN7I_H2zoOJsA&s',
           'position': 8},


{'title': 'Tesla (TSLA) Resumes Orders for Model 3 Long Range After '
                    'Backlog',
           'link': 'https://www.bloomberg.com/news/articles/2023-05-03/tesla-resumes-orders-for-popular-model-3-long-range-at-47-240',
           'snippet': 'Tesla Inc. has resumed taking orders for its Model 3 '
                      'Long Range edition with a starting price of $47240, '
                      'according to its website.',
           'date': '5 hours ago',
           'source': 'Bloomberg.com',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTWWIC4VpMTfRvSyqiomODOoLg0xhoBf-Tc1qweKnSuaiTk-Y1wMJZM3jct0w&s',
           'position': 9}]}
If you want to only receive news articles published in the last hour, you can do the following:
search = GoogleSerperAPIWrapper(type="news", tbs="qdr:h")
results = search.results("Tesla Inc.")
pprint.pp(results)
{'searchParameters': {'q': 'Tesla Inc.',
                      'gl': 'us',
                      'hl': 'en',
                      'num': 10,
                      'type': 'news',
                      'tbs': 'qdr:h'},
 'news': [{'title': 'Oklahoma Gov. Stitt sees growing foreign interest in '
                    'investments in ...',
           'link': 'https://www.reuters.com/world/us/oklahoma-gov-stitt-sees-growing-foreign-interest-investments-state-2023-05-04/',


'snippet': 'T)), a battery supplier to electric vehicle maker Tesla '
                      'Inc (TSLA.O), said on Sunday it is considering building '
                      'a battery plant in Oklahoma, its third in...',
           'date': '53 mins ago',
           'source': 'Reuters',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSSTcsXeenqmEKdiekvUgAmqIPR4nlAmgjTkBqLpza-lLfjX1CwB84MoNVj0Q&s',
           'position': 1},
          {'title': 'Ryder lanza solución llave en mano para vehículos '
                    'eléctricos en EU',
           'link': 'https://www.tyt.com.mx/nota/ryder-lanza-solucion-llave-en-mano-para-vehiculos-electricos-en-eu',
           'snippet': 'Ryder System Inc. presentó RyderElectric+ TM como su '
                      'nueva solución llave en mano ... Ryder también tiene '
                      'reservados los semirremolques Tesla y continúa...',
           'date': '56 mins ago',
           'source': 'Revista Transportes y Turismo',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJhXTQQtjSUZf9YPM235WQhFU5_d7lEA76zB8DGwZfixcgf1_dhPJyKA1Nbw&s',
           'position': 2},
          {'title': '"I think people can get by with $999 million," Bernie '
                    'Sanders tells American Billionaires.',


'link': 'https://thebharatexpressnews.com/i-think-people-can-get-by-with-999-million-bernie-sanders-tells-american-billionaires-heres-how-the-ultra-rich-can-pay-less-income-tax-than-you-legally/',
           'snippet': 'The report noted that in 2007 and 2011, Amazon.com Inc. '
                      'founder Jeff Bezos “did not pay a dime in federal ... '
                      'If you want to bet on Musk, check out Tesla.',
           'date': '11 mins ago',
           'source': 'THE BHARAT EXPRESS NEWS',
           'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR_X9qqSwVFBBdos2CK5ky5IWIE3aJPCQeRYR9O1Jz4t-MjaEYBuwK7AU3AJQ&s',
           'position': 3}]}
Some examples of the tbs parameter:
qdr:h (past hour)
qdr:d (past day)
qdr:w (past week)
qdr:m (past month)
qdr:y (past year)
You can specify intermediate time periods by adding a number:
qdr:h12 (past 12 hours)
qdr:d3 (past 3 days)
qdr:w2 (past 2 weeks)
qdr:m6 (past 6 months)
qdr:m2 (past 2 years)
For all supported filters simply go to Google Search, search for something, click on “Tools”, add your date filter and check the URL for “tbs=”.
Searching for Google Places#
We can also query Google Places using this wrapper. For example:
search = GoogleSerperAPIWrapper(type="places")


results = search.results("Italian restaurants in Upper East Side")
pprint.pp(results)
{'searchParameters': {'q': 'Italian restaurants in Upper East Side',
                      'gl': 'us',
                      'hl': 'en',
                      'num': 10,
                      'type': 'places'},
 'places': [{'position': 1,
             'title': "L'Osteria",
             'address': '1219 Lexington Ave',
             'latitude': 40.777154599999996,
             'longitude': -73.9571363,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNjU7BWEq_aYQANBCbX52Kb0lDpd_lFIx5onw40=w92-h92-n-k-no',
             'rating': 4.7,
             'ratingCount': 91,
             'category': 'Italian'},
            {'position': 2,
             'title': "Tony's Di Napoli",
             'address': '1081 3rd Ave',
             'latitude': 40.7643567,
             'longitude': -73.9642373,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNbNv6jZkJ9nyVi60__8c1DQbe_eEbugRAhIYye=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 2265,
             'category': 'Italian'},
            {'position': 3,
             'title': 'Caravaggio',
             'address': '23 E 74th St',
             'latitude': 40.773412799999996,


'longitude': -73.96473379999999,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPDGchokDvppoLfmVEo6X_bWd3Fz0HyxIHTEe9V=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 276,
             'category': 'Italian'},
            {'position': 4,
             'title': 'Luna Rossa',
             'address': '347 E 85th St',
             'latitude': 40.776593999999996,
             'longitude': -73.950351,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNPCpCPuqPAb1Mv6_fOP7cjb8Wu1rbqbk2sMBlh=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 140,
             'category': 'Italian'},
            {'position': 5,
             'title': "Paola's",
             'address': '1361 Lexington Ave',
             'latitude': 40.7822019,
             'longitude': -73.9534096,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPJr2Vcx-B6K-GNQa4koOTffggTePz8TKRTnWi3=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 344,
             'category': 'Italian'},
            {'position': 6,
             'title': 'Come Prima',
             'address': '903 Madison Ave',


'latitude': 40.772124999999996,
             'longitude': -73.965012,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNrX19G0NVdtDyMovCQ-M-m0c_gLmIxrWDQAAbz=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 176,
             'category': 'Italian'},
            {'position': 7,
             'title': 'Botte UES',
             'address': '1606 1st Ave.',
             'latitude': 40.7750785,
             'longitude': -73.9504801,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPPN5GXxfH3NDacBc0Pt3uGAInd9OChS5isz9RF=w92-h92-n-k-no',
             'rating': 4.4,
             'ratingCount': 152,
             'category': 'Italian'},
            {'position': 8,
             'title': 'Piccola Cucina Uptown',
             'address': '106 E 60th St',
             'latitude': 40.7632468,
             'longitude': -73.9689825,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPifIgzOCD5SjgzzqBzGkdZCBp0MQsK5k7M7znn=w92-h92-n-k-no',
             'rating': 4.6,
             'ratingCount': 941,
             'category': 'Italian'},
            {'position': 9,


'title': 'Pinocchio Restaurant',
             'address': '300 E 92nd St',
             'latitude': 40.781453299999995,
             'longitude': -73.9486788,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNtxlIyEEJHtDtFtTR9nB38S8A2VyMu-mVVz72A=w92-h92-n-k-no',
             'rating': 4.5,
             'ratingCount': 113,
             'category': 'Italian'},
            {'position': 10,
             'title': 'Barbaresco',
             'address': '843 Lexington Ave #1',
             'latitude': 40.7654332,
             'longitude': -73.9656873,
             'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipMb9FbPuXF_r9g5QseOHmReejxSHgSahPMPJ9-8=w92-h92-n-k-no',
             'rating': 4.3,
             'ratingCount': 122,
             'locationHint': 'In The Touraine',
             'category': 'Italian'}]}
previous
Google Search
next
Gradio Tools
 Contents

As part of a Self Ask With Search Chain
Obtaining results with metadata
Searching for Google Images
Searching for Google News
Searching for Google Places
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Wolfram Alpha
Wolfram Alpha#
This notebook goes over how to use the wolfram alpha component.
First, you need to set up your Wolfram Alpha developer account and get your APP ID:
Go to wolfram alpha and sign up for a developer account here
Create an app and get your APP ID
pip install wolframalpha
Then we will need to set some environment variables:
Save your APP ID into WOLFRAM_ALPHA_APPID env variable
pip install wolframalpha
import os
os.environ["WOLFRAM_ALPHA_APPID"] = ""
from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper
wolfram = WolframAlphaAPIWrapper()
wolfram.run("What is 2x+5 = -3x + 7?")
'x = 2/5'
previous
Wikipedia
next
YouTubeSearchTool
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Bing Search
 Contents
Number of results
Metadata Results
Bing Search#
This notebook goes over how to use the bing search component.
First, you need to set up the proper API keys and environment variables. To set it up, follow the instructions found here.
Then we will need to set some environment variables.
import os
os.environ["BING_SUBSCRIPTION_KEY"] = ""
os.environ["BING_SEARCH_URL"] = ""
from langchain.utilities import BingSearchAPIWrapper
search = BingSearchAPIWrapper()
search.run("python")


'Thanks to the flexibility of Python and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with Python by Dan Taylor. Python releases by version number: Release version Release date Click for more. Python 3.11.1 Dec. 6, 2022 Download Release Notes. Python 3.10.9 Dec. 6, 2022 Download Release Notes. Python 3.9.16 Dec. 6, 2022 Download Release Notes. Python 3.8.16 Dec. 6, 2022 Download Release Notes. Python 3.7.16 Dec. 6, 2022 Download Release Notes. In this lesson, we will look at the += operator in Python and see how it works with several simple examples.. The operator ‘+=’ is a shorthand for the addition assignment operator.It adds two values and assigns the sum to a variable (left operand). W3Schools offers free online tutorials, references and exercises in all the major languages of the web. Covering popular subjects like HTML, CSS, JavaScript, Python, SQL, Java, and many, many more. This tutorial introduces the reader informally to the basic concepts and features of the Python language and system. It helps to have a Python interpreter handy for hands-on experience, but all examples are self-contained, so the tutorial can be read off-line as well. For a description of standard objects


and modules, see The Python Standard ... Python is a general-purpose, versatile, and powerful programming language. It's a great first language because Python code is concise and easy to read. Whatever you want to do, python can do it. From web development to machine learning to data science, Python is the language for you. To install Python using the Microsoft Store: Go to your Start menu (lower left Windows icon), type "Microsoft Store", select the link to open the store. Once the store is open, select Search from the upper-right menu and enter "Python". Select which version of Python you would like to use from the results under Apps. Under the “Python Releases for Mac OS X” heading, click the link for the Latest Python 3 Release - Python 3.x.x. As of this writing, the latest version was Python 3.8.4. Scroll to the bottom and click macOS 64-bit installer to start the download. When the installer is finished downloading, move on to the next step. Step 2: Run the Installer'


Number of results#
You can use the k parameter to set the number of results
search = BingSearchAPIWrapper(k=1)
search.run("python")
'Thanks to the flexibility of Python and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with Python by Dan Taylor.'
Metadata Results#
Run query through BingSearch and return snippet, title, and link metadata.
Snippet: The description of the result.
Title: The title of the result.
Link: The link to the result.
search = BingSearchAPIWrapper()
search.results("apples", 5)
[{'snippet': 'Lady Alice. Pink Lady apples aren’t the only lady in the apple family. Lady Alice apples were discovered growing, thanks to bees pollinating, in Washington. They are smaller and slightly more stout in appearance than other varieties. Their skin color appears to have red and yellow stripes running from stem to butt.',
  'title': '25 Types of Apples - Jessica Gavin',
  'link': 'https://www.jessicagavin.com/types-of-apples/'},
 {'snippet': 'Apples can do a lot for you, thanks to plant chemicals called flavonoids. And they have pectin, a fiber that breaks down in your gut. If you take off the apple’s skin before eating it, you won ...',
  'title': 'Apples: Nutrition & Health Benefits - WebMD',
  'link': 'https://www.webmd.com/food-recipes/benefits-apples'},


{'snippet': 'Apples boast many vitamins and minerals, though not in high amounts. However, apples are usually a good source of vitamin C. Vitamin C. Also called ascorbic acid, this vitamin is a common ...',
  'title': 'Apples 101: Nutrition Facts and Health Benefits',
  'link': 'https://www.healthline.com/nutrition/foods/apples'},
 {'snippet': 'Weight management. The fibers in apples can slow digestion, helping one to feel greater satisfaction after eating. After following three large prospective cohorts of 133,468 men and women for 24 years, researchers found that higher intakes of fiber-rich fruits with a low glycemic load, particularly apples and pears, were associated with the least amount of weight gain over time.',
  'title': 'Apples | The Nutrition Source | Harvard T.H. Chan School of Public Health',
  'link': 'https://www.hsph.harvard.edu/nutritionsource/food-features/apples/'}]
previous
Shell Tool
next
Brave Search
 Contents

Number of results
Metadata Results
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Gradio Tools
 Contents
Using a tool
Using within an agent
Gradio Tools#
There are many 1000s of Gradio apps on Hugging Face Spaces. This library puts them at the tips of your LLM’s fingers 🦾
Specifically, gradio-tools is a Python library for converting Gradio apps into tools that can be leveraged by a large language model (LLM)-based agent to complete its task. For example, an LLM could use a Gradio tool to transcribe a voice recording it finds online and then summarize it for you. Or it could use a different Gradio tool to apply OCR to a document on your Google Drive and then answer questions about it.
It’s very easy to create you own tool if you want to use a space that’s not one of the pre-built tools. Please see this section of the gradio-tools documentation for information on how to do that. All contributions are welcome!
# !pip install gradio_tools
Using a tool#
from gradio_tools.tools import StableDiffusionTool
local_file_path = StableDiffusionTool().langchain.run("Please create a photo of a dog riding a skateboard")
local_file_path
Loaded as API: https://gradio-client-demos-stable-diffusion.hf.space ✔
Job Status: Status.STARTING eta: None
'/Users/harrisonchase/workplace/langchain/docs/modules/agents/tools/examples/b61c1dd9-47e2-46f1-a47c-20d27640993d/tmp4ap48vnm.jpg'
from PIL import Image
im = Image.open(local_file_path)
display(im)
Using within an agent#
from langchain.agents import initialize_agent
from langchain.llms import OpenAI


from gradio_tools.tools import (StableDiffusionTool, ImageCaptioningTool, StableDiffusionPromptGeneratorTool,
                                TextToVideoTool)
from langchain.memory import ConversationBufferMemory
llm = OpenAI(temperature=0)
memory = ConversationBufferMemory(memory_key="chat_history")
tools = [StableDiffusionTool().langchain, ImageCaptioningTool().langchain,
         StableDiffusionPromptGeneratorTool().langchain, TextToVideoTool().langchain]
agent = initialize_agent(tools, llm, memory=memory, agent="conversational-react-description", verbose=True)
output = agent.run(input=("Please create a photo of a dog riding a skateboard "
                          "but improve my prompt prior to using an image generator."
                          "Please caption the generated image and create a video for it using the improved prompt."))
Loaded as API: https://gradio-client-demos-stable-diffusion.hf.space ✔
Loaded as API: https://taesiri-blip-2.hf.space ✔
Loaded as API: https://microsoft-promptist.hf.space ✔
Loaded as API: https://damo-vilab-modelscope-text-to-video-synthesis.hf.space ✔
> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: StableDiffusionPromptGenerator
Action Input: A dog riding a skateboard
Job Status: Status.STARTING eta: None
Observation: A dog riding a skateboard, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha
Thought: Do I need to use a tool? Yes
Action: StableDiffusion


Action Input: A dog riding a skateboard, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha
Job Status: Status.STARTING eta: None
Job Status: Status.PROCESSING eta: None
Observation: /Users/harrisonchase/workplace/langchain/docs/modules/agents/tools/examples/2e280ce4-4974-4420-8680-450825c31601/tmpfmiz2g1c.jpg
Thought: Do I need to use a tool? Yes
Action: ImageCaptioner
Action Input: /Users/harrisonchase/workplace/langchain/docs/modules/agents/tools/examples/2e280ce4-4974-4420-8680-450825c31601/tmpfmiz2g1c.jpg
Job Status: Status.STARTING eta: None
Observation: a painting of a dog sitting on a skateboard
Thought: Do I need to use a tool? Yes
Action: TextToVideo
Action Input: a painting of a dog sitting on a skateboard
Job Status: Status.STARTING eta: None
Due to heavy traffic on this app, the prediction will take approximately 73 seconds.For faster predictions without waiting in queue, you may duplicate the space using: Client.duplicate(damo-vilab/modelscope-text-to-video-synthesis)
Job Status: Status.IN_QUEUE eta: 73.89824726581574
Due to heavy traffic on this app, the prediction will take approximately 42 seconds.For faster predictions without waiting in queue, you may duplicate the space using: Client.duplicate(damo-vilab/modelscope-text-to-video-synthesis)
Job Status: Status.IN_QUEUE eta: 42.49370198879602
Job Status: Status.IN_QUEUE eta: 21.314297944849187


Observation: /var/folders/bm/ylzhm36n075cslb9fvvbgq640000gn/T/tmp5snj_nmzf20_cb3m.mp4
Thought: Do I need to use a tool? No
AI: Here is a video of a painting of a dog sitting on a skateboard.
> Finished chain.
previous
Google Serper API
next
GraphQL tool
 Contents

Using a tool
Using within an agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
YouTubeSearchTool
YouTubeSearchTool#
This notebook shows how to use a tool to search YouTube
Adapted from venuv/langchain_yt_tools
#! pip install youtube_search
from langchain.tools import YouTubeSearchTool
tool = YouTubeSearchTool()
tool.run("lex friedman")
"['/watch?v=VcVfceTsD0A&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=gPfriiHBBek&pp=ygUMbGV4IGZyaWVkbWFu']"
You can also specify the number of results that are returned
tool.run("lex friedman,5")
"['/watch?v=VcVfceTsD0A&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=YVJ8gTnDC4Y&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=Udh22kuLebg&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=gPfriiHBBek&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=L_Guz73e6fw&pp=ygUMbGV4IGZyaWVkbWFu']"
previous
Wolfram Alpha
next
Zapier Natural Language Actions API
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
SceneXplain
 Contents
Usage in an Agent
SceneXplain#
SceneXplain is an ImageCaptioning service accessible through the SceneXplain Tool.
To use this tool, you’ll need to make an account and fetch your API Token from the website. Then you can instantiate the tool.
import os
os.environ["SCENEX_API_KEY"] = "<YOUR_API_KEY>"
from langchain.agents import load_tools
tools = load_tools(["sceneXplain"])
Or directly instantiate the tool.
from langchain.tools import SceneXplainTool
tool = SceneXplainTool()
Usage in an Agent#
The tool can be used in any LangChain agent as follows:
from langchain.llms import OpenAI
from langchain.agents import initialize_agent
from langchain.memory import ConversationBufferMemory
llm = OpenAI(temperature=0)
memory = ConversationBufferMemory(memory_key="chat_history")
agent = initialize_agent(
    tools, llm, memory=memory, agent="conversational-react-description", verbose=True
)
output = agent.run(
    input=(
        "What is in this image https://storage.googleapis.com/causal-diffusion.appspot.com/imagePrompts%2F0rw369i5h9t%2Foriginal.png. "
        "Is it movie or a game? If it is a movie, what is the name of the movie?"
    )
)
print(output)
> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: Image Explainer
Action Input: https://storage.googleapis.com/causal-diffusion.appspot.com/imagePrompts%2F0rw369i5h9t%2Foriginal.png


Observation: In a charmingly whimsical scene, a young girl is seen braving the rain alongside her furry companion, the lovable Totoro. The two are depicted standing on a bustling street corner, where they are sheltered from the rain by a bright yellow umbrella. The girl, dressed in a cheerful yellow frock, holds onto the umbrella with both hands while gazing up at Totoro with an expression of wonder and delight.
Totoro, meanwhile, stands tall and proud beside his young friend, holding his own umbrella aloft to protect them both from the downpour. His furry body is rendered in rich shades of grey and white, while his large ears and wide eyes lend him an endearing charm.
In the background of the scene, a street sign can be seen jutting out from the pavement amidst a flurry of raindrops. A sign with Chinese characters adorns its surface, adding to the sense of cultural diversity and intrigue. Despite the dreary weather, there is an undeniable sense of joy and camaraderie in this heartwarming image.
Thought: Do I need to use a tool? No
AI: This image appears to be a still from the 1988 Japanese animated fantasy film My Neighbor Totoro. The film follows two young girls, Satsuki and Mei, as they explore the countryside and befriend the magical forest spirits, including the titular character Totoro.
> Finished chain.
This image appears to be a still from the 1988 Japanese animated fantasy film My Neighbor Totoro. The film follows two young girls, Satsuki and Mei, as they explore the countryside and befriend the magical forest spirits, including the titular character Totoro.
previous
Requests
next
Search Tools
 Contents

Usage in an Agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.


Last updated on Jun 04, 2023.


.ipynb
.pdf
OpenWeatherMap API
 Contents
Use the wrapper
Use the tool
OpenWeatherMap API#
This notebook goes over how to use the OpenWeatherMap component to fetch weather information.
First, you need to sign up for an OpenWeatherMap API key:
Go to OpenWeatherMap and sign up for an API key here
pip install pyowm
Then we will need to set some environment variables:
Save your API KEY into OPENWEATHERMAP_API_KEY env variable
Use the wrapper#
from langchain.utilities import OpenWeatherMapAPIWrapper
import os
os.environ["OPENWEATHERMAP_API_KEY"] = ""
weather = OpenWeatherMapAPIWrapper()
weather_data = weather.run("London,GB")
print(weather_data)
In London,GB, the current weather is as follows:
Detailed status: broken clouds
Wind speed: 2.57 m/s, direction: 240°
Humidity: 55%
Temperature:
  - Current: 20.12°C
  - High: 21.75°C
  - Low: 18.68°C
  - Feels like: 19.62°C
Rain: {}
Heat index: None
Cloud cover: 75%
Use the tool#
from langchain.llms import OpenAI
from langchain.agents import load_tools, initialize_agent, AgentType
import os
os.environ["OPENAI_API_KEY"] = ""
os.environ["OPENWEATHERMAP_API_KEY"] = ""
llm = OpenAI(temperature=0)
tools = load_tools(["openweathermap-api"], llm)
agent_chain = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)


agent_chain.run("What's the weather like in London?")
> Entering new AgentExecutor chain...
 I need to find out the current weather in London.
Action: OpenWeatherMap
Action Input: London,GB
Observation: In London,GB, the current weather is as follows:
Detailed status: broken clouds
Wind speed: 2.57 m/s, direction: 240°
Humidity: 56%
Temperature:
  - Current: 20.11°C
  - High: 21.75°C
  - Low: 18.68°C
  - Feels like: 19.64°C
Rain: {}
Heat index: None
Cloud cover: 75%
Thought: I now know the current weather in London.
Final Answer: The current weather in London is broken clouds, with a wind speed of 2.57 m/s, direction 240°, humidity of 56%, temperature of 20.11°C, high of 21.75°C, low of 18.68°C, and a heat index of None.
> Finished chain.
'The current weather in London is broken clouds, with a wind speed of 2.57 m/s, direction 240°, humidity of 56%, temperature of 20.11°C, high of 21.75°C, low of 18.68°C, and a heat index of None.'
previous
Metaphor Search
next
PubMed Tool
 Contents

Use the wrapper
Use the tool
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
IFTTT WebHooks
 Contents
Creating a webhook
Configuring the “If This”
Configuring the “Then That”
Finishing up
IFTTT WebHooks#
This notebook shows how to use IFTTT Webhooks.
From https://github.com/SidU/teams-langchain-js/wiki/Connecting-IFTTT-Services.
Creating a webhook#
Go to https://ifttt.com/create
Configuring the “If This”#
Click on the “If This” button in the IFTTT interface.
Search for “Webhooks” in the search bar.
Choose the first option for “Receive a web request with a JSON payload.”
Choose an Event Name that is specific to the service you plan to connect to.
This will make it easier for you to manage the webhook URL.
For example, if you’re connecting to Spotify, you could use “Spotify” as your
Event Name.
Click the “Create Trigger” button to save your settings and create your webhook.
Configuring the “Then That”#
Tap on the “Then That” button in the IFTTT interface.
Search for the service you want to connect, such as Spotify.
Choose an action from the service, such as “Add track to a playlist”.
Configure the action by specifying the necessary details, such as the playlist name,
e.g., “Songs from AI”.
Reference the JSON Payload received by the Webhook in your action. For the Spotify
scenario, choose “{{JsonPayload}}” as your search query.
Tap the “Create Action” button to save your action settings.
Once you have finished configuring your action, click the “Finish” button to
complete the setup.
Congratulations! You have successfully connected the Webhook to the desired
service, and you’re ready to start receiving data and triggering actions 🎉


Finishing up#
To get your webhook URL go to https://ifttt.com/maker_webhooks/settings
Copy the IFTTT key value from there. The URL is of the form
https://maker.ifttt.com/use/YOUR_IFTTT_KEY. Grab the YOUR_IFTTT_KEY value.
from langchain.tools.ifttt import IFTTTWebhook
import os
key = os.environ["IFTTTKey"]
url = f"https://maker.ifttt.com/trigger/spotify/json/with/key/{key}"
tool = IFTTTWebhook(name="Spotify", description="Add a song to spotify playlist", url=url)
tool.run("taylor swift")
"Congratulations! You've fired the spotify JSON event"
previous
Human as a tool
next
Metaphor Search
 Contents

Creating a webhook
Configuring the “If This”
Configuring the “Then That”
Finishing up
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Metaphor Search
 Contents
Metaphor Search
Call the API
Use Metaphor as a tool
Metaphor Search#
This notebook goes over how to use Metaphor search.
First, you need to set up the proper API keys and environment variables. Request an API key [here](Sign up for early access here).
Then enter your API key as an environment variable.
import os
os.environ["METAPHOR_API_KEY"] = ""
from langchain.utilities import MetaphorSearchAPIWrapper
search = MetaphorSearchAPIWrapper()
Call the API#
results takes in a Metaphor-optimized search query and a number of results (up to 500). It returns a list of results with title, url, author, and creation date.
search.results("The best blog post about AI safety is definitely this: ", 10)


{'results': [{'url': 'https://www.anthropic.com/index/core-views-on-ai-safety', 'title': 'Core Views on AI Safety: When, Why, What, and How', 'dateCreated': '2023-03-08', 'author': None, 'score': 0.1998831331729889}, {'url': 'https://aisafety.wordpress.com/', 'title': 'Extinction Risk from Artificial Intelligence', 'dateCreated': '2013-10-08', 'author': None, 'score': 0.19801370799541473}, {'url': 'https://www.lesswrong.com/posts/WhNxG4r774bK32GcH/the-simple-picture-on-ai-safety', 'title': 'The simple picture on AI safety - LessWrong', 'dateCreated': '2018-05-27', 'author': 'Alex Flint', 'score': 0.19735534489154816}, {'url': 'https://slatestarcodex.com/2015/05/29/no-time-like-the-present-for-ai-safety-work/', 'title': 'No Time Like The Present For AI Safety Work', 'dateCreated': '2015-05-29', 'author': None, 'score': 0.19408763945102692}, {'url': 'https://www.lesswrong.com/posts/5BJvusxdwNXYQ4L9L/so-you-want-to-save-the-world', 'title': 'So You Want to Save the World - LessWrong', 'dateCreated': '2012-01-01', 'author': 'Lukeprog', 'score': 0.18853715062141418}, {'url': 'https://openai.com/blog/planning-for-agi-and-beyond', 'title': 'Planning for AGI and beyond', 'dateCreated':


'2023-02-24', 'author': 'Authors', 'score': 0.18665121495723724}, {'url': 'https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html', 'title': 'The Artificial Intelligence Revolution: Part 1 - Wait But Why', 'dateCreated': '2015-01-22', 'author': 'Tim Urban', 'score': 0.18604731559753418}, {'url': 'https://forum.effectivealtruism.org/posts/uGDCaPFaPkuxAowmH/anthropic-core-views-on-ai-safety-when-why-what-and-how', 'title': 'Anthropic: Core Views on AI Safety: When, Why, What, and How - EA Forum', 'dateCreated': '2023-03-09', 'author': 'Jonmenaster', 'score': 0.18415069580078125}, {'url': 'https://www.lesswrong.com/posts/xBrpph9knzWdtMWeQ/the-proof-of-doom', 'title': 'The Proof of Doom - LessWrong', 'dateCreated': '2022-03-09', 'author': 'Johnlawrenceaspden', 'score': 0.18159329891204834}, {'url': 'https://intelligence.org/why-ai-safety/', 'title': 'Why AI Safety? - Machine Intelligence Research Institute', 'dateCreated': '2017-03-01', 'author': None, 'score': 0.1814115345478058}]}


[{'title': 'Core Views on AI Safety: When, Why, What, and How',
  'url': 'https://www.anthropic.com/index/core-views-on-ai-safety',
  'author': None,
  'date_created': '2023-03-08'},
 {'title': 'Extinction Risk from Artificial Intelligence',
  'url': 'https://aisafety.wordpress.com/',
  'author': None,
  'date_created': '2013-10-08'},
 {'title': 'The simple picture on AI safety - LessWrong',
  'url': 'https://www.lesswrong.com/posts/WhNxG4r774bK32GcH/the-simple-picture-on-ai-safety',
  'author': 'Alex Flint',
  'date_created': '2018-05-27'},
 {'title': 'No Time Like The Present For AI Safety Work',
  'url': 'https://slatestarcodex.com/2015/05/29/no-time-like-the-present-for-ai-safety-work/',
  'author': None,
  'date_created': '2015-05-29'},
 {'title': 'So You Want to Save the World - LessWrong',
  'url': 'https://www.lesswrong.com/posts/5BJvusxdwNXYQ4L9L/so-you-want-to-save-the-world',
  'author': 'Lukeprog',
  'date_created': '2012-01-01'},
 {'title': 'Planning for AGI and beyond',
  'url': 'https://openai.com/blog/planning-for-agi-and-beyond',
  'author': 'Authors',
  'date_created': '2023-02-24'},


{'title': 'The Artificial Intelligence Revolution: Part 1 - Wait But Why',
  'url': 'https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html',
  'author': 'Tim Urban',
  'date_created': '2015-01-22'},
 {'title': 'Anthropic: Core Views on AI Safety: When, Why, What, and How - EA Forum',
  'url': 'https://forum.effectivealtruism.org/posts/uGDCaPFaPkuxAowmH/anthropic-core-views-on-ai-safety-when-why-what-and-how',
  'author': 'Jonmenaster',
  'date_created': '2023-03-09'},
 {'title': 'The Proof of Doom - LessWrong',
  'url': 'https://www.lesswrong.com/posts/xBrpph9knzWdtMWeQ/the-proof-of-doom',
  'author': 'Johnlawrenceaspden',
  'date_created': '2022-03-09'},
 {'title': 'Why AI Safety? - Machine Intelligence Research Institute',
  'url': 'https://intelligence.org/why-ai-safety/',
  'author': None,
  'date_created': '2017-03-01'}]
Use Metaphor as a tool#
Metaphor can be used as a tool that gets URLs that other tools such as browsing tools.
from langchain.agents.agent_toolkits import PlayWrightBrowserToolkit
from langchain.tools.playwright.utils import (
    create_async_playwright_browser,# A synchronous browser is available, though it isn't compatible with jupyter.
)
async_browser = create_async_playwright_browser()
toolkit = PlayWrightBrowserToolkit.from_browser(async_browser=async_browser)


tools = toolkit.get_tools()
tools_by_name = {tool.name: tool for tool in tools}
print(tools_by_name.keys())
navigate_tool = tools_by_name["navigate_browser"]
extract_text = tools_by_name["extract_text"]
from langchain.agents import initialize_agent, AgentType
from langchain.chat_models import ChatOpenAI
from langchain.tools import MetaphorSearchResults
llm = ChatOpenAI(model_name="gpt-4", temperature=0.7)
metaphor_tool = MetaphorSearchResults(api_wrapper=search)
agent_chain = initialize_agent([metaphor_tool, extract_text, navigate_tool], llm, agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
agent_chain.run("find me an interesting tweet about AI safety using Metaphor, then tell me the first sentence in the post. Do not finish until able to retrieve the first sentence.")
> Entering new AgentExecutor chain...
Thought: I need to find a tweet about AI safety using Metaphor Search.
Action:
```
{
  "action": "Metaphor Search Results JSON",
  "action_input": {
    "query": "interesting tweet AI safety",
    "num_results": 1
  }
}
```
{'results': [{'url': 'https://safe.ai/', 'title': 'Center for AI Safety', 'dateCreated': '2022-01-01', 'author': None, 'score': 0.18083244562149048}]}
Observation: [{'title': 'Center for AI Safety', 'url': 'https://safe.ai/', 'author': None, 'date_created': '2022-01-01'}]
Thought:I need to navigate to the URL provided in the search results to find the tweet.
> Finished chain.


'I need to navigate to the URL provided in the search results to find the tweet.'
previous
IFTTT WebHooks
next
OpenWeatherMap API
 Contents

Metaphor Search
Call the API
Use Metaphor as a tool
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
SerpAPI
 Contents
Custom Parameters
SerpAPI#
This notebook goes over how to use the SerpAPI component to search the web.
from langchain.utilities import SerpAPIWrapper
search = SerpAPIWrapper()
search.run("Obama's first name?")
'Barack Hussein Obama II'
Custom Parameters#
You can also customize the SerpAPI wrapper with arbitrary parameters. For example, in the below example we will use bing instead of google.
params = {
    "engine": "bing",
    "gl": "us",
    "hl": "en",
}
search = SerpAPIWrapper(params=params)
search.run("Obama's first name?")
'Barack Hussein Obama II is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, Obama was the first African-American presi…New content will be added above the current area of focus upon selectionBarack Hussein Obama II is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, Obama was the first African-American president of the United States. He previously served as a U.S. senator from Illinois from 2005 to 2008 and as an Illinois state senator from 1997 to 2004, and previously worked as a civil rights lawyer before entering politics.Wikipediabarackobama.com'
from langchain.agents import Tool
# You can create the tool to pass to an agent
repl_tool = Tool(
    name="python_repl",


description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",
    func=search.run,
)
previous
SearxNG Search API
next
Twilio
 Contents

Custom Parameters
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Human as a tool
 Contents
Configuring the Input Function
Human as a tool#
Human are AGI so they can certainly be used as a tool to help out AI agent
when it is confused.
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
llm = ChatOpenAI(temperature=0.0)
math_llm = OpenAI(temperature=0.0)
tools = load_tools(
    ["human", "llm-math"],
    llm=math_llm,
)
agent_chain = initialize_agent(
    tools,
    llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
)
In the above code you can see the tool takes input directly from command line.
You can customize prompt_func and input_func according to your need (as shown below).
agent_chain.run("What's my friend Eric's surname?")
# Answer with 'Zhu'
> Entering new AgentExecutor chain...
I don't know Eric's surname, so I should ask a human for guidance.
Action: Human
Action Input: "What is Eric's surname?"
What is Eric's surname?
 Zhu
Observation: Zhu
Thought:I now know Eric's surname is Zhu.
Final Answer: Eric's surname is Zhu.
> Finished chain.
"Eric's surname is Zhu."
Configuring the Input Function#
By default, the HumanInputRun tool uses the python input function to get input from the user.
You can customize the input_func to be anything you’d like.
For instance, if you want to accept multi-line input, you could do the following:
def get_input() -> str:


print("Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.")
    contents = []
    while True:
        try:
            line = input()
        except EOFError:
            break
        if line == "q":
            break
        contents.append(line)
    return "\n".join(contents)
# You can modify the tool when loading
tools = load_tools(
    ["human", "ddg-search"],
    llm=math_llm,
    input_func=get_input
)
# Or you can directly instantiate the tool
from langchain.tools import HumanInputRun
tool = HumanInputRun(input_func=get_input)
agent_chain = initialize_agent(
    tools,
    llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
)
agent_chain.run("I need help attributing a quote")
> Entering new AgentExecutor chain...
I should ask a human for guidance
Action: Human
Action Input: "Can you help me attribute a quote?"
Can you help me attribute a quote?
Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.
 vini
 vidi
 vici
 q
Observation: vini
vidi
vici
Thought:I need to provide more context about the quote
Action: Human
Action Input: "The quote is 'Veni, vidi, vici'"
The quote is 'Veni, vidi, vici'
Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.
 oh who said it
 q
Observation: oh who said it
Thought:I can use DuckDuckGo Search to find out who said the quote


Action: DuckDuckGo Search
Action Input: "Who said 'Veni, vidi, vici'?"


Observation: Updated on September 06, 2019. "Veni, vidi, vici" is a famous phrase said to have been spoken by the Roman Emperor Julius Caesar (100-44 BCE) in a bit of stylish bragging that impressed many of the writers of his day and beyond. The phrase means roughly "I came, I saw, I conquered" and it could be pronounced approximately Vehnee, Veedee ... Veni, vidi, vici (Classical Latin: [weːniː wiːdiː wiːkiː], Ecclesiastical Latin: [ˈveni ˈvidi ˈvitʃi]; "I came; I saw; I conquered") is a Latin phrase used to refer to a swift, conclusive victory.The phrase is popularly attributed to Julius Caesar who, according to Appian, used the phrase in a letter to the Roman Senate around 47 BC after he had achieved a quick victory in his short ... veni, vidi, vici Latin quotation from Julius Caesar ve· ni, vi· di, vi· ci ˌwā-nē ˌwē-dē ˈwē-kē ˌvā-nē ˌvē-dē ˈvē-chē : I came, I saw, I conquered Articles Related to veni, vidi, vici 'In Vino Veritas' and Other Latin... Dictionary Entries Near veni, vidi, vici Venite veni, vidi, vici Venizélos See More Nearby Entries Cite this Entry Style The simplest explanation for why veni, vidi, vici is a popular saying is that it comes from Julius Caesar, one of history's most famous figures, and has a simple, strong meaning: I'm powerful and fast. But it's not just the meaning that makes


the phrase so powerful. Caesar was a gifted writer, and the phrase makes use of Latin grammar to ... One of the best known and most frequently quoted Latin expression, veni, vidi, vici may be found hundreds of times throughout the centuries used as an expression of triumph. The words are said to have been used by Caesar as he was enjoying a triumph.


Thought:I now know the final answer
Final Answer: Julius Caesar said the quote "Veni, vidi, vici" which means "I came, I saw, I conquered".
> Finished chain.
'Julius Caesar said the quote "Veni, vidi, vici" which means "I came, I saw, I conquered".'
previous
HuggingFace Tools
next
IFTTT WebHooks
 Contents

Configuring the Input Function
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
HuggingFace Tools
HuggingFace Tools#
Huggingface Tools supporting text I/O can be
loaded directly using the load_huggingface_tool function.
# Requires transformers>=4.29.0 and huggingface_hub>=0.14.1
!pip install --upgrade transformers huggingface_hub > /dev/null
from langchain.agents import load_huggingface_tool
tool = load_huggingface_tool("lysandre/hf-model-downloads")
print(f"{tool.name}: {tool.description}")
model_download_counter: This is a tool that returns the most downloaded model of a given task on the Hugging Face Hub. It takes the name of the category (such as text-classification, depth-estimation, etc), and returns the name of the checkpoint
tool.run("text-classification")
'facebook/bart-large-mnli'
previous
GraphQL tool
next
Human as a tool
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
ArXiv API Tool
 Contents
The ArXiv API Wrapper
ArXiv API Tool#
This notebook goes over how to use the arxiv component.
First, you need to install arxiv python package.
!pip install arxiv
from langchain.chat_models import ChatOpenAI
from langchain.agents import load_tools, initialize_agent, AgentType
llm = ChatOpenAI(temperature=0.0)
tools = load_tools(
    ["arxiv"],
)
agent_chain = initialize_agent(
    tools,
    llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
)
agent_chain.run(
    "What's the paper 1605.08386 about?",
)
> Entering new AgentExecutor chain...
I need to use Arxiv to search for the paper.
Action: Arxiv
Action Input: "1605.08386"
Observation: Published: 2016-05-26
Title: Heat-bath random walks with Markov bases
Authors: Caprice Stanley, Tobias Windisch
Summary: Graphs on lattice points are studied whose edges come from a finite set of
allowed moves of arbitrary length. We show that the diameter of these graphs on
fibers of a fixed integer matrix can be bounded from above by a constant. We
then study the mixing behaviour of heat-bath random walks on these graphs. We
also state explicit conditions on the set of moves so that the heat-bath random
walk, a generalization of the Glauber dynamics, is an expander in fixed
dimension.
Thought:The paper is about heat-bath random walks with Markov bases on graphs of lattice points.


Final Answer: The paper 1605.08386 is about heat-bath random walks with Markov bases on graphs of lattice points.
> Finished chain.
'The paper 1605.08386 is about heat-bath random walks with Markov bases on graphs of lattice points.'
The ArXiv API Wrapper#
The tool wraps the API Wrapper. Below, we can explore some of the features it provides.
from langchain.utilities import ArxivAPIWrapper
Run a query to get information about some scientific article/articles. The query text is limited to 300 characters.
It returns these article fields:
Publishing date
Title
Authors
Summary
Next query returns information about one article with arxiv Id equal “1605.08386”.
arxiv = ArxivAPIWrapper()
docs = arxiv.run("1605.08386")
docs
'Published: 2016-05-26\nTitle: Heat-bath random walks with Markov bases\nAuthors: Caprice Stanley, Tobias Windisch\nSummary: Graphs on lattice points are studied whose edges come from a finite set of\nallowed moves of arbitrary length. We show that the diameter of these graphs on\nfibers of a fixed integer matrix can be bounded from above by a constant. We\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\nalso state explicit conditions on the set of moves so that the heat-bath random\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\ndimension.'
Now, we want to get information about one author, Caprice Stanley.
This query returns information about three articles. By default, the query returns information only about three top articles.
docs = arxiv.run("Caprice Stanley")
docs


'Published: 2017-10-10\nTitle: On Mixing Behavior of a Family of Random Walks Determined by a Linear Recurrence\nAuthors: Caprice Stanley, Seth Sullivant\nSummary: We study random walks on the integers mod $G_n$ that are determined by an\ninteger sequence $\\{ G_n \\}_{n \\geq 1}$ generated by a linear recurrence\nrelation. Fourier analysis provides explicit formulas to compute the\neigenvalues of the transition matrices and we use this to bound the mixing time\nof the random walks.\n\nPublished: 2016-05-26\nTitle: Heat-bath random walks with Markov bases\nAuthors: Caprice Stanley, Tobias Windisch\nSummary: Graphs on lattice points are studied whose edges come from a finite set of\nallowed moves of arbitrary length. We show that the diameter of these graphs on\nfibers of a fixed integer matrix can be bounded from above by a constant. We\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\nalso state explicit conditions on the set of moves so that the heat-bath random\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\ndimension.\n\nPublished: 2003-03-18\nTitle: Calculation of fluxes of charged particles and neutrinos from atmospheric showers\nAuthors: V. Plyaskin\nSummary: The results on the fluxes of charged particles and neutrinos from a\n3-dimensional (3D) simulation of atmospheric showers are presented. An\nagreement of calculated fluxes with data on charged particles from the AMS and\nCAPRICE detectors is demonstrated. Predictions on neutrino fluxes at different\nexperimental sites are compared with results from other calculations.'


Now, we are trying to find information about non-existing article. In this case, the response is “No good Arxiv Result was found”
docs = arxiv.run("1605.08386WWW")
docs
'No good Arxiv Result was found'
previous
Apify
next
AWS Lambda API
 Contents

The ArXiv API Wrapper
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
ChatGPT Plugins
ChatGPT Plugins#
This example shows how to use ChatGPT Plugins within LangChain abstractions.
Note 1: This currently only works for plugins with no auth.
Note 2: There are almost certainly other ways to do this, this is just a first pass. If you have better ideas, please open a PR!
from langchain.chat_models import ChatOpenAI
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.tools import AIPluginTool
tool = AIPluginTool.from_plugin_url("https://www.klarna.com/.well-known/ai-plugin.json")
llm = ChatOpenAI(temperature=0)
tools = load_tools(["requests_all"] )
tools += [tool]
agent_chain = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
agent_chain.run("what t shirts are available in klarna?")
> Entering new AgentExecutor chain...
I need to check the Klarna Shopping API to see if it has information on available t shirts.
Action: KlarnaProducts
Action Input: None
Observation: Usage Guide: Use the Klarna plugin to get relevant product suggestions for any shopping or researching purpose. The query to be sent should not include stopwords like articles, prepositions and determinants. The api works best when searching for words that are related to products, like their name, brand, model or category. Links will always be returned and should be shown to the user.


OpenAPI Spec: {'openapi': '3.0.1', 'info': {'version': 'v0', 'title': 'Open AI Klarna product Api'}, 'servers': [{'url': 'https://www.klarna.com/us/shopping'}], 'tags': [{'name': 'open-ai-product-endpoint', 'description': 'Open AI Product Endpoint. Query for products.'}], 'paths': {'/public/openai/v0/products': {'get': {'tags': ['open-ai-product-endpoint'], 'summary': 'API for fetching Klarna product information', 'operationId': 'productsUsingGET', 'parameters': [{'name': 'q', 'in': 'query', 'description': 'query, must be between 2 and 100 characters', 'required': True, 'schema': {'type': 'string'}}, {'name': 'size', 'in': 'query', 'description': 'number of products returned', 'required': False, 'schema': {'type': 'integer'}}, {'name': 'budget', 'in': 'query', 'description': 'maximum price of the matching product in local currency, filters results', 'required': False, 'schema': {'type': 'integer'}}], 'responses': {'200': {'description': 'Products found', 'content': {'application/json': {'schema': {'$ref': '#/components/schemas/ProductResponse'}}}}, '503': {'description': 'one or more services are unavailable'}}, 'deprecated': False}}}, 'components': {'schemas': {'Product': {'type': 'object', 'properties': {'attributes': {'type': 'array', 'items': {'type': 'string'}}, 'name': {'type': 'string'}, 'price': {'type': 'string'}, 'url': {'type': 'string'}}, 'title': 'Product'}, 'ProductResponse': {'type': 'object', 'properties':


{'products': {'type': 'array', 'items': {'$ref': '#/components/schemas/Product'}}}, 'title': 'ProductResponse'}}}}


Thought:I need to use the Klarna Shopping API to search for t shirts.
Action: requests_get
Action Input: https://www.klarna.com/us/shopping/public/openai/v0/products?q=t%20shirts


Observation: {"products":[{"name":"Lacoste Men's Pack of Plain T-Shirts","url":"https://www.klarna.com/us/shopping/pl/cl10001/3202043025/Clothing/Lacoste-Men-s-Pack-of-Plain-T-Shirts/?utm_source=openai","price":"$26.60","attributes":["Material:Cotton","Target Group:Man","Color:White,Black"]},{"name":"Hanes Men's Ultimate 6pk. Crewneck T-Shirts","url":"https://www.klarna.com/us/shopping/pl/cl10001/3201808270/Clothing/Hanes-Men-s-Ultimate-6pk.-Crewneck-T-Shirts/?utm_source=openai","price":"$13.82","attributes":["Material:Cotton","Target Group:Man","Color:White"]},{"name":"Nike Boy's Jordan Stretch T-shirts","url":"https://www.klarna.com/us/shopping/pl/cl359/3201863202/Children-s-Clothing/Nike-Boy-s-Jordan-Stretch-T-shirts/?utm_source=openai","price":"$14.99","attributes":["Material:Cotton","Color:White,Green","Model:Boy","Size (Small-Large):S,XL,L,M"]},{"name":"Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack","url":"https://www.klarna.com/us/shopping/pl/cl10001/3203028500/Clothing/Polo-Classic-Fit-Cotton-V-Neck-T-Shirts-3-Pack/?utm_source=openai","price":"$29.95","attributes":["Material:Cotton","Target Group:Man","Color:White,Blue,Black"]},{"name":"adidas Comfort T-shirts Men's


Thought:The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.
Final Answer: The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.
> Finished chain.
"The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack."
previous
Brave Search
next
DuckDuckGo Search
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom LLM Agent
 Contents
Set up environment
Set up tool
Prompt Template
Output Parser
Set up LLM
Define the stop sequence
Set up the Agent
Use the Agent
Adding Memory
Custom LLM Agent#
This notebook goes through how to create your own custom LLM agent.
An LLM agent consists of three parts:
PromptTemplate: This is the prompt template that can be used to instruct the language model on what to do
LLM: This is the language model that powers the agent
stop sequence: Instructs the LLM to stop generating as soon as this string is found
OutputParser: This determines how to parse the LLMOutput into an AgentAction or AgentFinish object
The LLMAgent is used in an AgentExecutor. This AgentExecutor can largely be thought of as a loop that:
Passes user input and any previous steps to the Agent (in this case, the LLMAgent)
If the Agent returns an AgentFinish, then return that directly to the user
If the Agent returns an AgentAction, then use that to call a tool and get an Observation
Repeat, passing the AgentAction and Observation back to the Agent until an AgentFinish is emitted.
AgentAction is a response that consists of action and action_input. action refers to which tool to use, and action_input refers to the input to that tool. log can also be provided as more context (that can be used for logging, tracing, etc).
AgentFinish is a response that contains the final message to be sent back to the user. This should be used to end an agent run.
In this notebook we walk through how to create a custom LLM agent.
Set up environment#
Do necessary imports, etc.
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from langchain.prompts import StringPromptTemplate


from langchain import OpenAI, SerpAPIWrapper, LLMChain
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish
import re
Set up tool#
Set up any tools the agent may want to use. This may be necessary to put in the prompt (so that the agent knows to use these tools).
# Define which tools the agent can use to answer user queries
search = SerpAPIWrapper()
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events"
    )
]
Prompt Template#
This instructs the agent on what to do. Generally, the template should incorporate:
tools: which tools the agent has access and how and when to call them.
intermediate_steps: These are tuples of previous (AgentAction, Observation) pairs. These are generally not passed directly to the model, but the prompt template formats them in a specific way.
input: generic user input
# Set up the base template
template = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s
Question: {input}
{agent_scratchpad}"""


# Set up a prompt template
class CustomPromptTemplate(StringPromptTemplate):
    # The template to use
    template: str
    # The list of tools available
    tools: List[Tool]

    def format(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        return self.template.format(**kwargs)
prompt = CustomPromptTemplate(
    template=template,
    tools=tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically
    # This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps"]
)
Output Parser#
The output parser is responsible for parsing the LLM output into AgentAction and AgentFinish. This usually depends heavily on the prompt used.
This is where you can change the parsing to do retries, handle whitespace, etc
class CustomOutputParser(AgentOutputParser):

    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:


# Check if agent should finish
        if "Final Answer:" in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            raise ValueError(f"Could not parse LLM output: `{llm_output}`")
        action = match.group(1).strip()
        action_input = match.group(2)
        # Return the action and action input
        return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
output_parser = CustomOutputParser()
Set up LLM#
Choose the LLM you want to use!
llm = OpenAI(temperature=0)
Define the stop sequence#
This is important because it tells the LLM when to stop generation.
This depends heavily on the prompt and model you are using. Generally, you want this to be whatever token you use in the prompt to denote the start of an Observation (otherwise, the LLM may hallucinate an observation for you).
Set up the Agent#
We can now combine everything to set up our agent
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]


agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
Use the Agent#
Now we can use it!
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("How many people live in canada as of 2023?")
> Entering new AgentExecutor chain...
Thought: I need to find out the population of Canada in 2023
Action: Search
Action Input: Population of Canada in 2023
Observation:The current population of Canada is 38,658,314 as of Wednesday, April 12, 2023, based on Worldometer elaboration of the latest United Nations data. I now know the final answer
Final Answer: Arrr, there be 38,658,314 people livin' in Canada as of 2023!
> Finished chain.
"Arrr, there be 38,658,314 people livin' in Canada as of 2023!"
Adding Memory#
If you want to add memory to the agent, you’ll need to:
Add a place in the custom prompt for the chat_history
Add a memory object to the agent executor.
# Set up the base template
template_with_history = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action


... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s
Previous conversation history:
{history}
New question: {input}
{agent_scratchpad}"""
prompt_with_history = CustomPromptTemplate(
    template=template_with_history,
    tools=tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically
    # This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps", "history"]
)
llm_chain = LLMChain(llm=llm, prompt=prompt_with_history)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
from langchain.memory import ConversationBufferWindowMemory
memory=ConversationBufferWindowMemory(k=2)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True, memory=memory)
agent_executor.run("How many people live in canada as of 2023?")
> Entering new AgentExecutor chain...
Thought: I need to find out the population of Canada in 2023
Action: Search
Action Input: Population of Canada in 2023
Observation:The current population of Canada is 38,658,314 as of Wednesday, April 12, 2023, based on Worldometer elaboration of the latest United Nations data. I now know the final answer


Final Answer: Arrr, there be 38,658,314 people livin' in Canada as of 2023!
> Finished chain.
"Arrr, there be 38,658,314 people livin' in Canada as of 2023!"
agent_executor.run("how about in mexico?")
> Entering new AgentExecutor chain...
Thought: I need to find out how many people live in Mexico.
Action: Search
Action Input: How many people live in Mexico as of 2023?
Observation:The current population of Mexico is 132,679,922 as of Tuesday, April 11, 2023, based on Worldometer elaboration of the latest United Nations data. Mexico 2020 ... I now know the final answer.
Final Answer: Arrr, there be 132,679,922 people livin' in Mexico as of 2023!
> Finished chain.
"Arrr, there be 132,679,922 people livin' in Mexico as of 2023!"
previous
Custom Agent
next
Custom LLM Agent (with a ChatModel)
 Contents

Set up environment
Set up tool
Prompt Template
Output Parser
Set up LLM
Define the stop sequence
Set up the Agent
Use the Agent
Adding Memory
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom LLM Agent (with a ChatModel)
 Contents
Set up environment
Set up tool
Prompt Template
Output Parser
Set up LLM
Define the stop sequence
Set up the Agent
Use the Agent
Custom LLM Agent (with a ChatModel)#
This notebook goes through how to create your own custom agent based on a chat model.
An LLM chat agent consists of three parts:
PromptTemplate: This is the prompt template that can be used to instruct the language model on what to do
ChatModel: This is the language model that powers the agent
stop sequence: Instructs the LLM to stop generating as soon as this string is found
OutputParser: This determines how to parse the LLMOutput into an AgentAction or AgentFinish object
The LLMAgent is used in an AgentExecutor. This AgentExecutor can largely be thought of as a loop that:
Passes user input and any previous steps to the Agent (in this case, the LLMAgent)
If the Agent returns an AgentFinish, then return that directly to the user
If the Agent returns an AgentAction, then use that to call a tool and get an Observation
Repeat, passing the AgentAction and Observation back to the Agent until an AgentFinish is emitted.
AgentAction is a response that consists of action and action_input. action refers to which tool to use, and action_input refers to the input to that tool. log can also be provided as more context (that can be used for logging, tracing, etc).
AgentFinish is a response that contains the final message to be sent back to the user. This should be used to end an agent run.
In this notebook we walk through how to create a custom LLM agent.
Set up environment#
Do necessary imports, etc.
!pip install langchain
!pip install google-search-results
!pip install openai


from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from langchain.prompts import BaseChatPromptTemplate
from langchain import SerpAPIWrapper, LLMChain
from langchain.chat_models import ChatOpenAI
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish, HumanMessage
import re
from getpass import getpass
Set up tool#
Set up any tools the agent may want to use. This may be necessary to put in the prompt (so that the agent knows to use these tools).
SERPAPI_API_KEY = getpass()
# Define which tools the agent can use to answer user queries
search = SerpAPIWrapper(serpapi_api_key=SERPAPI_API_KEY)
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events"
    )
]
Prompt Template#
This instructs the agent on what to do. Generally, the template should incorporate:
tools: which tools the agent has access and how and when to call them.
intermediate_steps: These are tuples of previous (AgentAction, Observation) pairs. These are generally not passed directly to the model, but the prompt template formats them in a specific way.
input: generic user input
# Set up the base template
template = """Complete the objective as best you can. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action


... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
These were previous tasks you completed:
Begin!
Question: {input}
{agent_scratchpad}"""
# Set up a prompt template
class CustomPromptTemplate(BaseChatPromptTemplate):
    # The template to use
    template: str
    # The list of tools available
    tools: List[Tool]

    def format_messages(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        formatted = self.template.format(**kwargs)
        return [HumanMessage(content=formatted)]
prompt = CustomPromptTemplate(
    template=template,
    tools=tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically
    # This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps"]
)
Output Parser#


The output parser is responsible for parsing the LLM output into AgentAction and AgentFinish. This usually depends heavily on the prompt used.
This is where you can change the parsing to do retries, handle whitespace, etc
class CustomOutputParser(AgentOutputParser):

    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
        # Check if agent should finish
        if "Final Answer:" in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            raise ValueError(f"Could not parse LLM output: `{llm_output}`")
        action = match.group(1).strip()
        action_input = match.group(2)
        # Return the action and action input
        return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
output_parser = CustomOutputParser()
Set up LLM#
Choose the LLM you want to use!
OPENAI_API_KEY = getpass()
llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY, temperature=0)
Define the stop sequence#
This is important because it tells the LLM when to stop generation.


This depends heavily on the prompt and model you are using. Generally, you want this to be whatever token you use in the prompt to denote the start of an Observation (otherwise, the LLM may hallucinate an observation for you).
Set up the Agent#
We can now combine everything to set up our agent
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
Use the Agent#
Now we can use it!
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("Search for Leo DiCaprio's girlfriend on the internet.")
> Entering new AgentExecutor chain...
Thought: I should use a reliable search engine to get accurate information.
Action: Search
Action Input: "Leo DiCaprio girlfriend"
Observation:He went on to date Gisele Bündchen, Bar Refaeli, Blake Lively, Toni Garrn and Nina Agdal, among others, before finally settling down with current girlfriend Camila Morrone, who is 23 years his junior.
I have found the answer to the question.
Final Answer: Leo DiCaprio's current girlfriend is Camila Morrone.
> Finished chain.
"Leo DiCaprio's current girlfriend is Camila Morrone."
previous
Custom LLM Agent
next
Custom MRKL Agent
 Contents

Set up environment
Set up tool
Prompt Template
Output Parser
Set up LLM
Define the stop sequence
Set up the Agent
Use the Agent


By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom MRKL Agent
 Contents
Custom LLMChain
Multiple inputs
Custom MRKL Agent#
This notebook goes through how to create your own custom MRKL agent.
A MRKL agent consists of three parts:
- Tools: The tools the agent has available to use.
- LLMChain: The LLMChain that produces the text that is parsed in a certain way to determine which action to take.
- The agent class itself: this parses the output of the LLMChain to determine which action to take.
In this notebook we walk through how to create a custom MRKL agent by creating a custom LLMChain.
Custom LLMChain#
The first way to create a custom agent is to use an existing Agent class, but use a custom LLMChain. This is the simplest way to create a custom Agent. It is highly recommended that you work with the ZeroShotAgent, as at the moment that is by far the most generalizable one.
Most of the work in creating the custom LLMChain comes down to the prompt. Because we are using an existing agent class to parse the output, it is very important that the prompt say to produce text in that format. Additionally, we currently require an agent_scratchpad input variable to put notes on previous actions and observations. This should almost always be the final part of the prompt. However, besides those instructions, you can customize the prompt as you wish.
To ensure that the prompt contains the appropriate instructions, we will utilize a helper method on that class. The helper method for the ZeroShotAgent takes the following arguments:
tools: List of tools the agent will have access to, used to format the prompt.
prefix: String to put before the list of tools.
suffix: String to put after the list of tools.
input_variables: List of input variables the final prompt will expect.


For this exercise, we will give our agent access to Google Search, and we will customize it in that we will have it answer as a pirate.
from langchain.agents import ZeroShotAgent, Tool, AgentExecutor
from langchain import OpenAI, SerpAPIWrapper, LLMChain
search = SerpAPIWrapper()
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events"
    )
]
prefix = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:"""
suffix = """Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Args"
Question: {input}
{agent_scratchpad}"""
prompt = ZeroShotAgent.create_prompt(
    tools,
    prefix=prefix,
    suffix=suffix,
    input_variables=["input", "agent_scratchpad"]
)
In case we are curious, we can now take a look at the final prompt template to see what it looks like when its all put together.
print(prompt.template)
Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
Search: useful for when you need to answer questions about current events
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Search]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question


Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Args"
Question: {input}
{agent_scratchpad}
Note that we are able to feed agents a self-defined prompt template, i.e. not restricted to the prompt generated by the create_prompt function, assuming it meets the agent’s requirements.
For example, for ZeroShotAgent, we will need to ensure that it meets the following requirements. There should a string starting with “Action:” and a following string starting with “Action Input:”, and both should be separated by a newline.
llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = ZeroShotAgent(llm_chain=llm_chain, allowed_tools=tool_names)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("How many people live in canada as of 2023?")
> Entering new AgentExecutor chain...
Thought: I need to find out the population of Canada
Action: Search
Action Input: Population of Canada 2023
Observation: The current population of Canada is 38,661,927 as of Sunday, April 16, 2023, based on Worldometer elaboration of the latest United Nations data.
Thought: I now know the final answer
Final Answer: Arrr, Canada be havin' 38,661,927 people livin' there as of 2023!
> Finished chain.
"Arrr, Canada be havin' 38,661,927 people livin' there as of 2023!"
Multiple inputs#
Agents can also work with prompts that require multiple inputs.
prefix = """Answer the following questions as best you can. You have access to the following tools:"""


suffix = """When answering, you MUST speak in the following language: {language}.
Question: {input}
{agent_scratchpad}"""
prompt = ZeroShotAgent.create_prompt(
    tools,
    prefix=prefix,
    suffix=suffix,
    input_variables=["input", "language", "agent_scratchpad"]
)
llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)
agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run(input="How many people live in canada as of 2023?", language="italian")
> Entering new AgentExecutor chain...
Thought: I should look for recent population estimates.
Action: Search
Action Input: Canada population 2023
Observation: 39,566,248
Thought: I should double check this number.
Action: Search
Action Input: Canada population estimates 2023
Observation: Canada's population was estimated at 39,566,248 on January 1, 2023, after a record population growth of 1,050,110 people from January 1, 2022, to January 1, 2023.
Thought: I now know the final answer.
Final Answer: La popolazione del Canada è stata stimata a 39.566.248 il 1° gennaio 2023, dopo un record di crescita demografica di 1.050.110 persone dal 1° gennaio 2022 al 1° gennaio 2023.
> Finished chain.


'La popolazione del Canada è stata stimata a 39.566.248 il 1° gennaio 2023, dopo un record di crescita demografica di 1.050.110 persone dal 1° gennaio 2022 al 1° gennaio 2023.'
previous
Custom LLM Agent (with a ChatModel)
next
Custom MultiAction Agent
 Contents

Custom LLMChain
Multiple inputs
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom Agent
Custom Agent#
This notebook goes through how to create your own custom agent.
An agent consists of two parts:
- Tools: The tools the agent has available to use.
- The agent class itself: this decides which action to take.
In this notebook we walk through how to create a custom agent.
from langchain.agents import Tool, AgentExecutor, BaseSingleActionAgent
from langchain import OpenAI, SerpAPIWrapper
search = SerpAPIWrapper()
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events",
        return_direct=True
    )
]
from typing import List, Tuple, Any, Union
from langchain.schema import AgentAction, AgentFinish
class FakeAgent(BaseSingleActionAgent):
    """Fake Custom Agent."""

    @property
    def input_keys(self):
        return ["input"]

    def plan(
        self, intermediate_steps: List[Tuple[AgentAction, str]], **kwargs: Any
    ) -> Union[AgentAction, AgentFinish]:
        """Given input, decided what to do.
        Args:
            intermediate_steps: Steps the LLM has taken to date,
                along with observations
            **kwargs: User inputs.
        Returns:
            Action specifying what tool to use.
        """
        return AgentAction(tool="Search", tool_input=kwargs["input"], log="")
    async def aplan(
        self, intermediate_steps: List[Tuple[AgentAction, str]], **kwargs: Any
    ) -> Union[AgentAction, AgentFinish]:
        """Given input, decided what to do.
        Args:
            intermediate_steps: Steps the LLM has taken to date,


along with observations
            **kwargs: User inputs.
        Returns:
            Action specifying what tool to use.
        """
        return AgentAction(tool="Search", tool_input=kwargs["input"], log="")
agent = FakeAgent()
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("How many people live in canada as of 2023?")
> Entering new AgentExecutor chain...
The current population of Canada is 38,669,152 as of Monday, April 24, 2023, based on Worldometer elaboration of the latest United Nations data.
> Finished chain.
'The current population of Canada is 38,669,152 as of Monday, April 24, 2023, based on Worldometer elaboration of the latest United Nations data.'
previous
Agent Types
next
Custom LLM Agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom Agent with Tool Retrieval
 Contents
Set up environment
Set up tools
Tool Retriever
Prompt Template
Output Parser
Set up LLM, stop sequence, and the agent
Use the Agent
Custom Agent with Tool Retrieval#
This notebook builds off of this notebook and assumes familiarity with how agents work.
The novel idea introduced in this notebook is the idea of using retrieval to select the set of tools to use to answer an agent query. This is useful when you have many many tools to select from. You cannot put the description of all the tools in the prompt (because of context length issues) so instead you dynamically select the N tools you do want to consider using at run time.
In this notebook we will create a somewhat contrieved example. We will have one legitimate tool (search) and then 99 fake tools which are just nonsense. We will then add a step in the prompt template that takes the user input and retrieves tool relevant to the query.
Set up environment#
Do necessary imports, etc.
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from langchain.prompts import StringPromptTemplate
from langchain import OpenAI, SerpAPIWrapper, LLMChain
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish
import re
Set up tools#
We will create one legitimate tool (search) and then 99 fake tools
# Define which tools the agent can use to answer user queries
search = SerpAPIWrapper()
search_tool = Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events"
    )
def fake_func(inp: str) -> str:
    return "foo"
fake_tools = [
    Tool(


name=f"foo-{i}",
        func=fake_func,
        description=f"a silly function that you can use to get more information about the number {i}"
    )
    for i in range(99)
]
ALL_TOOLS = [search_tool] + fake_tools
Tool Retriever#
We will use a vectorstore to create embeddings for each tool description. Then, for an incoming query we can create embeddings for that query and do a similarity search for relevant tools.
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.schema import Document
docs = [Document(page_content=t.description, metadata={"index": i}) for i, t in enumerate(ALL_TOOLS)]
vector_store = FAISS.from_documents(docs, OpenAIEmbeddings())
retriever = vector_store.as_retriever()
def get_tools(query):
    docs = retriever.get_relevant_documents(query)
    return [ALL_TOOLS[d.metadata["index"]] for d in docs]
We can now test this retriever to see if it seems to work.
get_tools("whats the weather?")
[Tool(name='Search', description='useful for when you need to answer questions about current events', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<bound method SerpAPIWrapper.run of SerpAPIWrapper(search_engine=<class 'serpapi.google_search.GoogleSearch'>, params={'engine': 'google', 'google_domain': 'google.com', 'gl': 'us', 'hl': 'en'}, serpapi_api_key='', aiosession=None)>, coroutine=None),


Tool(name='foo-95', description='a silly function that you can use to get more information about the number 95', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),
 Tool(name='foo-12', description='a silly function that you can use to get more information about the number 12', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),
 Tool(name='foo-15', description='a silly function that you can use to get more information about the number 15', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None)]
get_tools("whats the number 13?")
[Tool(name='foo-13', description='a silly function that you can use to get more information about the number 13', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),
 Tool(name='foo-12', description='a silly function that you can use to get more information about the number 12', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),


Tool(name='foo-14', description='a silly function that you can use to get more information about the number 14', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None),
 Tool(name='foo-11', description='a silly function that you can use to get more information about the number 11', return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x114b28a90>, func=<function fake_func at 0x15e5bd1f0>, coroutine=None)]
Prompt Template#
The prompt template is pretty standard, because we’re not actually changing that much logic in the actual prompt template, but rather we are just changing how retrieval is done.
# Set up the base template
template = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s
Question: {input}
{agent_scratchpad}"""
The custom prompt template now has the concept of a tools_getter, which we call on the input to select the tools to use
from typing import Callable


# Set up a prompt template
class CustomPromptTemplate(StringPromptTemplate):
    # The template to use
    template: str
    ############## NEW ######################
    # The list of tools available
    tools_getter: Callable

    def format(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        ############## NEW ######################
        tools = self.tools_getter(kwargs["input"])
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in tools])
        return self.template.format(**kwargs)
prompt = CustomPromptTemplate(
    template=template,
    tools_getter=get_tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically
    # This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps"]
)
Output Parser#
The output parser is unchanged from the previous notebook, since we are not changing anything about the output format.
class CustomOutputParser(AgentOutputParser):

    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:


# Check if agent should finish
        if "Final Answer:" in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            raise ValueError(f"Could not parse LLM output: `{llm_output}`")
        action = match.group(1).strip()
        action_input = match.group(2)
        # Return the action and action input
        return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
output_parser = CustomOutputParser()
Set up LLM, stop sequence, and the agent#
Also the same as the previous notebook
llm = OpenAI(temperature=0)
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)
tools = get_tools("whats the weather?")
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
Use the Agent#
Now we can use it!


agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("What's the weather in SF?")
> Entering new AgentExecutor chain...
Thought: I need to find out what the weather is in SF
Action: Search
Action Input: Weather in SF
Observation:Mostly cloudy skies early, then partly cloudy in the afternoon. High near 60F. ENE winds shifting to W at 10 to 15 mph. Humidity71%. UV Index6 of 10. I now know the final answer
Final Answer: 'Arg, 'tis mostly cloudy skies early, then partly cloudy in the afternoon. High near 60F. ENE winds shiftin' to W at 10 to 15 mph. Humidity71%. UV Index6 of 10.
> Finished chain.
"'Arg, 'tis mostly cloudy skies early, then partly cloudy in the afternoon. High near 60F. ENE winds shiftin' to W at 10 to 15 mph. Humidity71%. UV Index6 of 10."
previous
Custom MultiAction Agent
next
Conversation Agent (for Chat Models)
 Contents

Set up environment
Set up tools
Tool Retriever
Prompt Template
Output Parser
Set up LLM, stop sequence, and the agent
Use the Agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 04, 2023.


.ipynb
.pdf
Custom MultiAction Agent
Custom MultiAction Agent#
This notebook goes through how to create your own custom agent.
An agent consists of two parts:
- Tools: The tools the agent has available to use.
- The agent class itself: this decides which action to take.
In this notebook we walk through how to create a custom agent that predicts/takes multiple steps at a time.
from langchain.agents import Tool, AgentExecutor, BaseMultiActionAgent
from langchain import OpenAI, SerpAPIWrapper
def random_word(query: str) -> str:
    print("\nNow I'm doing this!")
    return "foo"
search = SerpAPIWrapper()
tools = [
    Tool(
        name = "Search",
        func=search.run,
        description="useful for when you need to answer questions about current events"
    ),
    Tool(
        name = "RandomWord",
        func=random_word,
        description="call this to get a random word."

    )
]
from typing import List, Tuple, Any, Union
from langchain.schema import AgentAction, AgentFinish
class FakeAgent(BaseMultiActionAgent):
    """Fake Custom Agent."""

    @property
    def input_keys(self):
        return ["input"]

    def plan(
        self, intermediate_steps: List[Tuple[AgentAction, str]], **kwargs: Any
    ) -> Union[List[AgentAction], AgentFinish]:
        """Given input, decided what to do.
        Args:
            intermediate_steps: Steps the LLM has taken to date,
                along with observations
            **kwargs: User inputs.
        Returns:
            Action specifying what tool to use.
        """
        if len(intermediate_steps) == 0:
            return [