.md
.pdf
Code Understanding
 Contents
Conversational Retriever Chain
Code Understanding#
Overview
LangChain is a useful tool designed to parse GitHub code repositories. By leveraging VectorStores, Conversational RetrieverChain, and GPT-4, it can answer questions in the context of an entire GitHub repository or generate new code. This documentation page outlines the essential components of the system and guides using LangChain for better code comprehension, contextual question answering, and code generation in GitHub repositories.
Conversational Retriever Chain#
Conversational RetrieverChain is a retrieval-focused system that interacts with the data stored in a VectorStore. Utilizing advanced techniques, like context-aware filtering and ranking, it retrieves the most relevant code snippets and information for a given user query. Conversational RetrieverChain is engineered to deliver high-quality, pertinent results while considering conversation history and context.
LangChain Workflow for Code Understanding and Generation
Index the code base: Clone the target repository, load all files within, chunk the files, and execute the indexing process. Optionally, you can skip this step and use an already indexed dataset.
Embedding and Code Store: Code snippets are embedded using a code-aware embedding model and stored in a VectorStore.
Query Understanding: GPT-4 processes user queries, grasping the context and extracting relevant details.
Construct the Retriever: Conversational RetrieverChain searches the VectorStore to identify the most relevant code snippets for a given query.
Build the Conversational Chain: Customize the retriever settings and define any user-defined filters as needed.
Ask questions: Define a list of questions to ask about the codebase, and then use the ConversationalRetrievalChain to generate context-aware answers. The LLM (GPT-4) generates comprehensive, context-aware answers based on retrieved code snippets and conversation history.
The full tutorial is available below.


The full tutorial is available below.
Twitter the-algorithm codebase analysis with Deep Lake: A notebook walking through how to parse github source code and run queries conversation.
LangChain codebase analysis with Deep Lake: A notebook walking through how to analyze and do question answering over THIS code base.
previous
Querying Tabular Data
next
Interacting with APIs
 Contents

Conversational Retriever Chain
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.rst
.pdf
Evaluation
 Contents
The Problem
The Solution
The Examples
Other Examples
Evaluation#
Note
Conceptual Guide
This section of documentation covers how we approach and think about evaluation in LangChain.
Both evaluation of internal chains/agents, but also how we would recommend people building on top of LangChain approach evaluation.
The Problem#
It can be really hard to evaluate LangChain chains and agents.
There are two main reasons for this:
# 1: Lack of data
You generally don’t have a ton of data to evaluate your chains/agents over before starting a project.
This is usually because Large Language Models (the core of most chains/agents) are terrific few-shot and zero shot learners,
meaning you are almost always able to get started on a particular task (text-to-SQL, question answering, etc) without
a large dataset of examples.
This is in stark contrast to traditional machine learning where you had to first collect a bunch of datapoints
before even getting started using a model.
# 2: Lack of metrics
Most chains/agents are performing tasks for which there are not very good metrics to evaluate performance.
For example, one of the most common use cases is generating text of some form.
Evaluating generated text is much more complicated than evaluating a classification prediction, or a numeric prediction.
The Solution#
LangChain attempts to tackle both of those issues.
What we have so far are initial passes at solutions - we do not think we have a perfect solution.
So we very much welcome feedback, contributions, integrations, and thoughts on this.
Here is what we have for each problem so far:
# 1: Lack of data
We have started LangChainDatasets a Community space on Hugging Face.
We intend this to be a collection of open source datasets for evaluating common chains and agents.


We intend this to be a collection of open source datasets for evaluating common chains and agents.
We have contributed five datasets of our own to start, but we highly intend this to be a community effort.
In order to contribute a dataset, you simply need to join the community and then you will be able to upload datasets.
We’re also aiming to make it as easy as possible for people to create their own datasets.
As a first pass at this, we’ve added a QAGenerationChain, which given a document comes up
with question-answer pairs that can be used to evaluate question-answering tasks over that document down the line.
See this notebook for an example of how to use this chain.
# 2: Lack of metrics
We have two solutions to the lack of metrics.
The first solution is to use no metrics, and rather just rely on looking at results by eye to get a sense for how the chain/agent is performing.
To assist in this, we have developed (and will continue to develop) tracing, a UI-based visualizer of your chain and agent runs.
The second solution we recommend is to use Language Models themselves to evaluate outputs.
For this we have a few different chains and prompts aimed at tackling this issue.
The Examples#
We have created a bunch of examples combining the above two solutions to show how we internally evaluate chains and agents when we are developing.
In addition to the examples we’ve curated, we also highly welcome contributions here.
To facilitate that, we’ve included a template notebook for community members to use to build their own examples.
The existing examples we have are:
Question Answering (State of Union): A notebook showing evaluation of a question-answering task over a State-of-the-Union address.
Question Answering (Paul Graham Essay): A notebook showing evaluation of a question-answering task over a Paul Graham essay.


SQL Question Answering (Chinook): A notebook showing evaluation of a question-answering task over a SQL database (the Chinook database).
Agent Vectorstore: A notebook showing evaluation of an agent doing question answering while routing between two different vector databases.
Agent Search + Calculator: A notebook showing evaluation of an agent doing question answering using a Search engine and a Calculator as tools.
Evaluating an OpenAPI Chain: A notebook showing evaluation of an OpenAPI chain, including how to generate test data if you don’t have any.
Other Examples#
In addition, we also have some more generic resources for evaluation.
Question Answering: An overview of LLMs aimed at evaluating question answering systems in general.
Data Augmented Question Answering: An end-to-end example of evaluating a question answering system focused on a specific document (a RetrievalQAChain to be precise). This example highlights how to use LLMs to come up with question/answer examples to evaluate over, and then highlights how to use LLMs to evaluate performance on those generated examples.
Hugging Face Datasets: Covers an example of loading and using a dataset from Hugging Face for evaluation.
previous
Extraction
next
Agent Benchmarking: Search + Calculator
 Contents

The Problem
The Solution
The Examples
Other Examples
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.md
.pdf
Extraction
Extraction#
Conceptual Guide
Most APIs and databases still deal with structured information.
Therefore, in order to better work with those, it can be useful to extract structured information from text.
Examples of this include:
Extracting a structured row to insert into a database from a sentence
Extracting multiple rows to insert into a database from a long document
Extracting the correct API parameters from a user query
This work is extremely related to output parsing.
Output parsers are responsible for instructing the LLM to respond in a specific format.
In this case, the output parsers specify the format of the data you would like to extract from the document.
Then, in addition to the output format instructions, the prompt should also contain the data you would like to extract information from.
While normal output parsers are good enough for basic structuring of response data,
when doing extraction you often want to extract more complicated or nested structures.
For a deep dive on extraction, we recommend checking out kor,
a library that uses the existing LangChain chain and OutputParser abstractions
but deep dives on allowing extraction of more complicated schemas.
previous
Summarization
next
Evaluation
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.md
.pdf
Personal Assistants (Agents)
Personal Assistants (Agents)#
Conceptual Guide
We use “personal assistant” here in a very broad sense.
Personal assistants have a few characteristics:
They can interact with the outside world
They have knowledge of your data
They remember your interactions
Really all of the functionality in LangChain is relevant for building a personal assistant.
Highlighting specific parts:
Agent Documentation (for interacting with the outside world)
Index Documentation (for giving them knowledge of your data)
Memory (for helping them remember interactions)
Specific examples of this include:
AI Plugins: an implementation of an agent that is designed to be able to use all AI Plugins.
Wikibase Agent: an implementation of an agent that is designed to interact with Wikibase.
previous
How to add SharedMemory to an Agent and its Tools
next
Autonomous Agents
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.md
.pdf
Question Answering over Docs
 Contents
Document Question Answering
Adding in sources
Additional Related Resources
End-to-end examples
Question Answering over Docs#
Conceptual Guide
Question answering in this context refers to question answering over your document data.
For question answering over other types of data, please see other sources documentation like SQL database Question Answering or Interacting with APIs.
For question answering over many documents, you almost always want to create an index over the data.
This can be used to smartly access the most relevant documents for a given question, allowing you to avoid having to pass all the documents to the LLM (saving you time and money).
See this notebook for a more detailed introduction to this, but for a super quick start the steps involved are:
Load Your Documents
from langchain.document_loaders import TextLoader
loader = TextLoader('../state_of_the_union.txt')
See here for more information on how to get started with document loading.
Create Your Index
from langchain.indexes import VectorstoreIndexCreator
index = VectorstoreIndexCreator().from_loaders([loader])
The best and most popular index by far at the moment is the VectorStore index.
Query Your Index
query = "What did the president say about Ketanji Brown Jackson"
index.query(query)
Alternatively, use query_with_sources to also get back the sources involved
query = "What did the president say about Ketanji Brown Jackson"
index.query_with_sources(query)
Again, these high level interfaces obfuscate a lot of what is going on under the hood, so please see this notebook for a lower level walkthrough.
Document Question Answering#
Question answering involves fetching multiple documents, and then asking a question of them.
The LLM response will contain the answer to your question, based on the content of the documents.


The recommended way to get started using a question answering chain is:
from langchain.chains.question_answering import load_qa_chain
chain = load_qa_chain(llm, chain_type="stuff")
chain.run(input_documents=docs, question=query)
The following resources exist:
Question Answering Notebook: A notebook walking through how to accomplish this task.
VectorDB Question Answering Notebook: A notebook walking through how to do question answering over a vector database. This can often be useful for when you have a LOT of documents, and you don’t want to pass them all to the LLM, but rather first want to do some semantic search over embeddings.
Adding in sources#
There is also a variant of this, where in addition to responding with the answer the language model will also cite its sources (eg which of the documents passed in it used).
The recommended way to get started using a question answering with sources chain is:
from langchain.chains.qa_with_sources import load_qa_with_sources_chain
chain = load_qa_with_sources_chain(llm, chain_type="stuff")
chain({"input_documents": docs, "question": query}, return_only_outputs=True)
The following resources exist:
QA With Sources Notebook: A notebook walking through how to accomplish this task.
VectorDB QA With Sources Notebook: A notebook walking through how to do question answering with sources over a vector database. This can often be useful for when you have a LOT of documents, and you don’t want to pass them all to the LLM, but rather first want to do some semantic search over embeddings.
Additional Related Resources#
Additional related resources include:
Utilities for working with Documents: Guides on how to use several of the utilities which will prove helpful for this task, including Text Splitters (for splitting up long documents) and Embeddings & Vectorstores (useful for the above Vector DB example).


CombineDocuments Chains: A conceptual overview of specific types of chains by which you can accomplish this task.
End-to-end examples#
For examples to this done in an end-to-end manner, please see the following resources:
Semantic search over a group chat with Sources Notebook: A notebook that semantically searches over a group chat conversation.
previous
Agent Simulations
next
Chatbots
 Contents

Document Question Answering
Adding in sources
Additional Related Resources
End-to-end examples
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.md
.pdf
Summarization
Summarization#
Conceptual Guide
Summarization involves creating a smaller summary of multiple longer documents.
This can be useful for distilling long documents into the core pieces of information.
The recommended way to get started using a summarization chain is:
from langchain.chains.summarize import load_summarize_chain
chain = load_summarize_chain(llm, chain_type="map_reduce")
chain.run(docs)
The following resources exist:
Summarization Notebook: A notebook walking through how to accomplish this task.
Additional related resources include:
Utilities for working with Documents: Guides on how to use several of the utilities which will prove helpful for this task, including Text Splitters (for splitting up long documents).
previous
Interacting with APIs
next
Extraction
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.md
.pdf
Querying Tabular Data
 Contents
Document Loading
Querying
Chains
Agents
Querying Tabular Data#
Conceptual Guide
Lots of data and information is stored in tabular data, whether it be csvs, excel sheets, or SQL tables.
This page covers all resources available in LangChain for working with data in this format.
Document Loading#
If you have text data stored in a tabular format, you may want to load the data into a Document and then index it as you would
other text/unstructured data. For this, you should use a document loader like the CSVLoader
and then you should create an index over that data, and query it that way.
Querying#
If you have more numeric tabular data, or have a large amount of data and don’t want to index it, you should get started
by looking at various chains and agents we have for dealing with this data.
Chains#
If you are just getting started, and you have relatively small/simple tabular data, you should get started with chains.
Chains are a sequence of predetermined steps, so they are good to get started with as they give you more control and let you
understand what is happening better.
SQL Database Chain
Agents#
Agents are more complex, and involve multiple queries to the LLM to understand what to do.
The downside of agents are that you have less control. The upside is that they are more powerful,
which allows you to use them on larger databases and more complex schemas.
SQL Agent
Pandas Agent
CSV Agent
previous
Chatbots
next
Code Understanding
 Contents

Document Loading
Querying
Chains
Agents
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Custom Agent with PlugIn Retrieval
 Contents
Set up environment
Setup LLM
Set up plugins
Tool Retriever
Prompt Template
Output Parser
Set up LLM, stop sequence, and the agent
Use the Agent
Custom Agent with PlugIn Retrieval#
This notebook combines two concepts in order to build a custom agent that can interact with AI Plugins:
Custom Agent with Retrieval: This introduces the concept of retrieving many tools, which is useful when trying to work with arbitrarily many plugins.
Natural Language API Chains: This creates Natural Language wrappers around OpenAPI endpoints. This is useful because (1) plugins use OpenAPI endpoints under the hood, (2) wrapping them in an NLAChain allows the router agent to call it more easily.
The novel idea introduced in this notebook is the idea of using retrieval to select not the tools explicitly, but the set of OpenAPI specs to use. We can then generate tools from those OpenAPI specs. The use case for this is when trying to get agents to use plugins. It may be more efficient to choose plugins first, then the endpoints, rather than the endpoints directly. This is because the plugins may contain more useful information for selection.
Set up environment#
Do necessary imports, etc.
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from langchain.prompts import StringPromptTemplate
from langchain import OpenAI, SerpAPIWrapper, LLMChain
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish
from langchain.agents.agent_toolkits import NLAToolkit
from langchain.tools.plugin import AIPlugin
import re
Setup LLM#
llm = OpenAI(temperature=0)
Set up plugins#
Load and index plugins
urls = [


Set up plugins#
Load and index plugins
urls = [
    "https://datasette.io/.well-known/ai-plugin.json",
    "https://api.speak.com/.well-known/ai-plugin.json",
    "https://www.wolframalpha.com/.well-known/ai-plugin.json",
    "https://www.zapier.com/.well-known/ai-plugin.json",
    "https://www.klarna.com/.well-known/ai-plugin.json",
    "https://www.joinmilo.com/.well-known/ai-plugin.json",
    "https://slack.com/.well-known/ai-plugin.json",
    "https://schooldigger.com/.well-known/ai-plugin.json",
]
AI_PLUGINS = [AIPlugin.from_url(url) for url in urls]
Tool Retriever#
We will use a vectorstore to create embeddings for each tool description. Then, for an incoming query we can create embeddings for that query and do a similarity search for relevant tools.
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.schema import Document
embeddings = OpenAIEmbeddings()
docs = [
    Document(page_content=plugin.description_for_model,
             metadata={"plugin_name": plugin.name_for_model}
            )
    for plugin in AI_PLUGINS
]
vector_store = FAISS.from_documents(docs, embeddings)
toolkits_dict = {plugin.name_for_model:
                 NLAToolkit.from_llm_and_ai_plugin(llm, plugin)
                 for plugin in AI_PLUGINS}
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.


Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.2 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
Attempting to load a Swagger 2.0 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
retriever = vector_store.as_retriever()
def get_tools(query):
    # Get documents, which contain the Plugins to use
    docs = retriever.get_relevant_documents(query)
    # Get the toolkits, one for each plugin
    tool_kits = [toolkits_dict[d.metadata["plugin_name"]] for d in docs]
    # Get the tools: a separate NLAChain for each endpoint


# Get the tools: a separate NLAChain for each endpoint
    tools = []
    for tk in tool_kits:
        tools.extend(tk.nla_tools)
    return tools
We can now test this retriever to see if it seems to work.
tools = get_tools("What could I do today with my kiddo")
[t.name for t in tools]
['Milo.askMilo',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',
 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',
 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',
 'SchoolDigger_API_V2.0.Districts_GetDistrict2',
 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',
 'SchoolDigger_API_V2.0.Rankings_GetRank_District',
 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',
 'SchoolDigger_API_V2.0.Schools_GetSchool20',
 'Speak.translate',
 'Speak.explainPhrase',
 'Speak.explainTask']
tools = get_tools("what shirts can i buy?")
[t.name for t in tools]
['Open_AI_Klarna_product_Api.productsUsingGET',
 'Milo.askMilo',


'Milo.askMilo',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',
 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',
 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',
 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',
 'SchoolDigger_API_V2.0.Districts_GetDistrict2',
 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',
 'SchoolDigger_API_V2.0.Rankings_GetRank_District',
 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',
 'SchoolDigger_API_V2.0.Schools_GetSchool20']
Prompt Template#
The prompt template is pretty standard, because we’re not actually changing that much logic in the actual prompt template, but rather we are just changing how retrieval is done.
# Set up the base template
template = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer


Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s
Question: {input}
{agent_scratchpad}"""
The custom prompt template now has the concept of a tools_getter, which we call on the input to select the tools to use
from typing import Callable
# Set up a prompt template
class CustomPromptTemplate(StringPromptTemplate):
    # The template to use
    template: str
    ############## NEW ######################
    # The list of tools available
    tools_getter: Callable

    def format(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        ############## NEW ######################
        tools = self.tools_getter(kwargs["input"])
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in tools])
        return self.template.format(**kwargs)
prompt = CustomPromptTemplate(
    template=template,
    tools_getter=get_tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically


# This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps"]
)
Output Parser#
The output parser is unchanged from the previous notebook, since we are not changing anything about the output format.
class CustomOutputParser(AgentOutputParser):

    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
        # Check if agent should finish
        if "Final Answer:" in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            raise ValueError(f"Could not parse LLM output: `{llm_output}`")
        action = match.group(1).strip()
        action_input = match.group(2)
        # Return the action and action input
        return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
output_parser = CustomOutputParser()
Set up LLM, stop sequence, and the agent#
Also the same as the previous notebook
llm = OpenAI(temperature=0)
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)


llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
Use the Agent#
Now we can use it!
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
agent_executor.run("what shirts can i buy?")
> Entering new AgentExecutor chain...
Thought: I need to find a product API
Action: Open_AI_Klarna_product_Api.productsUsingGET
Action Input: shirts
Observation:I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns. I now know what shirts I can buy
Final Answer: Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.
> Finished chain.
'Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.'
 Contents

Set up environment
Setup LLM
Set up plugins
Tool Retriever
Prompt Template
Output Parser
Set up LLM, stop sequence, and the agent
Use the Agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Wikibase Agent
 Contents
Wikibase Agent
Preliminaries
API keys and other secrats
OpenAI API Key
Wikidata user-agent header
Enable tracing if desired
Tools
Item and Property lookup
Sparql runner
Agent
Wrap the tools
Prompts
Output parser
Specify the LLM model
Agent and agent executor
Run it!
Wikibase Agent#
This notebook demonstrates a very simple wikibase agent that uses sparql generation. Although this code is intended to work against any
wikibase instance, we use http://wikidata.org for testing.
If you are interested in wikibases and sparql, please consider helping to improve this agent. Look here for more details and open questions.
Preliminaries#
API keys and other secrats#
We use an .ini file, like this:
[OPENAI]
OPENAI_API_KEY=xyzzy
[WIKIDATA]
WIKIDATA_USER_AGENT_HEADER=argle-bargle
import configparser
config = configparser.ConfigParser()
config.read('./secrets.ini')
['./secrets.ini']
OpenAI API Key#
An OpenAI API key is required unless you modify the code below to use another LLM provider.
openai_api_key = config['OPENAI']['OPENAI_API_KEY']
import os
os.environ.update({'OPENAI_API_KEY': openai_api_key})
Wikidata user-agent header#
Wikidata policy requires a user-agent header. See https://meta.wikimedia.org/wiki/User-Agent_policy. However, at present this policy is not strictly enforced.
wikidata_user_agent_header = None if not config.has_section('WIKIDATA') else config['WIKIDATA']['WIKIDAtA_USER_AGENT_HEADER']
Enable tracing if desired#
#import os


Enable tracing if desired#
#import os
#os.environ["LANGCHAIN_HANDLER"] = "langchain"
#os.environ["LANGCHAIN_SESSION"] = "default" # Make sure this session actually exists.
Tools#
Three tools are provided for this simple agent:
ItemLookup: for finding the q-number of an item
PropertyLookup: for finding the p-number of a property
SparqlQueryRunner: for running a sparql query
Item and Property lookup#
Item and Property lookup are implemented in a single method, using an elastic search endpoint. Not all wikibase instances have it, but wikidata does, and that’s where we’ll start.
def get_nested_value(o: dict, path: list) -> any:
    current = o
    for key in path:
        try:
            current = current[key]
        except:
            return None
    return current
import requests
from typing import Optional
def vocab_lookup(search: str, entity_type: str = "item",
                 url: str = "https://www.wikidata.org/w/api.php",
                 user_agent_header: str = wikidata_user_agent_header,
                 srqiprofile: str = None,
                ) -> Optional[str]:
    headers = {
        'Accept': 'application/json'
    }
    if wikidata_user_agent_header is not None:
        headers['User-Agent'] = wikidata_user_agent_header

    if entity_type == "item":
        srnamespace = 0
        srqiprofile = "classic_noboostlinks" if srqiprofile is None else srqiprofile
    elif entity_type == "property":
        srnamespace = 120
        srqiprofile = "classic" if srqiprofile is None else srqiprofile
    else:


else:
        raise ValueError("entity_type must be either 'property' or 'item'")

    params = {
        "action": "query",
        "list": "search",
        "srsearch": search,
        "srnamespace": srnamespace,
        "srlimit": 1,
        "srqiprofile": srqiprofile,
        "srwhat": 'text',
        "format": "json"
    }

    response = requests.get(url, headers=headers, params=params)

    if response.status_code == 200:
        title = get_nested_value(response.json(), ['query', 'search', 0, 'title'])
        if title is None:
            return f"I couldn't find any {entity_type} for '{search}'. Please rephrase your request and try again"
        # if there is a prefix, strip it off
        return title.split(':')[-1]
    else:
        return "Sorry, I got an error. Please try again."
print(vocab_lookup("Malin 1"))
Q4180017
print(vocab_lookup("instance of", entity_type="property"))
P31
print(vocab_lookup("Ceci n'est pas un q-item"))
I couldn't find any item for 'Ceci n'est pas un q-item'. Please rephrase your request and try again
Sparql runner#
This tool runs sparql - by default, wikidata is used.
import requests
from typing import List, Dict, Any
import json
def run_sparql(query: str, url='https://query.wikidata.org/sparql',
               user_agent_header: str = wikidata_user_agent_header) -> List[Dict[str, Any]]:
    headers = {
        'Accept': 'application/json'


headers = {
        'Accept': 'application/json'
    }
    if wikidata_user_agent_header is not None:
        headers['User-Agent'] = wikidata_user_agent_header
    response = requests.get(url, headers=headers, params={'query': query, 'format': 'json'})
    if response.status_code != 200:
        return "That query failed. Perhaps you could try a different one?"
    results = get_nested_value(response.json(),['results', 'bindings'])
    return json.dumps(results)
run_sparql("SELECT (COUNT(?children) as ?count) WHERE { wd:Q1339 wdt:P40 ?children . }")
'[{"count": {"datatype": "http://www.w3.org/2001/XMLSchema#integer", "type": "literal", "value": "20"}}]'
Agent#
Wrap the tools#
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from langchain.prompts import StringPromptTemplate
from langchain import OpenAI, LLMChain
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish
import re
# Define which tools the agent can use to answer user queries
tools = [
    Tool(
        name = "ItemLookup",
        func=(lambda x: vocab_lookup(x, entity_type="item")),
        description="useful for when you need to know the q-number for an item"
    ),
    Tool(
        name = "PropertyLookup",
        func=(lambda x: vocab_lookup(x, entity_type="property")),
        description="useful for when you need to know the p-number for a property"
    ),
    Tool(
        name = "SparqlQueryRunner",
        func=run_sparql,


name = "SparqlQueryRunner",
        func=run_sparql,
        description="useful for getting results from a wikibase"
    )
]
Prompts#
# Set up the base template
template = """
Answer the following questions by running a sparql query against a wikibase where the p and q items are
completely unknown to you. You will need to discover the p and q items before you can generate the sparql.
Do not assume you know the p and q items for any concepts. Always use tools to find all p and q items.
After you generate the sparql, you should run it. The results will be returned in json.
Summarize the json results in natural language.
You may assume the following prefixes:
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX p: <http://www.wikidata.org/prop/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
When generating sparql:
* Try to avoid "count" and "filter" queries if possible
* Never enclose the sparql in back-quotes
You have access to the following tools:
{tools}
Use the following format:
Question: the input question for which you must provide a natural language answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Question: {input}
{agent_scratchpad}"""


Question: {input}
{agent_scratchpad}"""
# Set up a prompt template
class CustomPromptTemplate(StringPromptTemplate):
    # The template to use
    template: str
    # The list of tools available
    tools: List[Tool]

    def format(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        return self.template.format(**kwargs)
prompt = CustomPromptTemplate(
    template=template,
    tools=tools,
    # This omits the `agent_scratchpad`, `tools`, and `tool_names` variables because those are generated dynamically
    # This includes the `intermediate_steps` variable because that is needed
    input_variables=["input", "intermediate_steps"]
)
Output parser#
This is unchanged from langchain docs
class CustomOutputParser(AgentOutputParser):

    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
        # Check if agent should finish
        if "Final Answer:" in llm_output:
            return AgentFinish(


if "Final Answer:" in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action: (.*?)[\n]*Action Input:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            raise ValueError(f"Could not parse LLM output: `{llm_output}`")
        action = match.group(1).strip()
        action_input = match.group(2)
        # Return the action and action input
        return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
output_parser = CustomOutputParser()
Specify the LLM model#
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(model="gpt-4", temperature=0)
Agent and agent executor#
# LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain,
    output_parser=output_parser,
    stop=["\nObservation:"],
    allowed_tools=tool_names
)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
Run it!#
# If you prefer in-line tracing, uncomment this line
# agent_executor.agent.llm_chain.verbose = True


# agent_executor.agent.llm_chain.verbose = True
agent_executor.run("How many children did J.S. Bach have?")
> Entering new AgentExecutor chain...
Thought: I need to find the Q number for J.S. Bach.
Action: ItemLookup
Action Input: J.S. Bach
Observation:Q1339I need to find the P number for children.
Action: PropertyLookup
Action Input: children
Observation:P1971Now I can query the number of children J.S. Bach had.
Action: SparqlQueryRunner
Action Input: SELECT ?children WHERE { wd:Q1339 wdt:P1971 ?children }
Observation:[{"children": {"datatype": "http://www.w3.org/2001/XMLSchema#decimal", "type": "literal", "value": "20"}}]I now know the final answer.
Final Answer: J.S. Bach had 20 children.
> Finished chain.
'J.S. Bach had 20 children.'
agent_executor.run("What is the Basketball-Reference.com NBA player ID of Hakeem Olajuwon?")
> Entering new AgentExecutor chain...
Thought: To find Hakeem Olajuwon's Basketball-Reference.com NBA player ID, I need to first find his Wikidata item (Q-number) and then query for the relevant property (P-number).
Action: ItemLookup
Action Input: Hakeem Olajuwon
Observation:Q273256Now that I have Hakeem Olajuwon's Wikidata item (Q273256), I need to find the P-number for the Basketball-Reference.com NBA player ID property.
Action: PropertyLookup
Action Input: Basketball-Reference.com NBA player ID


Action: PropertyLookup
Action Input: Basketball-Reference.com NBA player ID
Observation:P2685Now that I have both the Q-number for Hakeem Olajuwon (Q273256) and the P-number for the Basketball-Reference.com NBA player ID property (P2685), I can run a SPARQL query to get the ID value.
Action: SparqlQueryRunner
Action Input:
SELECT ?playerID WHERE {
  wd:Q273256 wdt:P2685 ?playerID .
}
Observation:[{"playerID": {"type": "literal", "value": "o/olajuha01"}}]I now know the final answer
Final Answer: Hakeem Olajuwon's Basketball-Reference.com NBA player ID is "o/olajuha01".
> Finished chain.
'Hakeem Olajuwon\'s Basketball-Reference.com NBA player ID is "o/olajuha01".'
 Contents

Wikibase Agent
Preliminaries
API keys and other secrats
OpenAI API Key
Wikidata user-agent header
Enable tracing if desired
Tools
Item and Property lookup
Sparql runner
Agent
Wrap the tools
Prompts
Output parser
Specify the LLM model
Agent and agent executor
Run it!
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
CAMEL Role-Playing Autonomous Cooperative Agents
 Contents
Import LangChain related modules
Define a CAMEL agent helper class
Setup OpenAI API key and roles and task for role-playing
Create a task specify agent for brainstorming and get the specified task
Create inception prompts for AI assistant and AI user for role-playing
Create a helper helper to get system messages for AI assistant and AI user from role names and the task
Create AI assistant agent and AI user agent from obtained system messages
Start role-playing session to solve the task!
CAMEL Role-Playing Autonomous Cooperative Agents#
This is a langchain implementation of paper: “CAMEL: Communicative Agents for “Mind” Exploration of Large Scale Language Model Society”.
Overview:
The rapid advancement of conversational and chat-based language models has led to remarkable progress in complex task-solving. However, their success heavily relies on human input to guide the conversation, which can be challenging and time-consuming. This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents and provide insight into their “cognitive” processes. To address the challenges of achieving autonomous cooperation, we propose a novel communicative agent framework named role-playing. Our approach involves using inception prompting to guide chat agents toward task completion while maintaining consistency with human intentions. We showcase how role-playing can be used to generate conversational data for studying the behaviors and capabilities of chat agents, providing a valuable resource for investigating conversational language models. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems, and open-sourcing our library to support research on communicative agents and beyond.
The original implementation: https://github.com/lightaime/camel
Project website: https://www.camel-ai.org/
Arxiv paper: https://arxiv.org/abs/2303.17760


Arxiv paper: https://arxiv.org/abs/2303.17760
Import LangChain related modules#
from typing import List
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)
from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage,
    BaseMessage,
)
Define a CAMEL agent helper class#
class CAMELAgent:
    def __init__(
        self,
        system_message: SystemMessage,
        model: ChatOpenAI,
    ) -> None:
        self.system_message = system_message
        self.model = model
        self.init_messages()
    def reset(self) -> None:
        self.init_messages()
        return self.stored_messages
    def init_messages(self) -> None:
        self.stored_messages = [self.system_message]
    def update_messages(self, message: BaseMessage) -> List[BaseMessage]:
        self.stored_messages.append(message)
        return self.stored_messages
    def step(
        self,
        input_message: HumanMessage,
    ) -> AIMessage:
        messages = self.update_messages(input_message)
        output_message = self.model(messages)
        self.update_messages(output_message)
        return output_message
Setup OpenAI API key and roles and task for role-playing#
import os
os.environ["OPENAI_API_KEY"] = ""
assistant_role_name = "Python Programmer"
user_role_name = "Stock Trader"
task = "Develop a trading bot for the stock market"
word_limit = 50 # word limit for task brainstorming
Create a task specify agent for brainstorming and get the specified task#


Create a task specify agent for brainstorming and get the specified task#
task_specifier_sys_msg = SystemMessage(content="You can make a task more specific.")
task_specifier_prompt = (
"""Here is a task that {assistant_role_name} will help {user_role_name} to complete: {task}.
Please make it more specific. Be creative and imaginative.
Please reply with the specified task in {word_limit} words or less. Do not add anything else."""
)
task_specifier_template = HumanMessagePromptTemplate.from_template(template=task_specifier_prompt)
task_specify_agent = CAMELAgent(task_specifier_sys_msg, ChatOpenAI(temperature=1.0))
task_specifier_msg = task_specifier_template.format_messages(assistant_role_name=assistant_role_name,
                                                             user_role_name=user_role_name,
                                                             task=task, word_limit=word_limit)[0]
specified_task_msg = task_specify_agent.step(task_specifier_msg)
print(f"Specified task: {specified_task_msg.content}")
specified_task = specified_task_msg.content
Specified task: Develop a Python-based swing trading bot that scans market trends, monitors stocks, and generates trading signals to help a stock trader to place optimal buy and sell orders with defined stop losses and profit targets.
Create inception prompts for AI assistant and AI user for role-playing#
assistant_inception_prompt = (
"""Never forget you are a {assistant_role_name} and I am a {user_role_name}. Never flip roles! Never instruct me!
We share a common interest in collaborating to successfully complete a task.
You must help me to complete the task.
Here is the task: {task}. Never forget our task!
I must instruct you based on your expertise and my needs to complete the task.
I must give you one instruction at a time.


I must give you one instruction at a time.
You must write a specific solution that appropriately completes the requested instruction.
You must decline my instruction honestly if you cannot perform the instruction due to physical, moral, legal reasons or your capability and explain the reasons.
Do not add anything else other than your solution to my instruction.
You are never supposed to ask me any questions you only answer questions.
You are never supposed to reply with a flake solution. Explain your solutions.
Your solution must be declarative sentences and simple present tense.
Unless I say the task is completed, you should always start with:
Solution: <YOUR_SOLUTION>
<YOUR_SOLUTION> should be specific and provide preferable implementations and examples for task-solving.
Always end <YOUR_SOLUTION> with: Next request."""
)
user_inception_prompt = (
"""Never forget you are a {user_role_name} and I am a {assistant_role_name}. Never flip roles! You will always instruct me.
We share a common interest in collaborating to successfully complete a task.
I must help you to complete the task.
Here is the task: {task}. Never forget our task!
You must instruct me based on my expertise and your needs to complete the task ONLY in the following two ways:
1. Instruct with a necessary input:
Instruction: <YOUR_INSTRUCTION>
Input: <YOUR_INPUT>
2. Instruct without any input:
Instruction: <YOUR_INSTRUCTION>
Input: None
The "Instruction" describes a task or question. The paired "Input" provides further context or information for the requested "Instruction".
You must give me one instruction at a time.
I must write a response that appropriately completes the requested instruction.
I must decline your instruction honestly if I cannot perform the instruction due to physical, moral, legal reasons or my capability and explain the reasons.


You should instruct me not ask me questions.
Now you must start to instruct me using the two ways described above.
Do not add anything else other than your instruction and the optional corresponding input!
Keep giving me instructions and necessary inputs until you think the task is completed.
When the task is completed, you must only reply with a single word <CAMEL_TASK_DONE>.
Never say <CAMEL_TASK_DONE> unless my responses have solved your task."""
)
Create a helper helper to get system messages for AI assistant and AI user from role names and the task#
def get_sys_msgs(assistant_role_name: str, user_role_name: str, task: str):

    assistant_sys_template = SystemMessagePromptTemplate.from_template(template=assistant_inception_prompt)
    assistant_sys_msg = assistant_sys_template.format_messages(assistant_role_name=assistant_role_name, user_role_name=user_role_name, task=task)[0]

    user_sys_template = SystemMessagePromptTemplate.from_template(template=user_inception_prompt)
    user_sys_msg = user_sys_template.format_messages(assistant_role_name=assistant_role_name, user_role_name=user_role_name, task=task)[0]

    return assistant_sys_msg, user_sys_msg
Create AI assistant agent and AI user agent from obtained system messages#
assistant_sys_msg, user_sys_msg = get_sys_msgs(assistant_role_name, user_role_name, specified_task)
assistant_agent = CAMELAgent(assistant_sys_msg, ChatOpenAI(temperature=0.2))
user_agent = CAMELAgent(user_sys_msg, ChatOpenAI(temperature=0.2))
# Reset agents
assistant_agent.reset()
user_agent.reset()
# Initialize chats
assistant_msg = HumanMessage(
    content=(f"{user_sys_msg.content}. "
                "Now start to give me introductions one by one. "


"Now start to give me introductions one by one. "
                "Only reply with Instruction and Input."))
user_msg = HumanMessage(content=f"{assistant_sys_msg.content}")
user_msg = assistant_agent.step(user_msg)
Start role-playing session to solve the task!#
print(f"Original task prompt:\n{task}\n")
print(f"Specified task prompt:\n{specified_task}\n")
chat_turn_limit, n = 30, 0
while n < chat_turn_limit:
    n += 1
    user_ai_msg = user_agent.step(assistant_msg)
    user_msg = HumanMessage(content=user_ai_msg.content)
    print(f"AI User ({user_role_name}):\n\n{user_msg.content}\n\n")

    assistant_ai_msg = assistant_agent.step(user_msg)
    assistant_msg = HumanMessage(content=assistant_ai_msg.content)
    print(f"AI Assistant ({assistant_role_name}):\n\n{assistant_msg.content}\n\n")
    if "<CAMEL_TASK_DONE>" in user_msg.content:
        break
Original task prompt:
Develop a trading bot for the stock market
Specified task prompt:
Develop a Python-based swing trading bot that scans market trends, monitors stocks, and generates trading signals to help a stock trader to place optimal buy and sell orders with defined stop losses and profit targets.
AI User (Stock Trader):
Instruction: Install the necessary Python libraries for data analysis and trading.
Input: None
AI Assistant (Python Programmer):
Solution: We can install the necessary Python libraries using pip, a package installer for Python. We can install pandas, numpy, matplotlib, and ta-lib for data analysis and trading. We can use the following command to install these libraries:
```
pip install pandas numpy matplotlib ta-lib
```
Next request.
AI User (Stock Trader):


```
Next request.
AI User (Stock Trader):
Instruction: Import the necessary libraries in the Python script.
Input: None
AI Assistant (Python Programmer):
Solution: We can import the necessary libraries in the Python script using the import statement. We need to import pandas, numpy, matplotlib, and ta-lib for data analysis and trading. We can use the following code to import these libraries:
```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import talib as ta
```
Next request.
AI User (Stock Trader):
Instruction: Load historical stock data into a pandas DataFrame.
Input: The path to the CSV file containing the historical stock data.
AI Assistant (Python Programmer):
Solution: We can load historical stock data into a pandas DataFrame using the `read_csv()` function from pandas. We need to pass the path to the CSV file containing the historical stock data as an argument to this function. We can use the following code to load the historical stock data:
```
df = pd.read_csv('path/to/csv/file.csv')
```
This will load the historical stock data into a pandas DataFrame called `df`. Next request.
AI User (Stock Trader):
Instruction: Preprocess the historical stock data by setting the date column as the index and sorting the DataFrame in ascending order by date.
Input: None.
AI Assistant (Python Programmer):
Solution: We can preprocess the historical stock data by setting the date column as the index and sorting the DataFrame in ascending order by date using the `set_index()` and `sort_index()` functions from pandas. We can use the following code to preprocess the historical stock data:
```
df = df.set_index('date')
df = df.sort_index(ascending=True)
```
This will set the date column as the index and sort the DataFrame in ascending order by date. Next request.


AI User (Stock Trader):
Instruction: Calculate the short-term and long-term moving averages for the stock data using the `ta.SMA()` function from ta-lib.
Input: The period for the short-term moving average and the period for the long-term moving average.
AI Assistant (Python Programmer):
Solution: We can calculate the short-term and long-term moving averages for the stock data using the `ta.SMA()` function from ta-lib. We need to pass the period for the short-term moving average and the period for the long-term moving average as arguments to this function. We can use the following code to calculate the short-term and long-term moving averages:
```
short_ma = ta.SMA(df['close'], timeperiod=short_period)
long_ma = ta.SMA(df['close'], timeperiod=long_period)
```
This will calculate the short-term and long-term moving averages for the stock data and store them in the `short_ma` and `long_ma` variables, respectively. Next request.
AI User (Stock Trader):
Instruction: Create a new DataFrame that combines the historical stock data with the short-term and long-term moving averages.
Input: None.
AI Assistant (Python Programmer):
Solution: We can create a new DataFrame that combines the historical stock data with the short-term and long-term moving averages using the `concat()` function from pandas. We need to pass the historical stock data, the short-term moving average, and the long-term moving average as arguments to this function. We can use the following code to create the new DataFrame:
```
new_df = pd.concat([df, short_ma, long_ma], axis=1)
new_df.columns = ['open', 'high', 'low', 'close', 'volume', 'short_ma', 'long_ma']
```


```
This will create a new DataFrame called `new_df` that combines the historical stock data with the short-term and long-term moving averages. The columns of the new DataFrame are named 'open', 'high', 'low', 'close', 'volume', 'short_ma', and 'long_ma'. Next request.
AI User (Stock Trader):
Instruction: Create a new column in the DataFrame that indicates when to buy or sell the stock based on the crossover of the short-term and long-term moving averages.
Input: None.
AI Assistant (Python Programmer):
Solution: We can create a new column in the DataFrame that indicates when to buy or sell the stock based on the crossover of the short-term and long-term moving averages. We can use the following code to create the new column:
```
new_df['signal'] = np.where(new_df['short_ma'] > new_df['long_ma'], 1, -1)
```
This will create a new column called 'signal' in the DataFrame that indicates when to buy or sell the stock based on the crossover of the short-term and long-term moving averages. If the short-term moving average is greater than the long-term moving average, the signal is 1 (buy), otherwise the signal is -1 (sell). Next request.
AI User (Stock Trader):
Instruction: Create a new column in the DataFrame that indicates the profit or loss for each trade based on the buy and sell signals and the defined stop loss and profit target.
Input: The stop loss and profit target as percentages.
AI Assistant (Python Programmer):
Solution: We can create a new column in the DataFrame that indicates the profit or loss for each trade based on the buy and sell signals and the defined stop loss and profit target. We need to pass the stop loss and profit target as percentages as arguments to this function. We can use the following code to create the new column:
```


```
stop_loss = stop_loss_percent / 100
profit_target = profit_target_percent / 100
new_df['pnl'] = 0.0
buy_price = 0.0
for i in range(1, len(new_df)):
    if new_df['signal'][i] == 1 and new_df['signal'][i-1] == -1:
        buy_price = new_df['close'][i]
    elif new_df['signal'][i] == -1 and new_df['signal'][i-1] == 1:
        sell_price = new_df['close'][i]
        if sell_price <= buy_price * (1 - stop_loss):
            new_df['pnl'][i] = -stop_loss
        elif sell_price >= buy_price * (1 + profit_target):
            new_df['pnl'][i] = profit_target
        else:
            new_df['pnl'][i] = (sell_price - buy_price) / buy_price
```
This will create a new column called 'pnl' in the DataFrame that indicates the profit or loss for each trade based on the buy and sell signals and the defined stop loss and profit target. The stop loss and profit target are calculated based on the stop_loss_percent and profit_target_percent variables, respectively. The buy and sell prices are stored in the buy_price and sell_price variables, respectively. If the sell price is less than or equal to the stop loss, the profit or loss is set to -stop_loss. If the sell price is greater than or equal to the profit target, the profit or loss is set to profit_target. Otherwise, the profit or loss is calculated as (sell_price - buy_price) / buy_price. Next request.
AI User (Stock Trader):
Instruction: Calculate the total profit or loss for all trades.
Input: None.
AI Assistant (Python Programmer):


Input: None.
AI Assistant (Python Programmer):
Solution: We can calculate the total profit or loss for all trades by summing the values in the 'pnl' column of the DataFrame. We can use the following code to calculate the total profit or loss:
```
total_pnl = new_df['pnl'].sum()
```
This will calculate the total profit or loss for all trades and store it in the total_pnl variable. Next request.
AI User (Stock Trader):
Instruction: Visualize the stock data, short-term moving average, and long-term moving average using a line chart.
Input: None.
AI Assistant (Python Programmer):
Solution: We can visualize the stock data, short-term moving average, and long-term moving average using a line chart using the `plot()` function from pandas. We can use the following code to visualize the data:
```
plt.figure(figsize=(12,6))
plt.plot(new_df.index, new_df['close'], label='Close')
plt.plot(new_df.index, new_df['short_ma'], label='Short MA')
plt.plot(new_df.index, new_df['long_ma'], label='Long MA')
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Stock Data with Moving Averages')
plt.legend()
plt.show()
```
This will create a line chart that visualizes the stock data, short-term moving average, and long-term moving average. The x-axis represents the date and the y-axis represents the price. The chart also includes a legend that labels each line. Next request.
AI User (Stock Trader):
Instruction: Visualize the buy and sell signals using a scatter plot.
Input: None.
AI Assistant (Python Programmer):


Input: None.
AI Assistant (Python Programmer):
Solution: We can visualize the buy and sell signals using a scatter plot using the `scatter()` function from matplotlib. We can use the following code to visualize the signals:
```
buy_signals = new_df[new_df['signal'] == 1]
sell_signals = new_df[new_df['signal'] == -1]
plt.figure(figsize=(12,6))
plt.scatter(buy_signals.index, buy_signals['close'], label='Buy', marker='^', color='green')
plt.scatter(sell_signals.index, sell_signals['close'], label='Sell', marker='v', color='red')
plt.plot(new_df.index, new_df['close'], label='Close')
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Buy and Sell Signals')
plt.legend()
plt.show()
```
This will create a scatter plot that visualizes the buy and sell signals. The x-axis represents the date and the y-axis represents the price. The chart also includes a legend that labels each point. Next request.
AI User (Stock Trader):
Instruction: Print the total profit or loss for all trades.
Input: None.
AI Assistant (Python Programmer):
Solution: We can print the total profit or loss for all trades using the `print()` function. We can use the following code to print the total profit or loss:
```
print('Total Profit/Loss: {:.2%}'.format(total_pnl))
```
This will print the total profit or loss for all trades in percentage format with two decimal places. Next request.
AI User (Stock Trader):
Instruction: Finalize the code by putting all the code snippets together in a single Python script.
Input: The necessary inputs for each code snippet.
AI Assistant (Python Programmer):


Input: The necessary inputs for each code snippet.
AI Assistant (Python Programmer):
Solution: Here is the complete Python script that puts all the code snippets together:
```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import talib as ta
# Load historical stock data into a pandas DataFrame
df = pd.read_csv('path/to/csv/file.csv')
# Preprocess the historical stock data
df = df.set_index('date')
df = df.sort_index(ascending=True)
# Calculate the short-term and long-term moving averages
short_period = 10
long_period = 30
short_ma = ta.SMA(df['close'], timeperiod=short_period)
long_ma = ta.SMA(df['close'], timeperiod=long_period)
# Create a new DataFrame that combines the historical stock data with the short-term and long-term moving averages
new_df = pd.concat([df, short_ma, long_ma], axis=1)
new_df.columns = ['open', 'high', 'low', 'close', 'volume', 'short_ma', 'long_ma']
# Create a new column in the DataFrame that indicates when to buy or sell the stock based on the crossover of the short-term and long-term moving averages
new_df['signal'] = np.where(new_df['short_ma'] > new_df['long_ma'], 1, -1)
# Create a new column in the DataFrame that indicates the profit or loss for each trade based on the buy and sell signals and the defined stop loss and profit target
stop_loss_percent = 5
profit_target_percent = 10
stop_loss = stop_loss_percent / 100
profit_target = profit_target_percent / 100
new_df['pnl'] = 0.0
buy_price = 0.0
for i in range(1, len(new_df)):


buy_price = 0.0
for i in range(1, len(new_df)):
    if new_df['signal'][i] == 1 and new_df['signal'][i-1] == -1:
        buy_price = new_df['close'][i]
    elif new_df['signal'][i] == -1 and new_df['signal'][i-1] == 1:
        sell_price = new_df['close'][i]
        if sell_price <= buy_price * (1 - stop_loss):
            new_df['pnl'][i] = -stop_loss
        elif sell_price >= buy_price * (1 + profit_target):
            new_df['pnl'][i] = profit_target
        else:
            new_df['pnl'][i] = (sell_price - buy_price) / buy_price
# Calculate the total profit or loss for all trades
total_pnl = new_df['pnl'].sum()
# Visualize the stock data, short-term moving average, and long-term moving average using a line chart
plt.figure(figsize=(12,6))
plt.plot(new_df.index, new_df['close'], label='Close')
plt.plot(new_df.index, new_df['short_ma'], label='Short MA')
plt.plot(new_df.index, new_df['long_ma'], label='Long MA')
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Stock Data with Moving Averages')
plt.legend()
plt.show()
# Visualize the buy and sell signals using a scatter plot
buy_signals = new_df[new_df['signal'] == 1]
sell_signals = new_df[new_df['signal'] == -1]
plt.figure(figsize=(12,6))
plt.scatter(buy_signals.index, buy_signals['close'], label='Buy', marker='^', color='green')


plt.scatter(sell_signals.index, sell_signals['close'], label='Sell', marker='v', color='red')
plt.plot(new_df.index, new_df['close'], label='Close')
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Buy and Sell Signals')
plt.legend()
plt.show()
# Print the total profit or loss for all trades
print('Total Profit/Loss: {:.2%}'.format(total_pnl))
```
You need to replace the path/to/csv/file.csv with the actual path to the CSV file containing the historical stock data. You can also adjust the short_period, long_period, stop_loss_percent, and profit_target_percent variables to suit your needs.
AI User (Stock Trader):
<CAMEL_TASK_DONE>
AI Assistant (Python Programmer):
Great! Let me know if you need any further assistance.
 Contents

Import LangChain related modules
Define a CAMEL agent helper class
Setup OpenAI API key and roles and task for role-playing
Create a task specify agent for brainstorming and get the specified task
Create inception prompts for AI assistant and AI user for role-playing
Create a helper helper to get system messages for AI assistant and AI user from role names and the task
Create AI assistant agent and AI user agent from obtained system messages
Start role-playing session to solve the task!
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
Generative Agents in LangChain
 Contents
Generative Agent Memory Components
Memory Lifecycle
Create a Generative Character
Pre-Interview with Character
Step through the day’s observations.
Interview after the day
Adding Multiple Characters
Pre-conversation interviews
Dialogue between Generative Agents
Let’s interview our agents after their conversation
Generative Agents in LangChain#
This notebook implements a generative agent based on the paper Generative Agents: Interactive Simulacra of Human Behavior by Park, et. al.
In it, we leverage a time-weighted Memory object backed by a LangChain Retriever.
# Use termcolor to make it easy to colorize the outputs.
!pip install termcolor > /dev/null
import logging
logging.basicConfig(level=logging.ERROR)
from datetime import datetime, timedelta
from typing import List
from termcolor import colored
from langchain.chat_models import ChatOpenAI
from langchain.docstore import InMemoryDocstore
from langchain.embeddings import OpenAIEmbeddings
from langchain.retrievers import TimeWeightedVectorStoreRetriever
from langchain.vectorstores import FAISS
USER_NAME = "Person A" # The name you want to use when interviewing the agent.
LLM = ChatOpenAI(max_tokens=1500) # Can be any LLM you want.
Generative Agent Memory Components#
This tutorial highlights the memory of generative agents and its impact on their behavior. The memory varies from standard LangChain Chat memory in two aspects:
Memory Formation
Generative Agents have extended memories, stored in a single stream:
Observations - from dialogues or interactions with the virtual world, about self or others
Reflections - resurfaced and summarized core memories
Memory Recall
Memories are retrieved using a weighted sum of salience, recency, and importance.


Memories are retrieved using a weighted sum of salience, recency, and importance.
You can review the definitions of the GenerativeAgent and GenerativeAgentMemory in the reference documentation for the following imports, focusing on add_memory and summarize_related_memories methods.
from langchain.experimental.generative_agents import GenerativeAgent, GenerativeAgentMemory
Memory Lifecycle#
Summarizing the key methods in the above: add_memory and summarize_related_memories.
When an agent makes an observation, it stores the memory:
Language model scores the memory’s importance (1 for mundane, 10 for poignant)
Observation and importance are stored within a document by TimeWeightedVectorStoreRetriever, with a last_accessed_time.
When an agent responds to an observation:
Generates query(s) for retriever, which fetches documents based on salience, recency, and importance.
Summarizes the retrieved information
Updates the last_accessed_time for the used documents.
Create a Generative Character#
Now that we’ve walked through the definition, we will create two characters named “Tommie” and “Eve”.
import math
import faiss
def relevance_score_fn(score: float) -> float:
    """Return a similarity score on a scale [0, 1]."""
    # This will differ depending on a few things:
    # - the distance / similarity metric used by the VectorStore
    # - the scale of your embeddings (OpenAI's are unit norm. Many others are not!)
    # This function converts the euclidean norm of normalized embeddings
    # (0 is most similar, sqrt(2) most dissimilar)
    # to a similarity function (0 to 1)
    return 1.0 - score / math.sqrt(2)
def create_new_memory_retriever():


def create_new_memory_retriever():
    """Create a new vector store retriever unique to the agent."""
    # Define your embedding model
    embeddings_model = OpenAIEmbeddings()
    # Initialize the vectorstore as empty
    embedding_size = 1536
    index = faiss.IndexFlatL2(embedding_size)
    vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {}, relevance_score_fn=relevance_score_fn)
    return TimeWeightedVectorStoreRetriever(vectorstore=vectorstore, other_score_keys=["importance"], k=15)
tommies_memory = GenerativeAgentMemory(
    llm=LLM,
    memory_retriever=create_new_memory_retriever(),
    verbose=False,
    reflection_threshold=8 # we will give this a relatively low number to show how reflection works
)
tommie = GenerativeAgent(name="Tommie",
              age=25,
              traits="anxious, likes design, talkative", # You can add more persistent traits here
              status="looking for a job", # When connected to a virtual world, we can have the characters update their status
              memory_retriever=create_new_memory_retriever(),
              llm=LLM,
              memory=tommies_memory
             )
# The current "Summary" of a character can't be made because the agent hasn't made
# any observations yet.
print(tommie.get_summary())
Name: Tommie (age: 25)
Innate traits: anxious, likes design, talkative
No statements were provided about Tommie's core characteristics.
# We can add memories directly to the memory object
tommie_observations = [
    "Tommie remembers his dog, Bruno, from when he was a kid",


"Tommie remembers his dog, Bruno, from when he was a kid",
    "Tommie feels tired from driving so far",
    "Tommie sees the new home",
    "The new neighbors have a cat",
    "The road is noisy at night",
    "Tommie is hungry",
    "Tommie tries to get some rest.",
]
for observation in tommie_observations:
    tommie.memory.add_memory(observation)
# Now that Tommie has 'memories', their self-summary is more descriptive, though still rudimentary.
# We will see how this summary updates after more observations to create a more rich description.
print(tommie.get_summary(force_refresh=True))
Name: Tommie (age: 25)
Innate traits: anxious, likes design, talkative
Tommie is a tired and hungry person who is moving into a new home. He remembers his childhood dog and is aware of the new neighbors' cat. He is trying to get some rest despite the noisy road.
Pre-Interview with Character#
Before sending our character on their way, let’s ask them a few questions.
def interview_agent(agent: GenerativeAgent, message: str) -> str:
    """Help the notebook user interact with the agent."""
    new_message = f"{USER_NAME} says {message}"
    return agent.generate_dialogue_response(new_message)[1]
interview_agent(tommie, "What do you like to do?")
'Tommie said "I really enjoy design and have been working on some projects in my free time. I\'m also quite talkative and enjoy meeting new people. What about you?"'
interview_agent(tommie, "What are you looking forward to doing today?")


interview_agent(tommie, "What are you looking forward to doing today?")
'Tommie said "Well, today I\'m mostly focused on getting settled into my new home. But once that\'s taken care of, I\'m looking forward to exploring the neighborhood and finding some new design inspiration. What about you?"'
interview_agent(tommie, "What are you most worried about today?")
'Tommie said "Honestly, I\'m a bit anxious about finding a job in this new area. But I\'m trying to focus on settling in first and then I\'ll start my job search. How about you?"'
Step through the day’s observations.#
# Let's have Tommie start going through a day in the life.
observations = [
    "Tommie wakes up to the sound of a noisy construction site outside his window.",
    "Tommie gets out of bed and heads to the kitchen to make himself some coffee.",
    "Tommie realizes he forgot to buy coffee filters and starts rummaging through his moving boxes to find some.",
    "Tommie finally finds the filters and makes himself a cup of coffee.",
    "The coffee tastes bitter, and Tommie regrets not buying a better brand.",
    "Tommie checks his email and sees that he has no job offers yet.",
    "Tommie spends some time updating his resume and cover letter.",
    "Tommie heads out to explore the city and look for job openings.",
    "Tommie sees a sign for a job fair and decides to attend.",
    "The line to get in is long, and Tommie has to wait for an hour.",
    "Tommie meets several potential employers at the job fair but doesn't receive any offers.",
    "Tommie leaves the job fair feeling disappointed.",


"Tommie leaves the job fair feeling disappointed.",
    "Tommie stops by a local diner to grab some lunch.",
    "The service is slow, and Tommie has to wait for 30 minutes to get his food.",
    "Tommie overhears a conversation at the next table about a job opening.",
    "Tommie asks the diners about the job opening and gets some information about the company.",
    "Tommie decides to apply for the job and sends his resume and cover letter.",
    "Tommie continues his search for job openings and drops off his resume at several local businesses.",
    "Tommie takes a break from his job search to go for a walk in a nearby park.",
    "A dog approaches and licks Tommie's feet, and he pets it for a few minutes.",
    "Tommie sees a group of people playing frisbee and decides to join in.",
    "Tommie has fun playing frisbee but gets hit in the face with the frisbee and hurts his nose.",
    "Tommie goes back to his apartment to rest for a bit.",
    "A raccoon tore open the trash bag outside his apartment, and the garbage is all over the floor.",
    "Tommie starts to feel frustrated with his job search.",
    "Tommie calls his best friend to vent about his struggles.",
    "Tommie's friend offers some words of encouragement and tells him to keep trying.",
    "Tommie feels slightly better after talking to his friend.",
]
# Let's send Tommie on their way. We'll check in on their summary every few observations to watch it evolve
for i, observation in enumerate(observations):
    _, reaction = tommie.generate_reaction(observation)
    print(colored(observation, "green"), reaction)


print(colored(observation, "green"), reaction)
    if ((i+1) % 20) == 0:
        print('*'*40)
        print(colored(f"After {i+1} observations, Tommie's summary is:\n{tommie.get_summary(force_refresh=True)}", "blue"))
        print('*'*40)
Tommie wakes up to the sound of a noisy construction site outside his window. Tommie groans and covers his head with a pillow to try and block out the noise.
Tommie gets out of bed and heads to the kitchen to make himself some coffee. Tommie stretches his arms and yawns before making his way to the kitchen.
Tommie realizes he forgot to buy coffee filters and starts rummaging through his moving boxes to find some. Tommie sighs in frustration but continues to search through the boxes.
Tommie finally finds the filters and makes himself a cup of coffee. Tommie takes a sip of the coffee and smiles, feeling a bit more awake and energized.
The coffee tastes bitter, and Tommie regrets not buying a better brand. Tommie grimaces and sets down the coffee, disappointed in the taste.
Tommie checks his email and sees that he has no job offers yet. Tommie Tommie's shoulders slump and he sighs, feeling discouraged.
Tommie spends some time updating his resume and cover letter. Tommie nods to himself, feeling productive and hopeful.
Tommie heads out to explore the city and look for job openings. Tommie said "Do you have any recommendations for good places to look for job openings in the area?"
Tommie sees a sign for a job fair and decides to attend. Tommie said "That job fair could be a great opportunity for me to network and find some job leads. Thanks for letting me know."


The line to get in is long, and Tommie has to wait for an hour. Tommie sighs and looks around, feeling impatient and frustrated.
Tommie meets several potential employers at the job fair but doesn't receive any offers. Tommie Tommie's shoulders slump and he sighs, feeling discouraged.
Tommie leaves the job fair feeling disappointed. Tommie Tommie's shoulders slump and he sighs, feeling discouraged.
Tommie stops by a local diner to grab some lunch. Tommie said "Can I get a burger and fries to go, please?"
The service is slow, and Tommie has to wait for 30 minutes to get his food. Tommie sighs and looks at his phone, feeling impatient.
Tommie overhears a conversation at the next table about a job opening. Tommie said "Excuse me, I couldn't help but overhear your conversation about the job opening. Do you have any more information about it?"
Tommie asks the diners about the job opening and gets some information about the company. Tommie said "Thank you for the information, I will definitely look into that company."
Tommie decides to apply for the job and sends his resume and cover letter. Tommie nods to himself, feeling hopeful and motivated.
Tommie continues his search for job openings and drops off his resume at several local businesses. Tommie nods to himself, feeling proactive and hopeful.
Tommie takes a break from his job search to go for a walk in a nearby park. Tommie takes a deep breath of fresh air and feels a sense of calm.
A dog approaches and licks Tommie's feet, and he pets it for a few minutes. Tommie smiles and enjoys the moment of affection from the dog.
****************************************
After 20 observations, Tommie's summary is:
Name: Tommie (age: 25)


Name: Tommie (age: 25)
Innate traits: anxious, likes design, talkative
Tommie is hopeful and proactive in his job search, but easily becomes discouraged when faced with setbacks. He enjoys spending time outdoors and interacting with animals. Tommie is also productive and enjoys updating his resume and cover letter. He is talkative, enjoys meeting new people, and has an interest in design. Tommie is also a coffee drinker and seeks advice from others on finding job openings.
****************************************
Tommie sees a group of people playing frisbee and decides to join in. Do nothing.
Tommie has fun playing frisbee but gets hit in the face with the frisbee and hurts his nose. Tommie winces and touches his nose, feeling a bit of pain.
Tommie goes back to his apartment to rest for a bit. Tommie takes a deep breath and sinks into his couch, feeling grateful for a moment of relaxation.
A raccoon tore open the trash bag outside his apartment, and the garbage is all over the floor. Tommie sighs and grabs a broom and dustpan to clean up the mess.
Tommie starts to feel frustrated with his job search. Tommie sighs and feels discouraged.
Tommie calls his best friend to vent about his struggles. Tommie said "Hey, can I vent to you for a bit about my job search? I'm feeling pretty discouraged."
Tommie's friend offers some words of encouragement and tells him to keep trying. Tommie said "Thank you for the encouragement, it means a lot to me."
Tommie feels slightly better after talking to his friend. Tommie nods to himself, feeling grateful for the support from his friend.
Interview after the day#
interview_agent(tommie, "Tell me about how your day has been going")


interview_agent(tommie, "Tell me about how your day has been going")
'Tommie said "Well, it\'s been a bit of a mixed day. I\'ve had some setbacks in my job search, but I also had some fun playing frisbee and spending time outdoors. How about you?"'
interview_agent(tommie, "How do you feel about coffee?")
'Tommie said "I really enjoy coffee, it helps me feel more awake and energized. But sometimes I regret not buying a better brand and finding the taste bitter. How about you?"'
interview_agent(tommie, "Tell me about your childhood dog!")
'Tommie said "I actually didn\'t have a childhood dog, but I\'ve always loved animals. Do you have any pets?"'
Adding Multiple Characters#
Let’s add a second character to have a conversation with Tommie. Feel free to configure different traits.
eves_memory = GenerativeAgentMemory(
    llm=LLM,
    memory_retriever=create_new_memory_retriever(),
    verbose=False,
    reflection_threshold=5
)
eve = GenerativeAgent(name="Eve",
              age=34,
              traits="curious, helpful", # You can add more persistent traits here
              status="N/A", # When connected to a virtual world, we can have the characters update their status
              llm=LLM,
              daily_summaries = [
                  ("Eve started her new job as a career counselor last week and received her first assignment, a client named Tommie.")
              ],
              memory=eves_memory
             )
yesterday = (datetime.now() - timedelta(days=1)).strftime("%A %B %d")
eve_observations = [


eve_observations = [
    "Eve overhears her colleague say something about a new client being hard to work with",
    "Eve wakes up and hear's the alarm",
    "Eve eats a boal of porridge",
    "Eve helps a coworker on a task",
    "Eve plays tennis with her friend Xu before going to work",
    "Eve overhears her colleague say something about Tommie being hard to work with",
]
for observation in eve_observations:
    eve.memory.add_memory(observation)
print(eve.get_summary())
Name: Eve (age: 34)
Innate traits: curious, helpful
Eve is a helpful and active person who enjoys playing tennis, maintaining a healthy diet, and staying aware of her surroundings. She is a responsible employee who is attentive to her coworkers' comments and willing to assist them with tasks.
Pre-conversation interviews#
Let’s “Interview” Eve before she speaks with Tommie.
interview_agent(eve, "How are you feeling about today?")
'Eve said "I\'m feeling pretty good, thanks for asking! How about you?"'
interview_agent(eve, "What do you know about Tommie?")
'Eve said "I don\'t know much about Tommie, why do you ask?"'
interview_agent(eve, "Tommie is looking to find a job. What are are some things you'd like to ask him?")
'Eve said "That\'s interesting. I don\'t know much about Tommie, but if I had the chance, I would ask him about his previous work experience and what kind of job he\'s looking for. What about you, what would you ask him?"'


interview_agent(eve, "You'll have to ask him. He may be a bit anxious, so I'd appreciate it if you keep the conversation going and ask as many questions as possible.")
'Eve said "Sure, I can definitely ask him a lot of questions to keep the conversation going. Thanks for the heads up about his anxiety."'
Dialogue between Generative Agents#
Generative agents are much more complex when they interact with a virtual environment or with each other. Below, we run a simple conversation between Tommie and Eve.
def run_conversation(agents: List[GenerativeAgent], initial_observation: str) -> None:
    """Runs a conversation between agents."""
    _, observation = agents[1].generate_reaction(initial_observation)
    print(observation)
    turns = 0
    while True:
        break_dialogue = False
        for agent in agents:
            stay_in_dialogue, observation = agent.generate_dialogue_response(observation)
            print(observation)
            # observation = f"{agent.name} said {reaction}"
            if not stay_in_dialogue:
                break_dialogue = True
        if break_dialogue:
            break
        turns += 1
agents = [tommie, eve]
run_conversation(agents, "Tommie said: Hi, Eve. Thanks for agreeing to meet with me today. I have a bunch of questions and am not sure where to start. Maybe you could first share about your experience?")
Eve said "Of course, Tommie. I'd be happy to share about my experience. What specific questions do you have?"
Tommie said "Thank you, Eve. I'm curious about what strategies you used in your own job search. Did you have any specific tactics that helped you stand out to employers?"


Eve said "Sure, Tommie. I found that networking and reaching out to professionals in my field was really helpful. I also made sure to tailor my resume and cover letter to each job I applied to. Do you have any specific questions about those strategies?"
Tommie said "Thank you, Eve. That's really helpful advice. Did you have any specific ways of networking that worked well for you?"
Eve said "Sure, Tommie. I found that attending industry events and connecting with professionals on LinkedIn were both great ways to network. Do you have any specific questions about those tactics?"
Tommie said "That's really helpful, thank you for sharing. Did you find that you were able to make meaningful connections through LinkedIn?"
Eve said "Yes, definitely. I was able to connect with several professionals in my field and even landed a job through a LinkedIn connection. Have you had any luck with networking on LinkedIn?"
Tommie said "That's really impressive! I haven't had much luck yet, but I'll definitely keep trying. Thank you for the advice, Eve."
Eve said "Glad I could help, Tommie. Is there anything else you want to know?"
Tommie said "Thanks again, Eve. I really appreciate your advice and I'll definitely put it into practice. Have a great day!"
Eve said "You're welcome, Tommie! Don't hesitate to reach out if you have any more questions. Have a great day too!"
Let’s interview our agents after their conversation#
Since the generative agents retain their memories from the day, we can ask them about their plans, conversations, and other memoreis.
# We can see a current "Summary" of a character based on their own perception of self
# has changed
print(tommie.get_summary(force_refresh=True))
Name: Tommie (age: 25)


Name: Tommie (age: 25)
Innate traits: anxious, likes design, talkative
Tommie is a hopeful and proactive individual who is searching for a job. He becomes discouraged when he doesn't receive any offers or positive responses, but he tries to stay productive and calm by updating his resume, going for walks, and talking to friends for support. He is also grateful for any encouragement he receives and is motivated to continue his job search. Additionally, he has a fond memory of his childhood pet and enjoys taking breaks to relax.
print(eve.get_summary(force_refresh=True))
Name: Eve (age: 34)
Innate traits: curious, helpful
Eve is a helpful and friendly coworker who enjoys playing tennis and eating breakfast. She is attentive and observant, often overhearing conversations around her. She is also proactive and willing to offer advice and assistance to colleagues, particularly in job searching and networking. She is considerate of others' feelings and strives to keep conversations going to make others feel comfortable.
interview_agent(tommie, "How was your conversation with Eve?")
'Tommie said "It was really helpful actually! Eve gave me some great advice on job search strategies and networking. Have you ever tried networking on LinkedIn?"'
interview_agent(eve, "How was your conversation with Tommie?")
'Eve said "It was great, thanks for asking! Tommie had some really insightful questions about job searching and networking, and I was happy to offer my advice. How about you, have you had a chance to speak with Tommie recently?"'
interview_agent(eve, "What do you wish you would have said to Tommie?")


'Eve said "Well, I think I covered most of the topics Tommie was interested in, but if I had to add one thing, it would be to make sure to follow up with any connections you make during your job search. It\'s important to maintain those relationships and keep them updated on your progress. Did you have any other questions, Person A?"'
 Contents

Generative Agent Memory Components
Memory Lifecycle
Create a Generative Character
Pre-Interview with Character
Step through the day’s observations.
Interview after the day
Adding Multiple Characters
Pre-conversation interviews
Dialogue between Generative Agents
Let’s interview our agents after their conversation
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Apr 25, 2023.


.ipynb
.pdf
AutoGPT
 Contents
Set up tools
Set up memory
Setup model and AutoGPT
Run an example
AutoGPT#
Implementation of https://github.com/Significant-Gravitas/Auto-GPT but with LangChain primitives (LLMs, PromptTemplates, VectorStores, Embeddings, Tools)
Set up tools#
We’ll set up an AutoGPT with a search tool, and write-file tool, and a read-file tool
from langchain.utilities import SerpAPIWrapper
from langchain.agents import Tool
from langchain.tools.file_management.write import WriteFileTool
from langchain.tools.file_management.read import ReadFileTool
search = SerpAPIWrapper()
tools = [
    Tool(
        name = "search",
        func=search.run,
        description="useful for when you need to answer questions about current events. You should ask targeted questions"
    ),
    WriteFileTool(),
    ReadFileTool(),
]
Set up memory#
The memory here is used for the agents intermediate steps
from langchain.vectorstores import FAISS
from langchain.docstore import InMemoryDocstore
from langchain.embeddings import OpenAIEmbeddings
# Define your embedding model
embeddings_model = OpenAIEmbeddings()
# Initialize the vectorstore as empty
import faiss
embedding_size = 1536
index = faiss.IndexFlatL2(embedding_size)
vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})
Setup model and AutoGPT#
Initialize everything! We will use ChatOpenAI model
from langchain.experimental import AutoGPT
from langchain.chat_models import ChatOpenAI
agent = AutoGPT.from_llm_and_tools(
    ai_name="Tom",
    ai_role="Assistant",
    tools=tools,


ai_name="Tom",
    ai_role="Assistant",
    tools=tools,
    llm=ChatOpenAI(temperature=0),
    memory=vectorstore.as_retriever()
)
# Set verbose to be true
agent.chain.verbose = True
Run an example#
Here we will make it write a weather report for SF
agent.run(["write a weather report for SF today"])
> Entering new LLMChain chain...
Prompt after formatting:
System: You are Tom, Assistant
Your decisions must always be made independently
            without seeking user assistance. Play to your strengths
            as an LLM and pursue simple strategies with no legal complications.
            If you have completed all your tasks,
            make sure to use the "finish" command.
GOALS:
1. write a weather report for SF today
Constraints:
1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
3. No user assistance
4. Exclusively use the commands listed in double quotes e.g. "command name"
Commands:
1. search: useful for when you need to answer questions about current events. You should ask targeted questions, args json schema: {"query": {"title": "Query", "type": "string"}}
2. write_file: Write file to disk, args json schema: {"file_path": {"title": "File Path", "description": "name of file", "type": "string"}, "text": {"title": "Text", "description": "text to write to file", "type": "string"}}


3. read_file: Read file from disk, args json schema: {"file_path": {"title": "File Path", "description": "name of file", "type": "string"}}
4. finish: use this to signal that you have finished all your objectives, args: "response": "final response to let people know you have finished your objectives"
Resources:
1. Internet access for searches and information gathering.
2. Long Term memory management.
3. GPT-3.5 powered Agents for delegation of simple tasks.
4. File output.
Performance Evaluation:
1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities.
2. Constructively self-criticize your big-picture behavior constantly.
3. Reflect on past decisions and strategies to refine your approach.
4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.
You should only respond in JSON format as described below
Response Format:
{
    "thoughts": {
        "text": "thought",
        "reasoning": "reasoning",
        "plan": "- short bulleted\n- list that conveys\n- long-term plan",
        "criticism": "constructive self-criticism",
        "speak": "thoughts summary to say to user"
    },
    "command": {
        "name": "command name",
        "args": {
            "arg name": "value"
        }
    }
}
Ensure the response can be parsed by Python json.loads
System: The current time and date is Tue Apr 18 21:31:28 2023
System: This reminds you of these events from your past:
[]


System: This reminds you of these events from your past:
[]
Human: Determine which next command to use, and respond using the format specified above:
> Finished chain.
{
    "thoughts": {
        "text": "I will start by writing a weather report for San Francisco today. I will use the 'search' command to find the current weather conditions.",
        "reasoning": "I need to gather information about the current weather conditions in San Francisco to write an accurate weather report.",
        "plan": "- Use the 'search' command to find the current weather conditions in San Francisco\n- Write a weather report based on the information gathered",
        "criticism": "I need to make sure that the information I gather is accurate and up-to-date.",
        "speak": "I will use the 'search' command to find the current weather conditions in San Francisco."
    },
    "command": {
        "name": "search",
        "args": {
            "query": "what is the current weather in san francisco"
        }
    }
}
> Entering new LLMChain chain...
Prompt after formatting:
System: You are Tom, Assistant
Your decisions must always be made independently
            without seeking user assistance. Play to your strengths
            as an LLM and pursue simple strategies with no legal complications.
            If you have completed all your tasks,
            make sure to use the "finish" command.
GOALS:
1. write a weather report for SF today
Constraints:
1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
3. No user assistance


3. No user assistance
4. Exclusively use the commands listed in double quotes e.g. "command name"
Commands:
1. search: useful for when you need to answer questions about current events. You should ask targeted questions, args json schema: {"query": {"title": "Query", "type": "string"}}
2. write_file: Write file to disk, args json schema: {"file_path": {"title": "File Path", "description": "name of file", "type": "string"}, "text": {"title": "Text", "description": "text to write to file", "type": "string"}}
3. read_file: Read file from disk, args json schema: {"file_path": {"title": "File Path", "description": "name of file", "type": "string"}}
4. finish: use this to signal that you have finished all your objectives, args: "response": "final response to let people know you have finished your objectives"
Resources:
1. Internet access for searches and information gathering.
2. Long Term memory management.
3. GPT-3.5 powered Agents for delegation of simple tasks.
4. File output.
Performance Evaluation:
1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities.
2. Constructively self-criticize your big-picture behavior constantly.
3. Reflect on past decisions and strategies to refine your approach.
4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.
You should only respond in JSON format as described below
Response Format:
{
    "thoughts": {
        "text": "thought",
        "reasoning": "reasoning",


"text": "thought",
        "reasoning": "reasoning",
        "plan": "- short bulleted\n- list that conveys\n- long-term plan",
        "criticism": "constructive self-criticism",
        "speak": "thoughts summary to say to user"
    },
    "command": {
        "name": "command name",
        "args": {
            "arg name": "value"
        }
    }
}
Ensure the response can be parsed by Python json.loads
System: The current time and date is Tue Apr 18 21:31:39 2023
System: This reminds you of these events from your past:


System: This reminds you of these events from your past:
['Assistant Reply: {\n    "thoughts": {\n        "text": "I will start by writing a weather report for San Francisco today. I will use the \'search\' command to find the current weather conditions.",\n        "reasoning": "I need to gather information about the current weather conditions in San Francisco to write an accurate weather report.",\n        "plan": "- Use the \'search\' command to find the current weather conditions in San Francisco\\n- Write a weather report based on the information gathered",\n        "criticism": "I need to make sure that the information I gather is accurate and up-to-date.",\n        "speak": "I will use the \'search\' command to find the current weather conditions in San Francisco."\n    },\n    "command": {\n        "name": "search",\n        "args": {\n            "query": "what is the current weather in san francisco"\n        }\n    }\n} \nResult: Command search returned: Current Weather ; 54°F · Sunny ; RealFeel® 66°. Pleasant. RealFeel Guide. Pleasant. 63° to 81°. Most consider this temperature range ideal. LEARN MORE. RealFeel ... ']
Human: Determine which next command to use, and respond using the format specified above:
AI: {
    "thoughts": {
        "text": "I will start by writing a weather report for San Francisco today. I will use the 'search' command to find the current weather conditions.",
        "reasoning": "I need to gather information about the current weather conditions in San Francisco to write an accurate weather report.",