Interacting with APIs | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesInteracting with APIsOn this pageInteracting with APIsLots of data and information is stored behind APIs.
This page covers all resources available in LangChain for working with APIs.Chainsâ€‹If you are just getting started, and you have relatively simple apis, you should get started with chains.
Chains are a sequence of predetermined steps, so they are good to get started with as they give you more control and let you
understand what is happening better.API ChainAgentsâ€‹Agents are more complex, and involve multiple queries to the LLM to understand what to do.
The downside of agents are that you have less control. The upside is that they are more powerful,
which allows you to use them on larger and more complex schemas. OpenAPI AgentPreviousExtractionNextChatbotsChainsAgentsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Analyzing structured data | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesAnalyzing structured dataOn this pageAnalyzing structured dataLots of data and information is stored in tabular data, whether it be csvs, excel sheets, or SQL tables.
This page covers all resources available in LangChain for working with data in this format.Document loadingâ€‹If you have text data stored in a tabular format, you may want to load the data into a Document and then index it as you would
other text/unstructured data. For this, you should use a document loader like the CSVLoader
and then you should create an index over that data, and query it that way.Queryingâ€‹If you have more numeric tabular data, or have a large amount of data and don't want to index it, you should get started
by looking at various chains and agents we have for dealing with this data.Chainsâ€‹If you are just getting started, and you have relatively small/simple tabular data, you should get started with chains.
Chains are a sequence of predetermined steps, so they are good to get started with as they give you more control and let you
understand what is happening better.SQL Database ChainAgentsâ€‹Agents are more complex, and involve multiple queries to the LLM to understand what to do.
The downside of agents are that you have less control. The upside is that they are more powerful,


which allows you to use them on larger databases and more complex schemas. SQL AgentPandas AgentCSV AgentPreviousQuestion answering over a group chat messages using Activeloop's DeepLakeNextExtractionDocument loadingQueryingChainsAgentsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


QA and Chat over Documents | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsContext aware text splitting and QA / ChatRunning LLMs locallyQuestion answering over a group chat messages using Activeloop's DeepLakeAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesQA and Chat over DocumentsOn this pageQA and Chat over DocumentsChat and Question-Answering (QA) over data are popular LLM use-cases.data can include many things, including:Unstructured data (e.g., PDFs)Structured data (e.g., SQL)Code (e.g., Python)LangChain supports Chat and QA on various data types:See here and here for Code See here for Structured dataBelow we will review Chat and QA on Unstructured data. Unstructured data can be loaded from many sources.Use the LangChain integration hub to browse the full set of loaders.Each loader returns data as a LangChain Document.Documents are turned into a Chat or QA app following the general steps below:Splitting: Text splitters break Documents into splits of specified size Storage: Storage (e.g., often a vectorstore) will house and often embed the splitsRetrieval: The app retrieves splits from storage (e.g., often with similar embeddings to the input question) Output: An LLM produces an answer using a prompt that includes the question and the retrieved splits Quickstartâ€‹The above pipeline can be wrapped with a VectorstoreIndexCreator.In particular: Specify a Document loaderThe splitting, storage, retrieval, and output generation stages are wrappedLet's load this blog post on agents as an example Document.We have a QA app in a few lines of code.Set environment variables and get packages:pip install openaipip install chromadbexport


lines of code.Set environment variables and get packages:pip install openaipip install chromadbexport OPENAI_API_KEY="..."Run:from langchain.document_loaders import WebBaseLoaderfrom langchain.indexes import VectorstoreIndexCreator# Document loaderloader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")# Index that wraps above stepsindex = VectorstoreIndexCreator().from_loaders([loader])# Question-answeringquestion = "What is Task Decomposition?"index.query(question)' Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It can be done using LLM with simple prompting, task-specific instructions, or human inputs. Tree of Thoughts (Yao et al. 2023) is an example of a task decomposition technique that explores multiple reasoning possibilities at each step and generates multiple thoughts per step, creating a tree structure.'Of course, some users do not want this level of abstraction.Below, we will discuss each stage in more detail.1. Loading, Splitting, Storageâ€‹1.1 Getting startedâ€‹Specify a Document loader.# Document loaderfrom langchain.document_loaders import WebBaseLoaderloader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")data = loader.load()Split the Document into chunks for embedding and vector storage.# Splitfrom langchain.text_splitter import RecursiveCharacterTextSplittertext_splitter = RecursiveCharacterTextSplitter(chunk_size = 500, chunk_overlap = 0)all_splits = text_splitter.split_documents(data)Embed and store the splits in a vector database (Chroma).# Store from langchain.vectorstores import Chromafrom langchain.embeddings import OpenAIEmbeddingsvectorstore = Chroma.from_documents(documents=all_splits,embedding=OpenAIEmbeddings())Here are the three pieces together:1.2 Going


are the three pieces together:1.2 Going Deeperâ€‹1.2.1 Integrationsâ€‹Document LoadersBrowse the > 120 data loader integrations here.See further documentation on loaders here.Document TransformersAll can ingest loaded Documents and process them (e.g., split).See further documentation on transformers here.VectorstoresBrowse the > 35 vectorstores integrations here.See further documentation on vectorstores here. 1.2.2 Retaining metadataâ€‹Context-aware splitters keep the location ("context") of each split in the original Document:Markdown filesCode (py or js)Documents2. Retrievalâ€‹2.1 Getting startedâ€‹Retrieve relevant splits for any question using similarity_search.question = "What are the approaches to Task Decomposition?"docs = vectorstore.similarity_search(question)len(docs)42.2 Going Deeperâ€‹2.2.1 Retrievalâ€‹Vectorstores are commonly used for retrieval.But, they are not the only option.For example, SVMs (see thread here) can also be used.LangChain has many retrievers including, but not limited to, vectorstores.All retrievers implement some common methods, such as get_relevant_documents().from langchain.retrievers import SVMRetrieversvm_retriever = SVMRetriever.from_documents(all_splits,OpenAIEmbeddings())docs_svm=svm_retriever.get_relevant_documents(question)len(docs)42.2.2 Advanced retrievalâ€‹Improve on similarity_search:MultiQueryRetriever generates variants of the input question to improve retrieval.Max marginal relevance selects for relevance and diversity among the retrieved documents.Documents can be filtered during retrieval using metadata filters.# MultiQueryRetrieverimport loggingfrom langchain.chat_models import ChatOpenAIfrom langchain.retrievers.multi_query import


loggingfrom langchain.chat_models import ChatOpenAIfrom langchain.retrievers.multi_query import MultiQueryRetrieverlogging.basicConfig()logging.getLogger('langchain.retrievers.multi_query').setLevel(logging.INFO)retriever_from_llm = MultiQueryRetriever.from_llm(retriever=vectorstore.as_retriever(),                                                  llm=ChatOpenAI(temperature=0))unique_docs = retriever_from_llm.get_relevant_documents(query=question)len(unique_docs)INFO:langchain.retrievers.multi_query:Generated queries: ['1. How can Task Decomposition be approached?', '2. What are the different methods for Task Decomposition?', '3. What are the various approaches to decomposing tasks?']53. QAâ€‹3.1 Getting startedâ€‹Distill the retrieved documents into an answer using an LLM (e.g., gpt-3.5-turbo) with RetrievalQA chain.from langchain.chat_models import ChatOpenAIllm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)from langchain.chains import RetrievalQAqa_chain = RetrievalQA.from_chain_type(llm,retriever=vectorstore.as_retriever())qa_chain({"query": question}){'query': 'What are the approaches to Task Decomposition?', 'result': 'The approaches to task decomposition include:\n\n1. Simple prompting: This approach involves using simple prompts or questions to guide the agent in breaking down a task into smaller subgoals. For example, the agent can be prompted with "Steps for XYZ" and asked to list the subgoals for achieving XYZ.\n\n2.


"Steps for XYZ" and asked to list the subgoals for achieving XYZ.\n\n2. Task-specific instructions: In this approach, task-specific instructions are provided to the agent to guide the decomposition process. For example, if the task is to write a novel, the agent can be instructed to "Write a story outline" as a subgoal.\n\n3. Human inputs: This approach involves incorporating human inputs in the task decomposition process. Humans can provide guidance, feedback, and suggestions to help the agent break down complex tasks into manageable subgoals.\n\nThese approaches aim to enable efficient handling of complex tasks by breaking them down into smaller, more manageable parts.'}3.2 Going Deeperâ€‹3.2.1 Integrationsâ€‹LLMsBrowse the > 55 LLM integrations here.See further documentation on LLMs here. 3.2.2 Running LLMs locallyâ€‹The popularity of PrivateGPT and GPT4All underscore the importance of running LLMs locally.LangChain has integrations with many open source LLMs that can be run locally.Using GPT4All is as simple as downloading the binary and then:from langchain.llms import GPT4Allfrom langchain.chains import RetrievalQAllm = GPT4All(model="/Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.bin",max_tokens=2048)qa_chain = RetrievalQA.from_chain_type(llm,retriever=vectorstore.as_retriever())qa_chain({"query": question}){'query': 'What are the approaches to Task Decomposition?', 'result': ' There are three main approaches to task decomposition: (1) using language models like GPT-3 for simple prompting such as "Steps for XYZ.\\n1.", (2) using task-specific instructions, and


such as "Steps for XYZ.\\n1.", (2) using task-specific instructions, and (3) with human inputs.'}3.2.2 Customizing the promptâ€‹The prompt in RetrievalQA chain can be easily customized.# Build promptfrom langchain.prompts import PromptTemplatetemplate = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum and keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. {context}Question: {question}Helpful Answer:"""QA_CHAIN_PROMPT = PromptTemplate(input_variables=["context", "question"],template=template,)# Run chainfrom langchain.chains import RetrievalQAllm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)qa_chain = RetrievalQA.from_chain_type(llm,                                       retriever=vectorstore.as_retriever(),                                       chain_type_kwargs={"prompt": QA_CHAIN_PROMPT})result = qa_chain({"query": question})result["result"]'The approaches to Task Decomposition are (1) using simple prompting by LLM, (2) using task-specific instructions, and (3) with human inputs. Thanks for asking!'3.2.3 Returning source documentsâ€‹The full set of retrieved documents used for answer distillation can be returned using return_source_documents=True.from langchain.chains import


documents used for answer distillation can be returned using return_source_documents=True.from langchain.chains import RetrievalQAqa_chain = RetrievalQA.from_chain_type(llm,retriever=vectorstore.as_retriever(),                                       return_source_documents=True)result = qa_chain({"query": question})print(len(result['source_documents']))result['source_documents'][0]4Document(page_content='Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.', metadata={'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'title': "LLM Powered Autonomous Agents | Lil'Log", 'description': 'Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.\nAgent System Overview In a LLM-powered autonomous agent system, LLM functions as the agentâ€™s brain, complemented by several key components:', 'language': 'en'})3.2.4 Citationsâ€‹Answer citations can be returned using RetrievalQAWithSourcesChain.from langchain.chains import RetrievalQAWithSourcesChainqa_chain = RetrievalQAWithSourcesChain.from_chain_type(llm,retriever=vectorstore.as_retriever())result =


= qa_chain({"question": question})result{'question': 'What are the approaches to Task Decomposition?', 'answer': 'The approaches to Task Decomposition include (1) using LLM with simple prompting, (2) using task-specific instructions, and (3) incorporating human inputs.\n', 'sources': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}3.2.5 Customizing retrieved docs in the LLM promptâ€‹Retrieved documents can be fed to an LLM for answer distillation in a few different ways.stuff, refine, map-reduce, and map-rerank chains for passing documents to an LLM prompt are well summarized here.stuff is commonly used because it simply "stuffs" all retrieved documents into the prompt.The load_qa_chain is an easy way to pass documents to an LLM using these various approaches (e.g., see chain_type).from langchain.chains.question_answering import load_qa_chainchain = load_qa_chain(llm, chain_type="stuff")chain({"input_documents": unique_docs, "question": question},return_only_outputs=True){'output_text': 'The approaches to task decomposition include (1) using simple prompting to break down tasks into subgoals, (2) providing task-specific instructions to guide the decomposition process, and (3) incorporating human inputs for task decomposition.'}We can also pass the chain_type to RetrievalQA.qa_chain = RetrievalQA.from_chain_type(llm,retriever=vectorstore.as_retriever(),                                       chain_type="stuff")result = qa_chain({"query": question})In summary, the user can choose the desired level of abstraction for QA:4.


question})In summary, the user can choose the desired level of abstraction for QA:4. Chatâ€‹4.1 Getting startedâ€‹To keep chat history, first specify a Memory buffer to track the conversation inputs / outputs.from langchain.memory import ConversationBufferMemorymemory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)The ConversationalRetrievalChain uses chat in the Memory buffer. from langchain.chains import ConversationalRetrievalChainretriever=vectorstore.as_retriever()chat = ConversationalRetrievalChain.from_llm(llm,retriever=retriever,memory=memory)result = chat({"question": "What are some of the main ideas in self-reflection?"})result['answer']"Some of the main ideas in self-reflection include:\n1. Iterative improvement: Self-reflection allows autonomous agents to improve by refining past action decisions and correcting mistakes.\n2. Trial and error: Self-reflection is crucial in real-world tasks where trial and error are inevitable.\n3. Two-shot examples: Self-reflection is created by showing pairs of failed trajectories and ideal reflections for guiding future changes in the plan.\n4. Working memory: Reflections are added to the agent's working memory, up to three, to be used as context for querying.\n5. Performance evaluation: Self-reflection involves continuously reviewing and analyzing actions, self-criticizing behavior, and reflecting on past decisions and strategies to refine approaches.\n6. Efficiency: Self-reflection encourages being smart and efficient, aiming to complete tasks in the least number of steps."The Memory buffer has context to resolve "it" ("self-reflection") in the below question.result = chat({"question": "How does the Reflexion paper handle it?"})result['answer']"The Reflexion paper handles self-reflection by showing two-shot examples to the Learning Language Model (LLM). Each example consists of


by showing two-shot examples to the Learning Language Model (LLM). Each example consists of a failed trajectory and an ideal reflection that guides future changes in the agent's plan. These reflections are then added to the agent's working memory, up to a maximum of three, to be used as context for querying the LLM. This allows the agent to iteratively improve its reasoning skills by refining past action decisions and correcting previous mistakes."4.2 Going deeperâ€‹The documentation on ConversationalRetrievalChain offers a few extensions, such as streaming and source documents.PreviousUse casesNextContext aware text splitting and QA / ChatQuickstart1. Loading, Splitting, Storage1.1 Getting started1.2 Going Deeper2. Retrieval2.1 Getting started2.2 Going Deeper3. QA3.1 Getting started3.2 Going Deeper4. Chat4.1 Getting started4.2 Going deeperCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Question answering over a group chat messages using Activeloop's DeepLake | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsContext aware text splitting and QA / ChatRunning LLMs locallyQuestion answering over a group chat messages using Activeloop's DeepLakeAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesQA and Chat over DocumentsQuestion answering over a group chat messages using Activeloop's DeepLakeOn this pageQuestion answering over a group chat messages using Activeloop's DeepLakeIn this tutorial, we are going to use Langchain + Activeloop's Deep Lake with GPT4 to semantically search and ask questions over a group chat.View a working demo here1. Install required packagesâ€‹python3 -m pip install --upgrade langchain 'deeplake[enterprise]' openai tiktoken2. Add API keysâ€‹import osimport getpassfrom langchain.document_loaders import PyPDFLoader, TextLoaderfrom langchain.embeddings.openai import OpenAIEmbeddingsfrom langchain.text_splitter import (    RecursiveCharacterTextSplitter,    CharacterTextSplitter,)from langchain.vectorstores import DeepLakefrom langchain.chains import ConversationalRetrievalChain, RetrievalQAfrom langchain.chat_models import ChatOpenAIfrom langchain.llms import OpenAIos.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")activeloop_token = getpass.getpass("Activeloop Token:")os.environ["ACTIVELOOP_TOKEN"] = activeloop_tokenos.environ["ACTIVELOOP_ORG"] = getpass.getpass("Activeloop Org:")org_id = os.environ["ACTIVELOOP_ORG"]embeddings = OpenAIEmbeddings()dataset_path = "hub://"


= OpenAIEmbeddings()dataset_path = "hub://" + org_id + "/data"2. Create sample dataâ€‹You can generate a sample group chat conversation using ChatGPT with this prompt:Generate a group chat conversation with three friends talking about their day, referencing real places and fictional names. Make it funny and as detailed as possible.I've already generated such a chat in messages.txt. We can keep it simple and use this for our example.3. Ingest chat embeddingsâ€‹We load the messages in the text file, chunk and upload to ActiveLoop Vector store.with open("messages.txt") as f:    state_of_the_union = f.read()text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)pages = text_splitter.split_text(state_of_the_union)text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)texts = text_splitter.create_documents(pages)print(texts)dataset_path = "hub://" + org + "/data"embeddings = OpenAIEmbeddings()db = DeepLake.from_documents(    texts, embeddings, dataset_path=dataset_path, overwrite=True)Optional: You can also use Deep Lake's Managed Tensor Database as a hosting service and run queries there. In order to do so, it is necessary to specify the runtime parameter as {'tensor_db': True} during the creation of the vector store. This configuration enables the execution of queries on the Managed Tensor Database, rather than on the client side. It should be noted that this functionality is not applicable to datasets stored locally or in-memory. In the event that a vector store has already been created outside of the Managed Tensor Database, it is possible to transfer it to the Managed Tensor Database by following the prescribed steps.# with open("messages.txt") as f:#     state_of_the_union = f.read()# text_splitter =


as f:#     state_of_the_union = f.read()# text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)# pages = text_splitter.split_text(state_of_the_union)# text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)# texts = text_splitter.create_documents(pages)# print(texts)# dataset_path = "hub://" + org + "/data"# embeddings = OpenAIEmbeddings()# db = DeepLake.from_documents(#     texts, embeddings, dataset_path=dataset_path, overwrite=True, runtime="tensor_db"# )4. Ask questionsâ€‹Now we can ask a question and get an answer back with a semantic search:db = DeepLake(dataset_path=dataset_path, read_only=True, embedding_function=embeddings)retriever = db.as_retriever()retriever.search_kwargs["distance_metric"] = "cos"retriever.search_kwargs["k"] = 4qa = RetrievalQA.from_chain_type(    llm=OpenAI(), chain_type="stuff", retriever=retriever, return_source_documents=False)# What was the restaurant the group was talking about called?query = input("Enter query:")# The Hungry Lobsterans = qa({"query": query})print(ans)PreviousRunning LLMs locallyNextAnalyzing structured data1. Install required packages2. Add API keys2. Create sample data3. Ingest chat embeddings4. Ask questionsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Context aware text splitting and QA / Chat | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsContext aware text splitting and QA / ChatRunning LLMs locallyQuestion answering over a group chat messages using Activeloop's DeepLakeAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesQA and Chat over DocumentsContext aware text splitting and QA / ChatContext aware text splitting and QA / ChatText splitting for vector storage often uses sentences or other delimiters to keep related text together. But many documents (such as Markdown files) have structure (headers) that can be explicitly used in splitting. The MarkdownHeaderTextSplitter lets a user split Markdown files files based on specified headers. This results in chunks that retain the header(s) that it came from in the metadata.This works nicely w/ SelfQueryRetriever.First, tell the retriever about our splits.Then, query based on the doc structure (e.g., "summarize the doc introduction"). Chunks only from that section of the Document will be filtered and used in chat / Q+A.Let's test this out on an example Notion page!First, I download the page to Markdown as explained here.# Load Notion page as a markdownfile filefrom langchain.document_loaders import NotionDirectoryLoaderpath = "../Notion_DB/"loader = NotionDirectoryLoader(path)docs = loader.load()md_file = docs[0].page_content# Let's create groups based on the section headers in our pagefrom langchain.text_splitter import MarkdownHeaderTextSplitterheaders_to_split_on = [    ("###", "Section"),]markdown_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)md_header_splits = markdown_splitter.split_text(md_file)Now, perform


= markdown_splitter.split_text(md_file)Now, perform text splitting on the header grouped documents. # Define our text splitterfrom langchain.text_splitter import RecursiveCharacterTextSplitterchunk_size = 500chunk_overlap = 0text_splitter = RecursiveCharacterTextSplitter(    chunk_size=chunk_size, chunk_overlap=chunk_overlap)all_splits = text_splitter.split_documents(md_header_splits)This sets us up well do perform metadata filtering based on the document structure.Let's bring this all togther by building a vectorstore first.pip install chromadb# Build vectorstore and keep the metadatafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.vectorstores import Chromavectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())Let's create a SelfQueryRetriever that can filter based upon metadata we defined.# Create retrieverfrom langchain.llms import OpenAIfrom langchain.retrievers.self_query.base import SelfQueryRetrieverfrom langchain.chains.query_constructor.base import AttributeInfo# Define our metadatametadata_field_info = [    AttributeInfo(        name="Section",        description="Part of the document that the text comes from",        type="string or list[string]",    ),]document_content_description = "Major sections of the document"# Define self query retriverllm = OpenAI(temperature=0)retriever = SelfQueryRetriever.from_llm(    llm, vectorstore, document_content_description, metadata_field_info, verbose=True)We can see that we can query only for texts in the Introduction of the document!# Testretriever.get_relevant_documents("Summarize the Introduction section of the document")    query='Introduction' filter=Comparison(comparator=<Comparator.EQ:


the document")    query='Introduction' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Introduction') limit=None    [Document(page_content='![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled.png)', metadata={'Section': 'Introduction'}),     Document(page_content='Q+A systems often use a two-step approach: retrieve relevant text chunks and then synthesize them into an answer. There many ways to approach this. For example, we recently [discussed](https://blog.langchain.dev/auto-evaluation-of-anthropic-100k-context-window/) the Retriever-Less option (at bottom in the below diagram), highlighting the Anthropic 100k context window model. Metadata filtering is an alternative approach that pre-filters chunks based on a user-defined criteria in a VectorDB using', metadata={'Section': 'Introduction'}),     Document(page_content='metadata tags prior to semantic search.', metadata={'Section': 'Introduction'})]# Testretriever.get_relevant_documents("Summarize the Introduction section of the document")    query='Introduction' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Introduction') limit=None    [Document(page_content='![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled.png)', metadata={'Section': 'Introduction'}),     Document(page_content='Q+A systems often use a two-step approach: retrieve relevant text chunks and then synthesize them into an answer. There many ways to approach this. For example, we recently


then synthesize them into an answer. There many ways to approach this. For example, we recently [discussed](https://blog.langchain.dev/auto-evaluation-of-anthropic-100k-context-window/) the Retriever-Less option (at bottom in the below diagram), highlighting the Anthropic 100k context window model. Metadata filtering is an alternative approach that pre-filters chunks based on a user-defined criteria in a VectorDB using', metadata={'Section': 'Introduction'}),     Document(page_content='metadata tags prior to semantic search.', metadata={'Section': 'Introduction'})]We can also look at other parts of the document.retriever.get_relevant_documents("Summarize the Testing section of the document")    query='Testing' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Testing') limit=None    [Document(page_content='![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled%202.png)', metadata={'Section': 'Testing'}),     Document(page_content='`SelfQueryRetriever` works well in [many cases](https://twitter.com/hwchase17/status/1656791488569954304/photo/1). For example, given [this test case](https://twitter.com/hwchase17/status/1656791488569954304?s=20):  \n![Untitled](Auto-Evaluation%20of%20Metadata%20Filtering%2018502448c85240828f33716740f9574b/Untitled%201.png)  \nThe query can be nicely broken up into semantic query and metadata filter:  \n```python\nsemantic query: "prompt injection"', metadata={'Section': 'Testing'}),


query: "prompt injection"', metadata={'Section': 'Testing'}),     Document(page_content='Below, we can see detailed results from the app:  \n- Kor extraction is above to perform the transformation between query and metadata format âœ…\n- Self-querying attempts to filter using the episode ID (`252`) in the query and fails ğŸš«\n- Baseline returns docs from 3 different episodes (one from `252`), confusing the answer ğŸš«', metadata={'Section': 'Testing'}),     Document(page_content='will use in retrieval [here](https://github.com/langchain-ai/auto-evaluator/blob/main/streamlit/kor_retriever_lex.py).', metadata={'Section': 'Testing'})]Now, we can create chat or Q+A apps that are aware of the explict document structure. The ability to retain document structure for metadata filtering can be helpful for complicated or longer documents.from langchain.chains import RetrievalQAfrom langchain.chat_models import ChatOpenAIllm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)qa_chain = RetrievalQA.from_chain_type(llm, retriever=retriever)qa_chain.run("Summarize the Testing section of the document")    query='Testing' filter=Comparison(comparator=<Comparator.EQ: 'eq'>, attribute='Section', value='Testing') limit=None    'The Testing section of the document describes the evaluation of the `SelfQueryRetriever` component in comparison to a baseline model. The evaluation was performed on a test case where the query was broken down into a semantic query and a metadata filter. The results showed that the `SelfQueryRetriever` component was able to perform the transformation between query and metadata format, but failed to filter using the episode ID in the query. The baseline model returned documents


metadata format, but failed to filter using the episode ID in the query. The baseline model returned documents from three different episodes, which confused the answer. The `SelfQueryRetriever` component was deemed to work well in many cases and will be used in retrieval.'PreviousQA and Chat over DocumentsNextRunning LLMs locallyCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Running LLMs locally | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsContext aware text splitting and QA / ChatRunning LLMs locallyQuestion answering over a group chat messages using Activeloop's DeepLakeAnalyzing structured dataExtractionInteracting with APIsChatbotsSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesQA and Chat over DocumentsRunning LLMs locallyOn this pageRunning LLMs locallyThe popularity of projects like PrivateGPT, llama.cpp, and GPT4All underscore the importance of running LLMs locally.LangChain has integrations with many open source LLMs that can be run locally.For example, here we show how to run GPT4All or Llama-v2 locally (e.g., on your laptop) using local embeddings and a local LLM.Document Loadingâ€‹First, install packages needed for local embeddings and vector storage.pip install gpt4all pip install chromadbLoad and split an example docucment.We'll use a blog post on agents as an example.from langchain.document_loaders import WebBaseLoaderloader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")data = loader.load()from langchain.text_splitter import RecursiveCharacterTextSplittertext_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)all_splits = text_splitter.split_documents(data)Next, the below steps will download the GPT4All embeddings locally (if you don't already have them).from langchain.vectorstores import Chromafrom langchain.embeddings import GPT4AllEmbeddingsvectorstore = Chroma.from_documents(documents=all_splits, embedding=GPT4AllEmbeddings())    Found model file at


embedding=GPT4AllEmbeddings())    Found model file at  /Users/rlm/.cache/gpt4all/ggml-all-MiniLM-L6-v2-f16.binTest similarity search is working with our local embeddings.question = "What are the approaches to Task Decomposition?"docs = vectorstore.similarity_search(question)len(docs)    4docs[0]    Document(page_content='Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.', metadata={'description': 'Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.\nAgent System Overview In a LLM-powered autonomous agent system, LLM functions as the agentâ€™s brain, complemented by several key components:', 'language': 'en', 'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'title': "LLM Powered Autonomous Agents | Lil'Log"})Modelâ€‹Llama-v2â€‹Download a GGML converted model (e.g., here).pip install llama-cpp-pythonTo enable use of GPU on Apple Silicon, follow the steps here to use the Python binding with Metal support.In particular, ensure that conda is using the correct virtual enviorment that you created (miniforge3).E.g., for me:conda activate


enviorment that you created (miniforge3).E.g., for me:conda activate /Users/rlm/miniforge3/envs/llamaWith this confirmed:CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install -U llama-cpp-python --no-cache-dirfrom langchain.llms import LlamaCppfrom langchain.callbacks.manager import CallbackManagerfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerSetting model parameters as noted in the llama.cpp docs.n_gpu_layers = 1  # Metal set to 1 is enough.n_batch = 512  # Should be between 1 and n_ctx, consider the amount of RAM of your Apple Silicon Chip.callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])# Make sure the model path is correct for your system!llm = LlamaCpp(    model_path="/Users/rlm/Desktop/Code/llama.cpp/llama-2-13b-chat.ggmlv3.q4_0.bin",    n_gpu_layers=n_gpu_layers,    n_batch=n_batch,    n_ctx=2048,    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls    callback_manager=callback_manager,    verbose=True,)    llama.cpp: loading model from /Users/rlm/Desktop/Code/llama.cpp/llama-2-13b-chat.ggmlv3.q4_0.bin    llama_model_load_internal: format     = ggjt v3 (latest)    llama_model_load_internal: n_vocab    = 32000    llama_model_load_internal: n_ctx      = 2048    llama_model_load_internal: n_embd     = 5120


llama_model_load_internal: n_embd     = 5120    llama_model_load_internal: n_mult     = 256    llama_model_load_internal: n_head     = 40    llama_model_load_internal: n_layer    = 40    llama_model_load_internal: n_rot      = 128    llama_model_load_internal: freq_base  = 10000.0    llama_model_load_internal: freq_scale = 1    llama_model_load_internal: ftype      = 2 (mostly Q4_0)    llama_model_load_internal: n_ff       = 13824    llama_model_load_internal: model size = 13B    llama_model_load_internal: ggml ctx size =    0.09 MB    llama_model_load_internal: mem required  = 8819.71 MB (+ 1608.00 MB per state)    llama_new_context_with_model: kv self size  = 1600.00 MB    ggml_metal_init: allocating    ggml_metal_init: using MPS    ggml_metal_init: loading '/Users/rlm/miniforge3/envs/llama/lib/python3.9/site-packages/llama_cpp/ggml-metal.metal'    ggml_metal_init: loaded kernel_add                            0x76add7460    ggml_metal_init: loaded kernel_mul                            0x76add5090


0x76add5090    ggml_metal_init: loaded kernel_mul_row                        0x76addae00    ggml_metal_init: loaded kernel_scale                          0x76adb2940    ggml_metal_init: loaded kernel_silu                           0x76adb8610    ggml_metal_init: loaded kernel_relu                           0x76addb700    ggml_metal_init: loaded kernel_gelu                           0x76addc100    ggml_metal_init: loaded kernel_soft_max                       0x76addcb80    ggml_metal_init: loaded kernel_diag_mask_inf                  0x76addd600    ggml_metal_init: loaded kernel_get_rows_f16                   0x295f16380    ggml_metal_init: loaded kernel_get_rows_q4_0


0x295f165e0    ggml_metal_init: loaded kernel_get_rows_q4_1                  0x295f16840    ggml_metal_init: loaded kernel_get_rows_q2_K                  0x295f16aa0    ggml_metal_init: loaded kernel_get_rows_q3_K                  0x295f16d00    ggml_metal_init: loaded kernel_get_rows_q4_K                  0x295f16f60    ggml_metal_init: loaded kernel_get_rows_q5_K                  0x295f171c0    ggml_metal_init: loaded kernel_get_rows_q6_K                  0x295f17420    ggml_metal_init: loaded kernel_rms_norm                       0x295f17680    ggml_metal_init: loaded kernel_norm                           0x295f178e0    ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x295f17b40


0x295f17b40    ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x295f17da0    ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x295f18000    ggml_metal_init: loaded kernel_mul_mat_q2_K_f32               0x7962b9900    ggml_metal_init: loaded kernel_mul_mat_q3_K_f32               0x7962bf5f0    ggml_metal_init: loaded kernel_mul_mat_q4_K_f32               0x7962bc630    ggml_metal_init: loaded kernel_mul_mat_q5_K_f32               0x142045960    ggml_metal_init: loaded kernel_mul_mat_q6_K_f32               0x7962ba2b0    ggml_metal_init: loaded kernel_rope                           0x7962c35f0    ggml_metal_init: loaded kernel_alibi_f32                      0x7962c30b0


0x7962c30b0    ggml_metal_init: loaded kernel_cpy_f32_f16                    0x7962c15b0    ggml_metal_init: loaded kernel_cpy_f32_f32                    0x7962beb10    ggml_metal_init: loaded kernel_cpy_f16_f16                    0x7962bf060    ggml_metal_init: recommendedMaxWorkingSetSize = 21845.34 MB    ggml_metal_init: hasUnifiedMemory             = true    ggml_metal_init: maxTransferRate              = built-in GPU    ggml_metal_add_buffer: allocated 'data            ' buffer, size =  6984.06 MB, (35852.94 / 21845.34), warning: current allocated size is greater than the recommended max working set size    ggml_metal_add_buffer: allocated 'eval            ' buffer, size =  1026.00 MB, (36878.94 / 21845.34), warning: current allocated size is greater than the recommended max working set size    ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  1602.00 MB, (38480.94 /


' buffer, size =  1602.00 MB, (38480.94 / 21845.34), warning: current allocated size is greater than the recommended max working set size    ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   298.00 MB, (38778.94 / 21845.34), warning: current allocated size is greater than the recommended max working set size    AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 |     ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   512.00 MB, (39290.94 / 21845.34), warning: current allocated size is greater than the recommended max working set sizeNote that these indicate that Metal was enabled properly:ggml_metal_init: allocatingggml_metal_init: using MPSprompt = """Question: A rap battle between Stephen Colbert and John Oliver"""llm(prompt)    Llama.generate: prefix-match hit        Setting: The Late Show with Stephen Colbert. The studio audience is filled with fans of both comedians, and the energy is electric. The two comedians are seated at a table, ready to begin their epic rap battle.        Stephen Colbert: (smirking) Oh, you think you can take me


Stephen Colbert: (smirking) Oh, you think you can take me down, John? You're just a Brit with a funny accent, and I'm the king of comedy!    John Oliver: (grinning) Oh, you think you're tough, Stephen? You're just a has-been from South Carolina, and I'm the future of comedy!    The battle begins, with each comedian delivering clever rhymes and witty insults. Here are a few lines that might be included:    Stephen Colbert: (rapping) You may have a big brain, John, but you can't touch my charm / I've got the audience in stitches, while you're just a blemish on the screen / Your accent is so thick, it's like trying to hear a speech through a mouthful of marshmallows / You may have        llama_print_timings:        load time =  2201.54 ms    llama_print_timings:      sample time =   182.54 ms /   256 runs   (    0.71 ms per token,  1402.41 tokens per second)    llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)    llama_print_timings:        eval time =  8484.62 ms /   256 runs   (   33.14 ms per token,    30.17 tokens per second)    llama_print_timings:       total time =  9000.62 ms


total time =  9000.62 ms    "\nSetting: The Late Show with Stephen Colbert. The studio audience is filled with fans of both comedians, and the energy is electric. The two comedians are seated at a table, ready to begin their epic rap battle.\n\nStephen Colbert: (smirking) Oh, you think you can take me down, John? You're just a Brit with a funny accent, and I'm the king of comedy!\nJohn Oliver: (grinning) Oh, you think you're tough, Stephen? You're just a has-been from South Carolina, and I'm the future of comedy!\nThe battle begins, with each comedian delivering clever rhymes and witty insults. Here are a few lines that might be included:\nStephen Colbert: (rapping) You may have a big brain, John, but you can't touch my charm / I've got the audience in stitches, while you're just a blemish on the screen / Your accent is so thick, it's like trying to hear a speech through a mouthful of marshmallows / You may have"GPT4Allâ€‹Similarly, we can use GPT4All.Download the GPT4All model binary.The Model Explorer on the GPT4All is a great way to choose and download a model.Then, specify the path that you downloaded to to.E.g., for me, the model lives here:/Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.binfrom langchain.llms import GPT4Allllm = GPT4All(    model="/Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.bin",    max_tokens=2048,)    Found model


max_tokens=2048,)    Found model file at  /Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.bin    objc[47842]: Class GGMLMetalClass is implemented in both /Users/rlm/anaconda3/envs/lcn2/lib/python3.9/site-packages/gpt4all/llmodel_DO_NOT_MODIFY/build/libreplit-mainline-metal.dylib (0x29f48c208) and /Users/rlm/anaconda3/envs/lcn2/lib/python3.9/site-packages/gpt4all/llmodel_DO_NOT_MODIFY/build/libllamamodel-mainline-metal.dylib (0x29f970208). One of the two will be used. Which one is undefined.    llama.cpp: using Metal    llama.cpp: loading model from /Users/rlm/Desktop/Code/gpt4all/models/nous-hermes-13b.ggmlv3.q4_0.bin    llama_model_load_internal: format     = ggjt v3 (latest)    llama_model_load_internal: n_vocab    = 32001    llama_model_load_internal: n_ctx      = 2048    llama_model_load_internal: n_embd     = 5120    llama_model_load_internal: n_mult     = 256    llama_model_load_internal: n_head     = 40    llama_model_load_internal: n_layer    = 40    llama_model_load_internal: n_rot      = 128    llama_model_load_internal: ftype      = 2 (mostly Q4_0)    llama_model_load_internal:


= 2 (mostly Q4_0)    llama_model_load_internal: n_ff       = 13824    llama_model_load_internal: n_parts    = 1    llama_model_load_internal: model size = 13B    llama_model_load_internal: ggml ctx size =    0.09 MB    llama_model_load_internal: mem required  = 9031.71 MB (+ 1608.00 MB per state)    llama_new_context_with_model: kv self size  = 1600.00 MB    ggml_metal_init: allocating    ggml_metal_init: using MPS    ggml_metal_init: loading '/Users/rlm/anaconda3/envs/lcn2/lib/python3.9/site-packages/gpt4all/llmodel_DO_NOT_MODIFY/build/ggml-metal.metal'    ggml_metal_init: loaded kernel_add                            0x115fcbfb0    ggml_metal_init: loaded kernel_mul                            0x115fcd4a0    ggml_metal_init: loaded kernel_mul_row                        0x115fce850    ggml_metal_init: loaded kernel_scale                          0x115fcd700    ggml_metal_init: loaded kernel_silu


ggml_metal_init: loaded kernel_silu                           0x115fcd960    ggml_metal_init: loaded kernel_relu                           0x115fcfd50    ggml_metal_init: loaded kernel_gelu                           0x115fd03c0    ggml_metal_init: loaded kernel_soft_max                       0x115fcf640    ggml_metal_init: loaded kernel_diag_mask_inf                  0x115fd07f0    ggml_metal_init: loaded kernel_get_rows_f16                   0x1147b2450    ggml_metal_init: loaded kernel_get_rows_q4_0                  0x11479d1d0    ggml_metal_init: loaded kernel_get_rows_q4_1                  0x1147ad1f0    ggml_metal_init: loaded kernel_get_rows_q2_k                  0x1147aef50    ggml_metal_init: loaded


0x1147aef50    ggml_metal_init: loaded kernel_get_rows_q3_k                  0x1147af1b0    ggml_metal_init: loaded kernel_get_rows_q4_k                  0x1147af410    ggml_metal_init: loaded kernel_get_rows_q5_k                  0x1147affa0    ggml_metal_init: loaded kernel_get_rows_q6_k                  0x1147b0200    ggml_metal_init: loaded kernel_rms_norm                       0x1147b0460    ggml_metal_init: loaded kernel_norm                           0x1147bfc90    ggml_metal_init: loaded kernel_mul_mat_f16_f32                0x1147c0230    ggml_metal_init: loaded kernel_mul_mat_q4_0_f32               0x1147c0490    ggml_metal_init: loaded kernel_mul_mat_q4_1_f32               0x1147c06f0    ggml_metal_init: loaded


0x1147c06f0    ggml_metal_init: loaded kernel_mul_mat_q2_k_f32               0x1147c0950    ggml_metal_init: loaded kernel_mul_mat_q3_k_f32               0x1147c0bb0    ggml_metal_init: loaded kernel_mul_mat_q4_k_f32               0x1147c0e10    ggml_metal_init: loaded kernel_mul_mat_q5_k_f32               0x1147c1070    ggml_metal_init: loaded kernel_mul_mat_q6_k_f32               0x1147c13d0    ggml_metal_init: loaded kernel_rope                           0x1147c1a00    ggml_metal_init: loaded kernel_alibi_f32                      0x1147c2120    ggml_metal_init: loaded kernel_cpy_f32_f16                    0x115fd1690    ggml_metal_init: loaded kernel_cpy_f32_f32                    0x115fd1c60


0x115fd1c60    ggml_metal_init: loaded kernel_cpy_f16_f16                    0x115fd2d40    ggml_metal_init: recommendedMaxWorkingSetSize = 21845.34 MB    ggml_metal_init: hasUnifiedMemory             = true    ggml_metal_init: maxTransferRate              = built-in GPU    ggml_metal_add_buffer: allocated 'data            ' buffer, size =  6984.06 MB, ( 6984.45 / 21845.34)    ggml_metal_add_buffer: allocated 'eval            ' buffer, size =  1024.00 MB, ( 8008.45 / 21845.34)    ggml_metal_add_buffer: allocated 'kv              ' buffer, size =  1602.00 MB, ( 9610.45 / 21845.34)    ggml_metal_add_buffer: allocated 'scr0            ' buffer, size =   512.00 MB, (10122.45 / 21845.34)    ggml_metal_add_buffer: allocated 'scr1            ' buffer, size =   512.00 MB, (10634.45 / 21845.34)LLMChainâ€‹Run an


/ 21845.34)LLMChainâ€‹Run an LLMChain (see here) with either model by passing in the retrieved docs and a simple prompt.It formats the prompt template using the input key values provided and passes the formatted string to GPT4All, LLama-V2, or another specified LLM.In this case, the list of retrieved documents (docs) above are pass into {context}.from langchain import PromptTemplate, LLMChain# Promptprompt = PromptTemplate.from_template(    "Summarize the main themes in these retrieved docs: {docs}")# Chainllm_chain = LLMChain(llm=llm, prompt=prompt)# Runquestion = "What are the approaches to Task Decomposition?"docs = vectorstore.similarity_search(question)result = llm_chain(docs)# Outputresult["text"]    Llama.generate: prefix-match hit        Based on the retrieved documents, the main themes are:    1. Task decomposition: The ability to break down complex tasks into smaller subtasks, which can be handled by an LLM or other components of the agent system.    2. LLM as the core controller: The use of a large language model (LLM) as the primary controller of an autonomous agent system, complemented by other key components such as a knowledge graph and a planner.    3. Potentiality of LLM: The idea that LLMs have the potential to be used as powerful general problem solvers, not just for generating well-written copies but also for solving complex tasks and achieving human-like intelligence.    4. Challenges in long-term planning: The challenges in planning over a lengthy history and effectively exploring the solution space, which are important limitations of current LLM-based autonomous agent systems.        llama_print_timings:


llama_print_timings:        load time =  1191.88 ms    llama_print_timings:      sample time =   134.47 ms /   193 runs   (    0.70 ms per token,  1435.25 tokens per second)    llama_print_timings: prompt eval time = 39470.18 ms /  1055 tokens (   37.41 ms per token,    26.73 tokens per second)    llama_print_timings:        eval time =  8090.85 ms /   192 runs   (   42.14 ms per token,    23.73 tokens per second)    llama_print_timings:       total time = 47943.12 ms    '\nBased on the retrieved documents, the main themes are:\n1. Task decomposition: The ability to break down complex tasks into smaller subtasks, which can be handled by an LLM or other components of the agent system.\n2. LLM as the core controller: The use of a large language model (LLM) as the primary controller of an autonomous agent system, complemented by other key components such as a knowledge graph and a planner.\n3. Potentiality of LLM: The idea that LLMs have the potential to be used as powerful general problem solvers, not just for generating well-written copies but also for solving complex tasks and achieving human-like intelligence.\n4. Challenges in long-term planning: The challenges in planning over a lengthy history and effectively exploring the solution space, which are important limitations of current LLM-based autonomous agent systems.'QA Chainâ€‹We can use a QA chain to handle


autonomous agent systems.'QA Chainâ€‹We can use a QA chain to handle our question above.chain_type="stuff" (see here) means that all the docs will be added (stuffed) into a prompt.from langchain.chains.question_answering import load_qa_chain# Prompttemplate = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum and keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. {context}Question: {question}Helpful Answer:"""QA_CHAIN_PROMPT = PromptTemplate(    input_variables=["context", "question"],    template=template,)# Chainchain = load_qa_chain(llm, chain_type="stuff", prompt=QA_CHAIN_PROMPT)# Runchain({"input_documents": docs, "question": question}, return_only_outputs=True)    Llama.generate: prefix-match hit     Hi there! There are three main approaches to task decomposition. One is using LLM with simple prompting like "Steps for XYZ." or "What are the subgoals for achieving XYZ?" Another approach is by using task-specific instructions, such as "Write a story outline" for writing a novel. Finally, task decomposition can also be done with human inputs. Thanks for asking!        llama_print_timings:        load time =  1191.88 ms    llama_print_timings:      sample time =    61.21 ms /    85 runs   (    0.72 ms per token,  1388.64 tokens per second)    llama_print_timings: prompt eval time =


tokens per second)    llama_print_timings: prompt eval time =  8014.11 ms /   267 tokens (   30.02 ms per token,    33.32 tokens per second)    llama_print_timings:        eval time =  2908.17 ms /    84 runs   (   34.62 ms per token,    28.88 tokens per second)    llama_print_timings:       total time = 11096.23 ms    {'output_text': ' Hi there! There are three main approaches to task decomposition. One is using LLM with simple prompting like "Steps for XYZ." or "What are the subgoals for achieving XYZ?" Another approach is by using task-specific instructions, such as "Write a story outline" for writing a novel. Finally, task decomposition can also be done with human inputs. Thanks for asking!'}RetrievalQAâ€‹For an even simpler flow, use RetrievalQA.This will use a QA default prompt (shown here) and will retrieve from the vectorDB.But, you can still pass in a prompt, as before, if desired.from langchain.chains import RetrievalQAqa_chain = RetrievalQA.from_chain_type(    llm,    retriever=vectorstore.as_retriever(),    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT},)qa_chain({"query": question})    Llama.generate: prefix-match hit         The three approaches to Task decomposition are LLMs with simple prompting, task-specific instructions, or human inputs. Thanks for asking!        llama_print_timings:        load time =  1191.88 ms


load time =  1191.88 ms    llama_print_timings:      sample time =    22.78 ms /    31 runs   (    0.73 ms per token,  1360.66 tokens per second)    llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)    llama_print_timings:        eval time =  1320.23 ms /    31 runs   (   42.59 ms per token,    23.48 tokens per second)    llama_print_timings:       total time =  1387.70 ms    {'query': 'What are the approaches to Task Decomposition?',     'result': ' \nThe three approaches to Task decomposition are LLMs with simple prompting, task-specific instructions, or human inputs. Thanks for asking!'}PreviousContext aware text splitting and QA / ChatNextQuestion answering over a group chat messages using Activeloop's DeepLakeDocument LoadingModelLlama-v2GPT4AllLLMChainQA ChainRetrievalQACommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Page Not Found | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKPage Not FoundWe could not find what you were looking for.Please contact the owner of the site that linked you to the original URL and let them know their link is broken.CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Chatbots | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsAnalyzing structured dataExtractionInteracting with APIsChatbotsVoice AssistantSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesChatbotsChatbotsSince language models are good at producing text, that makes them ideal for creating chatbots.
Aside from the base prompts/LLMs, an important concept to know for Chatbots is memory.
Most chat based applications rely on remembering what happened in previous interactions, which memory is designed to help with.The following resources exist:ChatGPT Clone: A notebook walking through how to recreate a ChatGPT-like experience with LangChain.Conversation Agent: A notebook walking through how to create an agent optimized for conversation.Additional related resources include:Memory concepts and examples: Explanation of key concepts related to memory along with how-to's and examples.More end-to-end examples include:Voice Assistant: A notebook walking through how to create a voice assistant using LangChain.PreviousInteracting with APIsNextVoice AssistantCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Voice Assistant | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKUse casesQA and Chat over DocumentsAnalyzing structured dataExtractionInteracting with APIsChatbotsVoice AssistantSummarizationCode UnderstandingAgent simulationsAgentsAutonomous (long-running) agentsMulti-modalUse casesChatbotsVoice AssistantVoice AssistantThis chain creates a clone of ChatGPT with a few modifications to make it a voice assistant.


It uses the pyttsx3 and speech_recognition libraries to convert text to speech and speech to text respectively. The prompt template is also changed to make it more suitable for voice assistant use.from langchain import OpenAI, LLMChain, PromptTemplatefrom langchain.memory import ConversationBufferWindowMemorytemplate = """Assistant is a large language model trained by OpenAI.Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.{history}Human: {human_input}Assistant:"""prompt = PromptTemplate(input_variables=["history", "human_input"], template=template)chatgpt_chain = LLMChain(


"human_input"], template=template)chatgpt_chain = LLMChain(    llm=OpenAI(temperature=0),    prompt=prompt,    verbose=True,    memory=ConversationBufferWindowMemory(k=2),)import speech_recognition as srimport pyttsx3engine = pyttsx3.init()def listen():    r = sr.Recognizer()    with sr.Microphone() as source:        print("Calibrating...")        r.adjust_for_ambient_noise(source, duration=5)        # optional parameters to adjust microphone sensitivity        # r.energy_threshold = 200        # r.pause_threshold=0.5        print("Okay, go!")        while 1:            text = ""            print("listening now...")            try:                audio = r.listen(source, timeout=5, phrase_time_limit=30)                print("Recognizing...")                # whisper model options are found here: https://github.com/openai/whisper#available-models-and-languages                # other speech recognition models are also available.                text = r.recognize_whisper(


audio,                    model="medium.en",                    show_dict=True,                )["text"]            except Exception as e:                unrecognized_speech_text = (                    f"Sorry, I didn't catch that. Exception was: {e}s"                )                text = unrecognized_speech_text            print(text)            response_text = chatgpt_chain.predict(human_input=text)            print(response_text)            engine.say(response_text)            engine.runAndWait()listen(None)    Calibrating...    Okay, go!    listening now...    Recognizing...    C:\Users\jaden\AppData\Roaming\Python\Python310\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html      from .autonotebook import tqdm as notebook_tqdm     Hello, Assistant.


.autonotebook import tqdm as notebook_tqdm     Hello, Assistant. What's going on?            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.            Human:  Hello, Assistant. What's going on?


Human:  Hello, Assistant. What's going on?    Assistant:        > Finished chain.     Hi there! It's great to hear from you. I'm doing well. How can I help you today?    listening now...    Recognizing...     That's cool. Isn't that neat? Yeah, I'm doing great.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account


from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Hello, Assistant. What's going on?    AI:  Hi there! It's great to hear from you. I'm doing well. How can I help you today?    Human:  That's cool. Isn't that neat? Yeah, I'm doing great.    Assistant:        > Finished chain.      That's great to hear! What can I do for you today?    listening now...    Recognizing...     Thank you.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.


in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Hello, Assistant. What's going on?    AI:  Hi there! It's great to hear from you. I'm doing well. How can I help you today?    Human:  That's cool. Isn't that neat? Yeah, I'm doing great.    AI:   That's great to hear! What can I do for you today?    Human:  Thank you.    Assistant:        > Finished chain.     You're welcome! Is there anything else I can help you with?    listening now...    Recognizing...     I'd like to learn more about neural networks.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant


to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  That's cool. Isn't that neat? Yeah, I'm doing great.    AI:   That's great to hear! What can I do for you today?    Human:  Thank you.    AI:  You're welcome! Is there anything else I can help you with?    Human:  I'd like to learn more about neural networks.    Assistant:        > Finished chain.     Sure! Neural networks are a


> Finished chain.     Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.    listening now...    Recognizing...     Tell me a fun fact about neural networks.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription.


human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Thank you.    AI:  You're welcome! Is there anything else I can help you with?    Human:  I'd like to learn more about neural networks.    AI:  Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.    Human:  Tell me a fun fact about neural networks.    Assistant:        > Finished chain.     Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.    listening now...    Recognizing...     Tell me about a brand new discovered bird species.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding


to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  I'd like to learn more about neural networks.    AI:  Sure! Neural networks are a type of artificial intelligence that use a network of interconnected nodes to process data and make decisions. They are used in a variety of applications, from image recognition to natural language processing. Neural networks are often used to solve complex problems that are too difficult for traditional algorithms.    Human:  Tell me a fun fact about neural networks.    AI:  Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can


of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.    Human:  Tell me about a brand new discovered bird species.    Assistant:        > Finished chain.     A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.    listening now...    Recognizing...     Tell me a children's story about the importance of honesty and trust.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable


Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Tell me a fun fact about neural networks.    AI:  Neural networks are inspired by the way the human brain works. They are composed of interconnected nodes that process data and make decisions, just like neurons in the brain. Neural networks can learn from their mistakes and improve their performance over time, just like humans do.    Human:  Tell me about a brand new discovered bird species.    AI:  A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.    Human:  Tell me a children's story about the importance of honesty and trust.    Assistant:        > Finished chain.     Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and


the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.    listening now...    Recognizing...     Wow, Assistant, that was a really good story. Congratulations!            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.


specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Tell me about a brand new discovered bird species.    AI:  A new species of bird was recently discovered in the Amazon rainforest. The species, called the Spix's Macaw, is a small, blue parrot that is believed to be extinct in the wild. It is the first new species of bird to be discovered in the Amazon in over 100 years.    Human:  Tell me a children's story about the importance of honesty and trust.    AI:  Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.    Human:  Wow, Assistant, that was a really good story. Congratulations!    Assistant:        > Finished chain.     Thank you! I'm


> Finished chain.     Thank you! I'm glad you enjoyed it.    listening now...    Recognizing...     Thank you.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.


the audio channel since it takes time to listen to a response.        Human:  Tell me a children's story about the importance of honesty and trust.    AI:  Once upon a time, there was a young boy named Jack who lived in a small village. Jack was always honest and trustworthy, and his friends and family knew they could always count on him. One day, Jack was walking through the forest when he stumbled upon a magical tree. The tree told Jack that if he was honest and trustworthy, he would be rewarded with a special gift. Jack was so excited, and he promised to always be honest and trustworthy. Sure enough, the tree rewarded Jack with a beautiful golden apple. From that day forward, Jack was always honest and trustworthy, and he was rewarded with many more magical gifts. The moral of the story is that honesty and trust are the most important things in life.    Human:  Wow, Assistant, that was a really good story. Congratulations!    AI:  Thank you! I'm glad you enjoyed it.    Human:  Thank you.    Assistant:        > Finished chain.     You're welcome!    listening now...    Recognizing...     Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of


range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Wow, Assistant, that was a really good story. Congratulations!    AI:  Thank you! I'm glad you enjoyed it.    Human:  Thank you.    AI:  You're welcome!    Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of


know of online brands like Photoshop and Freq that you don't have to download in some sort of way?    Assistant:        > Finished chain.     Yes, there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.    listening now...    Recognizing...     Our whole process of awesome is free.            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here


help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Thank you.    AI:  You're welcome!    Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?    AI:  Yes, there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.    Human:  Our whole process of awesome is free.    Assistant:        > Finished chain.     That's great! It's always nice to have access to free tools and resources.    listening now...    Recognizing...     No, I meant to ask, are those options that you mentioned free? No, I meant to ask, are those options that you mentioned free?            > Entering new LLMChain chain...    Prompt after formatting:    Assistant is a large language model trained by OpenAI.        Assistant is designed to be able to


model trained by OpenAI.        Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.        Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.        Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.        Assistant is aware that human input is being transcribed from audio and as such there may be some errors in the transcription. It will attempt to account for some words being swapped with similar-sounding words or phrases. Assistant will also keep responses concise, because human attention spans are more limited over the audio channel since it takes time to listen to a response.        Human:  Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way? Do you know of online brands like Photoshop and Freq that you don't have to download in some sort of way?    AI:  Yes, there are several online brands that offer photo editing and other creative tools without the need to download any


there are several online brands that offer photo editing and other creative tools without the need to download any software. Adobe Photoshop Express, Pixlr, and Fotor are some of the most popular online photo editing tools. Freq is an online music production platform that allows users to create and share music without downloading any software.    Human:  Our whole process of awesome is free.    AI:  That's great! It's always nice to have access to free tools and resources.    Human:  No, I meant to ask, are those options that you mentioned free? No, I meant to ask, are those options that you mentioned free?    Assistant:        > Finished chain.     Yes, the online brands I mentioned are all free to use. Adobe Photoshop Express, Pixlr, and Fotor are all free to use, and Freq is a free music production platform.    listening now...    ---------------------------------------------------------------------------    KeyboardInterrupt                         Traceback (most recent call last)    Cell In[6], line 1    ----> 1 listen(None)    Cell In[5], line 20, in listen(command_queue)         18 print('listening now...')         19 try:    ---> 20     audio = r.listen(source, timeout=5, phrase_time_limit=30)         21     # audio = r.record(source,duration = 5)         22     print('Recognizing...')    File


22     print('Recognizing...')    File c:\ProgramData\miniconda3\envs\lang\lib\site-packages\speech_recognition\__init__.py:523, in Recognizer.listen(self, source, timeout, phrase_time_limit, snowboy_configuration)        520 if phrase_time_limit and elapsed_time - phrase_start_time > phrase_time_limit:        521     break    --> 523 buffer = source.stream.read(source.CHUNK)        524 if len(buffer) == 0: break  # reached end of the stream        525 frames.append(buffer)    File c:\ProgramData\miniconda3\envs\lang\lib\site-packages\speech_recognition\__init__.py:199, in Microphone.MicrophoneStream.read(self, size)        198 def read(self, size):    --> 199     return self.pyaudio_stream.read(size, exception_on_overflow=False)    File c:\ProgramData\miniconda3\envs\lang\lib\site-packages\pyaudio\__init__.py:570, in PyAudio.Stream.read(self, num_frames, exception_on_overflow)        567 if not self._is_input:        568     raise IOError("Not input stream",        569                   paCanNotReadFromAnOutputOnlyStream)    --> 570 return pa.read_stream(self._stream, num_frames,        571


num_frames,        571                       exception_on_overflow)    KeyboardInterrupt: PreviousChatbotsNextSummarizationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Code Understanding | ğŸ¦œï¸�ğŸ”— Langchain