combining falcon 40b instruct with langchain
im want to build an llm where i can make it interact with my own data using langchain.
from langchain_community.llms import HuggingFaceHub
llm = HuggingFaceHub(
repo_id=model_name,
task="text-generation",
model_kwargs={
"max_new_tokens": 512,
"top_k": 30,
"temperature": 0.1,
"repetition_penalty": 1.03
},
huggingfacehub_api_token="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"
)
i reached the following stage yet when i run the following i get this error
from langchain_community.chat_models.huggingface import ChatHuggingFace
llm = ChatHuggingFace(llm=llm)
HfHubHTTPError: 401 Client Error: Unauthorized for url
and i am doing do this to be able to run the following:
qa_chain = RetrievalQA.from_chain_type(
llm=llm,
retriever=vector_db.as_retriever()
)
what am i missing and is there a way to be able to do this fully local like doing the falcon model and pass it to ChatHuggingFace ?
thank you!