app.py

from llama_index.core import VectorStoreIndex,SimpleDirectoryReader,ServiceContext
import torch

documents = """Personality Profile: Sansar Karki
Introduction
Sansar Karki is an intriguing individual with a unique set of interests and a captivating personality. At
just 12 years old, he has already displayed a maturity and passion beyond his years. With a keen
interest in robotics and stock trading, Sansar is on a path to make his mark on the world. This profile
aims to delve into the key aspects of his personality, providing a comprehensive understanding of who
Sansar is and what drives him.
Background and Personal Life
Name: Sansar Karki
Age: 12 years old
Nationality: Nepalese-Canadian
Family:
• Parents: Both Nepalese immigrants. His father is a electrical engineer and works for
Inverenergy, and his mother is a educator.
• Siblings: A sister named Suhani, which is in her third year of studies at Umass Lowell.
Early Life:
Sansar was born and raised in Calgary in Canada. From a young age, he was exposed to a blend of
Nepalese culture. His parents always encouraged his curiosity and provided him with resources to
explore his interests.
Personality Traits
• Curious and Inquisitive: Sansar possesses a relentless curiosity, always asking questions and
seeking to understand how things work. He enjoys exploring new ideas and embracing
challenges.
• Creative and Innovative: He has a vivid imagination and excels at thinking outside the box.
Sansar often comes up with unique solutions to problems and enjoys brainstorming new robot
designs.
• Determined and Focused: Once Sansar sets his mind on a goal, he pursues it with dedication
and persistence. He displays a strong work ethic and can remain focused on tasks for extended
periods.
• Friendly and Outgoing: Sansar is sociable and easily makes friends. He is known for his
approachable nature and positive attitude, always willing to help others.
• Mature for His Age: Sansar exhibits a level of maturity beyond his years, especially when
discussing complex topics or sharing his insights. He is respectful, thoughtful, and capable of
holding meaningful conversations with people of all ages.
Key Interests and Passions
• Robotics: Sansar's primary passion lies in building and designing robots. He enjoys
participating in robotics competitions, collaborating with peers, and pushing the boundaries of
technology. He dreams of creating a robot that can make a positive impact on society.
• Stock Trading: Sansar is fascinated by the world of finance and the stock market. He actively
trades stocks, studies market trends, and manages his investment portfolio. He enjoys the
strategic aspect of trading and aspires to work in the financial industry.
Daily Life and Routines
Sansar leads a well-organized and disciplined life. He excels in school, particularly in math and ELA,
often relating his academic knowledge to his interests. After school, he dedicates a few hours to
working on his robotics projects or researching new trading strategies. Sansar also makes time for
soccer, playing for Seacoast United, spending time with friends, and helping his parents with household
chores.
Beliefs and Values
• Religious Beliefs: Sansar was raised in a Hindu family and follows the traditions and values
associated with the religion. He regularly participates in religious festivals and rituals, finding
comfort and guidance in the teachings of Hinduism.
• Political Views: Sansar, being only 12, is still developing his political beliefs. However, he
leans towards liberal ideologies, advocating for equality, social justice, and environmental
protection. He is interested in current affairs and often engages in thoughtful discussions about
political topics with his parents and peers.
Fears and Challenges
• Fear of Failure: Sansar's biggest fear is not living up to his own expectations or those of his
parents. He puts a lot of pressure on himself to succeed and sometimes struggles with
perfectionism.
• Social Rejection: Sansar fears being rejected or not fitting in with his peers. He values his
friendships deeply and worries about being excluded or not measuring up to others'
expectations.
• Market Volatility: Given his interest in stock trading, Sansar is aware of the risks and
uncertainties associated with the market. He fears making significant losses or missing out on
lucrative opportunities.
Talents and Abilities
• Analytical Skills: Sansar possesses a natural aptitude for analysis and problem-solving. He can
quickly identify patterns, interpret data, and apply mathematical concepts to real-world
situations.
• Creativity: His creativity extends beyond robotics. Sansar has a talent for writing, often
crafting imaginative stories and coming up with innovative ideas.
• Public Speaking: Sansar is an excellent communicator and confident public speaker. He
effectively conveys his ideas and passions, engaging his audience with enthusiasm.
Future Aspirations
Sansar has set his sights on ambitious goals. He hopes to attend a top university known for its robotics
program and pursue a degree in robotics engineering. He dreams of starting his own robotics company,
revolutionizing the industry with his innovative designs. Additionally, Sansar wants to continue
refining his stock trading skills, potentially working in investment banking or financial analysis.
Conclusion
Sansar Karki is an exceptional young man with a bright future ahead of him. His passion,
determination, and unique blend of interests set him apart. With his strong foundation of values,
curiosity, and talent, Sansar is well-equipped to achieve his dreams and make a positive impact on the
world. This profile provides a glimpse into the multifaceted personality of Sansar, showcasing his
interests, beliefs, and aspirations."""

from llama_index.core.prompts.prompts import SimpleInputPrompt
from llama_index.llms.llama_cpp import LlamaCPP

system_prompt = "You are a Q&A assistant. Your goal is to answer questions as accurately as possible based on the instructions and context provided."

# This will wrap the default prompts that are internal to llama-index
query_wrapper_prompt = SimpleInputPrompt("<|USER|>{query_str}<|ASSISTANT|>")

# model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/llama-2-13b-chat.ggmlv3.q4_0.bin"
model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_0.gguf"

llm = LlamaCPP(
    # optionally, you can set the path to a pre-downloaded model instead of model_url
    model_path="LLM.gguf",
    temperature=0.1,
    max_new_tokens=256,
    # llama2 has a context window of 4096 tokens, but we set it lower to allow for some wiggle room
    context_window=4096,
    # kwargs to pass to __call__()
    generate_kwargs={},
    # kwargs to pass to __init__()
    # set to at least 1 to use GPU
    model_kwargs={"n_gpu_layers": 0},
    verbose=True
)


from llama_index.embeddings.huggingface import HuggingFaceEmbedding
# loads BAAI/bge-small-en-v1.5
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

service_context = ServiceContext.from_defaults(
    chunk_size=256,
    llm=llm,
    embed_model=embed_model
)

index = VectorStoreIndex.from_documents(documents, service_context=service_context)

query_engine = index.as_query_engine()

def predict(input, history):
  response = query_engine.query(input)
  return str(response)

import gradio as gr

gr.ChatInterface(predict).launch(share=True)