Use this repo for deploying HF spaces (#68)

We used to maintain a separate repo for buster's HF space. Now we use this repo to deploy directly.

* update prompt engineering

* add HF spaces metadata

* update to latest python

* add data to the repo with git LFS

.gitignore

@@ -1,11 +1,11 @@
-# Project specific stuff
-buster/data/
-
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]
 *$py.class
 
+# Macos
+*.DS_Store*
+
 albenchmark/data/
 
 # Ignore notebooks by default
@@ -137,4 +137,4 @@ dmypy.json
 .pyre/
 
 # VSCode
-.vscode/
+.vscode/

README.md

@@ -1,3 +1,14 @@
+---
+title: Buster
+emoji: 🤖
+colorFrom: red
+colorTo: blue
+sdk: gradio
+app_file: buster/apps/gradio_app.py
+python_version: 3.10.8
+pinned: false
+---
+
 # Buster, the QA documentation chatbot!
 
 Buster is a question-answering chatbot that can be tuned to specific documentations. You can try it [here](https://huggingface.co/spaces/jerpint/buster), where it will answer questions about [🤗 Transformers](https://huggingface.co/docs/transformers/index).
@@ -23,7 +34,7 @@ We send the prompt to the [OpenAI API](https://beta.openai.com/docs/api-referenc
 ### Currently used models
 
 - For embeddings: "text-embedding-ada-002"
-- For completion: "text-davinci-003"
+- For completion: We support both "text-davinci-003" and "gpt-3.5-turbo"
 
 ### Livestream
 

buster/apps/gradio_app.py

@@ -1,9 +1,14 @@
+import os
+import pathlib
+
 import gradio as gr
 
 from buster.buster import Buster, BusterConfig
 
+DATA_DIR = pathlib.Path(__file__).parent.parent.resolve() / "data"  # points to ../data/
+
 buster_cfg = BusterConfig(
-    documents_file="../data/document_embeddings_huggingface.tar.gz",
+    documents_file=os.path.join(DATA_DIR, "document_embeddings_huggingface.tar.gz"),
     unknown_prompt="I'm sorry, but I am an AI language model trained to assist with questions related to the huggingface transformers library. I cannot answer that question as it is not relevant to the library or its usage. Is there anything else I can assist you with?",
     embedding_model="text-embedding-ada-002",
     top_k=3,
@@ -11,18 +16,28 @@ buster_cfg = BusterConfig(
     max_words=3000,
     completer_cfg={
         "name": "ChatGPT",
+        "text_before_documents": (
+            "You are a chatbot assistant answering technical questions about huggingface transformers, a library to train transformers in python. "
+            "You can only respond to a question if the content necessary to answer the question is contained in the following provided documentation. "
+            "If it isn't, simply reply that you cannot answer the question. "
+            "Here is the documentation: "
+            "<BEGIN_DOCUMENTATION> "
+        ),
         "text_before_prompt": (
-            """You are a slack chatbot assistant answering technical questions about huggingface transformers, a library to train transformers in python. """
-            """Make sure to format your answers in Markdown format, including code block and snippets. """
-            """Do not include any links to urls or hyperlinks in your answers. """
-            """If you do not know the answer to a question, or if it is completely irrelevant to the library usage, let the user know you cannot answer with this response:\n"""
-            """'I'm sorry, but I am an AI language model trained to assist with questions related to the huggingface transformers library. I cannot answer that question as it is not relevant to the library or its usage. Is there anything else I can assist you with?'"""
-            """For example:\n"""
-            """What is the meaning of life for huggingface?\n"""
-            """I'm sorry, but I am an AI language model trained to assist with questions related to the huggingface transformers library. I cannot answer that question as it is not relevant to the library or its usage. Is there anything else I can assist you with?"""
-            """Now answer the following question:\n"""
+            "<\END_DOCUMENTATION>\n"
+            "REMINDER:\n"
+            "You are a chatbot assistant answering technical questions about huggingface transformers, a library to train transformers in python. "
+            "Here are the rules you must follow:\n"
+            "1) You must only respond with information contained in the documentation above. Say you do not know if the information is not provided.\n"
+            "2) Make sure to format your answers in Markdown format, including code block and snippets.\n"
+            "3) Do not include any links to urls or hyperlinks in your answers.\n"
+            "4) If you do not know the answer to a question, or if it is completely irrelevant to the library usage, simply reply with:\n"
+            "'I'm sorry, but I am an AI language model trained to assist with questions related to the huggingface transformers library. I cannot answer that question as it is not relevant to the library or its usage. Is there anything else I can assist you with?'"
+            "For example:\n"
+            "What is the meaning of life for huggingface?\n"
+            "I'm sorry, but I am an AI language model trained to assist with questions related to the huggingface transformers library. I cannot answer that question as it is not relevant to the library or its usage. Is there anything else I can assist you with?"
+            "Now answer the following question:\n"
         ),
-        "text_before_documents": "Only use these documents as reference:\n",
         "completion_kwargs": {
             "model": "gpt-3.5-turbo",
         },

buster/data/document_embeddings_huggingface.tar.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19453cb9ec85e644306af7dcc6fcad79cbb842d15a2087a66ddf48b5cbd9fbc9
+size 46918939