| --- |
| license: apache-2.0 |
| datasets: |
| - Sentdex/wsb_reddit_v002 |
| --- |
| |
| # Model Card for WSB-GPT-7B |
|
|
| This is a Llama 2 7B Chat model fine-tuned with QLoRA on 2017-2018ish /r/wallstreetbets subreddit comments and responses, with the hopes of learning more about QLoRA and creating models with a little more character. |
|
|
|
|
| ### Model Description |
|
|
| - **Developed by:** Sentdex |
| - **Shared by:** Sentdex |
| - **GPU Compute provided by:** [Lambda Labs](https://lambdalabs.com/service/gpu-cloud) |
|
|
| - **Model type:** Instruct/Chat |
| - **Language(s) (NLP):** Multilingual from Llama 2, but not sure what the fine-tune did to it, or if the fine-tuned behavior translates well to other languages. Let me know! |
| - **License:** Apache 2.0 |
| - **Finetuned from Llama 2 7B Chat** |
|
|
|
|
| - **Demo [optional]:** [More Information Needed] |
|
|
| ## Uses |
|
|
| This model's primary purpose is to be a fun chatbot and to learn more about QLoRA. It is not intended to be used for any other purpose and some people may find it abrasive/offensive. |
|
|
| ## Bias, Risks, and Limitations |
|
|
| This model is prone to using at least 3 words that were popularly used in the WSB subreddit in that era that are much more frowned-upon. As time goes on, I may wind up pruning or find-replacing these words in the training data, or leaving it. |
|
|
| Just be advised this model can be offensive and is not intended for all audiences! |
|
|
| ## How to Get Started with the Model |
| ### Prompt Format: |
| |
| ``` |
| ### Comment: |
| [parent comment text] |
| |
| ### REPLY: |
| [bot's reply] |
|
|
| ### END. |
| ``` |
| |
| Use the code below to get started with the model. |
| |
| ```py |
| from transformers import pipeline |
|
|
| # Initialize the pipeline for text generation using the Sentdex/WSB-GPT-7B model |
| pipe = pipeline("text-generation", model="Sentdex/WSB-GPT-7B") |
|
|
| # Define your prompt |
| prompt = """### Comment: |
| How does the stock market actually work? |
|
|
| ### REPLY: |
| """ |
|
|
| # Generate text based on the prompt |
| generated_text = pipe(prompt, max_length=128, num_return_sequences=1) |
|
|
| # Extract and print the generated text |
| print(generated_text[0]['generated_text'].split("### END.")[0]) |
| ``` |
| |
| Example continued generation from above: |
| |
| ``` |
| ### Comment: |
| How does the stock market actually work? |
| |
| ### REPLY: |
| You sell when you are up and buy when you are down. |
| ``` |
| |
| Despite `</s>` being the typical Llama stop token, I was never able to get this token to be generated in training/testing so the model would just never stop generating. I wound up testing with ### END. and that worked, but obviously isn't ideal. Will fix this in the future maybe(tm). |
| |
| #### Hardware |
| |
| This QLoRA was trained on a Lambda Labs 1x H100 80GB GPU instance. |
| |
| ## Citation |
| |
| - Llama 2 (Meta AI) for the base model. |
| - Farouk E / Far El: https://twitter.com/far__el for helping with all my silly questions about QLoRA |
| - Lambda Labs for the compute. The model itself only took a few hours to train, but it took me days to learn how to tie everything together. |
| - Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer for QLoRA + implementation on github: https://github.com/artidoro/qlora/ |
| - @eugene-yh and @jinyongyoo on Github + @ChrisHayduk for the QLoRA merge: https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930 |
| |
| |
| ## Model Card Contact |
| |
| harrison@pythonprogramming.net |
