| LoRA which is fine-tuned for "command" following. | |
| ### How to use? | |
| import torch | |
| from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig | |
| model_id = "EleutherAI/pythia-2.8b-deduped" | |
| bnb_config = BitsAndBytesConfig( | |
| load_in_4bit=True, | |
| bnb_4bit_use_double_quant=True, | |
| bnb_4bit_quant_type="nf4", | |
| bnb_4bit_compute_dtype=torch.bfloat16 | |
| ) | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0}) | |
| # load adapter | |
| if True: | |
| from peft import PeftModel | |
| model = PeftModel.from_pretrained( | |
| model, "Q4234/commandA-LoRA-2.8b", | |
| torch_dtype=torch.float16 | |
| ) | |
| ### Prompting | |
| Assistant can use any tools he likes by calling the tool as python functions. | |
| ex: | |
| var=google('what is life?') - to search for something using google and write the result as a string into variable var | |
| result0=calc('5 + 6') - to compute something using a calculator and put the result into the variable result0 | |
| ubuntuShell('ls -lah') - to execute command 'ls -lah' in the shell under linux ubuntu | |
| appendToFile(<filename>, <content>) - to append(write) string <content> to file <filename> | |
| str5 = """AB c 7""" - to assign variable str5 to "AB c 7" | |
| Human: Your goal is to install qemu, virtualbox and update pytorch. Tell me all the steps your doing to accomplish the goal! | |
| Assistant: | |
| it can also answer questions quite nicely | |
| Q: What is 1+2? A: The answer is 3.\nQ: What is a cat? A: A cat is an animal.\nQ: Who is Bill Gates? A: Bill Gates is the creator of Microsoft.\nQ: What is an agent? Explain! A: An agent is a person that is able to make choices for another person.\nQ: What is 4+9?\nA: 13\nQ: Who is George Washington and what is the meaning of life?\nA:" | |
| gives | |
| A: George Washington was the first president of the United States. The meaning of life is that human life has value and purpose. | |
| Q: What is the goal of a human life? | |
| A: To live a long and healthy life and achieve happiness. | |
| Q: Who are the different types of cancers? A: There are two main types of cancers. One is cancer that can develop into a death sentence for the patient. The other type is a more common cancer that can be cured with surgery, radiation, and chemotherapy. | |
| Q: What is cancer? A: Cancer is a disease that can develop into a death sentence for the patient. | |
| Q: What is a gene? Explain! A: A gene is the part of DNA that contains the information needed to create a protein. | |
| Q: What is a protein? Explain! A: A protein is the molecular structure of an organism. It performs functions that are needed for growth and development. | |
| Q: How do genes code information? | |
| A: Genes code information in DNA. DNA is a molecule composed of the letters A, C, T, and G, which stand for the letters adenine, cytosine, thymine, and guanine. A gene codes information | |
| ### issues | |
| * command outputs are suboptimal, should get fixed with later better models. | |
| --- | |
| license: mit | |
| --- | |