| Quantization made by Richard Erkhov. |
|
|
| [Github](https://github.com/RichardErkhov) |
|
|
| [Discord](https://discord.gg/pvy7H8DZMG) |
|
|
| [Request more models](https://github.com/RichardErkhov/quant_request) |
|
|
|
|
| opt-350m-instruct - bnb 4bits |
| - Model creator: https://huggingface.co/nnpy/ |
| - Original model: https://huggingface.co/nnpy/opt-350m-instruct/ |
|
|
|
|
|
|
|
|
| Original model description: |
| --- |
| license: apache-2.0 |
| datasets: |
| - openchat/openchat_sharegpt4_dataset |
| --- |
|
|
| ## Usage |
| ``` |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| tok = AutoTokenizer.from_pretrained('facebook/opt-350m') |
| model = AutoModelForCausalLM.from_pretrained('prasanna2003/opt-350m-instruct') |
| |
| system_message = "You are AI language model helps the human." |
| input_prompt = "Define data science." |
| |
| prompt = '<system>' + system_message + '<human>' + input_prompt + '<assistant>' |
| prompt = tokenizer(prompt, return_tensors='pt') |
| out = model.generate(**prompt, max_length=120) |
| print(tok.decode(out[0])) |
| ``` |
|
|
|
|
