Update the code format according to python syntax.
#4
by imjunaidafzal - opened
README.md
CHANGED
|
@@ -65,49 +65,49 @@ Any model can provide inaccurate or incomplete information, and should be used i
|
|
| 65 |
## How to Get Started with the Model
|
| 66 |
|
| 67 |
The fastest way to get started with BLING is through direct import in transformers:
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
Please refer to the generation_test .py files in the Files repository, which includes 200 samples and script to test the model. The **generation_test_llmware_script.py** includes built-in llmware capabilities for fact-checking, as well as easy integration with document parsing and actual retrieval to swap out the test set for RAG workflow consisting of business documents.
|
| 74 |
|
| 75 |
The DRAGON model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
|
| 80 |
|
| 81 |
1. Text Passage Context, and
|
| 82 |
2. Specific question or instruction based on the text passage
|
| 83 |
|
| 84 |
To get the best results, package "my_prompt" as follows:
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
|
| 89 |
If you are using a HuggingFace generation script:
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
|
| 112 |
## Model Card Contact
|
| 113 |
|
|
|
|
| 65 |
## How to Get Started with the Model
|
| 66 |
|
| 67 |
The fastest way to get started with BLING is through direct import in transformers:
|
| 68 |
+
```python
|
| 69 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 70 |
+
tokenizer = AutoTokenizer.from_pretrained("dragon-yi-6b-v0")
|
| 71 |
+
model = AutoModelForCausalLM.from_pretrained("dragon-yi-6b-v0")
|
| 72 |
+
```
|
| 73 |
Please refer to the generation_test .py files in the Files repository, which includes 200 samples and script to test the model. The **generation_test_llmware_script.py** includes built-in llmware capabilities for fact-checking, as well as easy integration with document parsing and actual retrieval to swap out the test set for RAG workflow consisting of business documents.
|
| 74 |
|
| 75 |
The DRAGON model was fine-tuned with a simple "\<human> and \<bot> wrapper", so to get the best results, wrap inference entries as:
|
| 76 |
+
```python
|
| 77 |
+
full_prompt = "<human>: " + my_prompt + "\n" + "<bot>:"
|
| 78 |
+
```
|
| 79 |
The BLING model was fine-tuned with closed-context samples, which assume generally that the prompt consists of two sub-parts:
|
| 80 |
|
| 81 |
1. Text Passage Context, and
|
| 82 |
2. Specific question or instruction based on the text passage
|
| 83 |
|
| 84 |
To get the best results, package "my_prompt" as follows:
|
| 85 |
+
```python
|
| 86 |
+
my_prompt = {{text_passage}} + "\n" + {{question/instruction}}
|
| 87 |
+
```
|
| 88 |
|
| 89 |
If you are using a HuggingFace generation script:
|
| 90 |
+
```python
|
| 91 |
+
# prepare prompt packaging used in fine-tuning process
|
| 92 |
+
new_prompt = "<human>: " + entries["context"] + "\n" + entries["query"] + "\n" + "<bot>:"
|
| 93 |
+
|
| 94 |
+
inputs = tokenizer(new_prompt, return_tensors="pt")
|
| 95 |
+
start_of_output = len(inputs.input_ids[0])
|
| 96 |
+
|
| 97 |
+
# temperature: set at 0.3 for consistency of output
|
| 98 |
+
# max_new_tokens: set at 100 - may prematurely stop a few of the summaries
|
| 99 |
+
|
| 100 |
+
outputs = model.generate(
|
| 101 |
+
inputs.input_ids.to(device),
|
| 102 |
+
eos_token_id=tokenizer.eos_token_id,
|
| 103 |
+
pad_token_id=tokenizer.eos_token_id,
|
| 104 |
+
do_sample=True,
|
| 105 |
+
temperature=0.3,
|
| 106 |
+
max_new_tokens=100,
|
| 107 |
+
)
|
| 108 |
+
|
| 109 |
+
output_only = tokenizer.decode(outputs[0][start_of_output:],skip_special_tokens=True)
|
| 110 |
+
```
|
| 111 |
|
| 112 |
## Model Card Contact
|
| 113 |
|