Update README.md
Browse files
README.md
CHANGED
|
@@ -90,4 +90,78 @@ I chose two models of a similar size from large AI researchers as benchmarks.
|
|
| 90 |
| ROUGE | **.1097** | .1053 | .1075 | **.2074** |
|
| 91 |
| BLEU | .0051 | .0032 | **.0059** | **.0702** |
|
| 92 |
|
| 93 |
-
SurfMine does better in all metrics
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
| ROUGE | **.1097** | .1053 | .1075 | **.2074** |
|
| 91 |
| BLEU | .0051 | .0032 | **.0059** | **.0702** |
|
| 92 |
|
| 93 |
+
SurfMine does better in all metrics when compared to the base model as well as the chosen benchmark models.
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
## Usage and Intended Uses
|
| 97 |
+
|
| 98 |
+
The model can be loaded using the transformers library
|
| 99 |
+
|
| 100 |
+
```{python}
|
| 101 |
+
tokenizer = AutoTokenizer.from_pretrained("pfost-bit/SurfMine", padding_side = 'left')
|
| 102 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 103 |
+
"pfost-bit/SurfMine",
|
| 104 |
+
torch_dtype="auto",
|
| 105 |
+
device_map="auto"
|
| 106 |
+
)
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
The model expects as input the text from a NWS near shore forecast and is trained to take these into account. Here is an example of a NWS input:
|
| 110 |
+
|
| 111 |
+
```{text}
|
| 112 |
+
'Wind: NE winds 5 kt, Seas: 2 ft, Wave Detail: SE 2 ft at 8 seconds and E 1 ft at 5 seconds.'
|
| 113 |
+
```
|
| 114 |
+
|
| 115 |
+
### Prompt Format
|
| 116 |
+
|
| 117 |
+
The prompt format for this model is as follows:
|
| 118 |
+
|
| 119 |
+
It has an instruction text and is then asked to respond the specific near shore forecast. I wrote a short python script to generate prompts
|
| 120 |
+
|
| 121 |
+
```{python}
|
| 122 |
+
def dynamic_prompt(prompt_text):
|
| 123 |
+
instruction_text = (
|
| 124 |
+
"Output a human-readable surf-forecast similar to that of a veteran surf-observer. "
|
| 125 |
+
"The response should take into account the winds, sea-state, and wave period. "
|
| 126 |
+
"The final output should be a few short sentences, with some surfing lingo and flair."
|
| 127 |
+
)
|
| 128 |
+
|
| 129 |
+
prompt = (
|
| 130 |
+
f"Q: {instruction_text}\n\n"
|
| 131 |
+
f"Now, respond to the following forecast data:\n"
|
| 132 |
+
f"Input: {prompt_text}\n"
|
| 133 |
+
f"A: "
|
| 134 |
+
)
|
| 135 |
+
|
| 136 |
+
return prompt
|
| 137 |
+
```
|
| 138 |
+
Combining this prompt with a general text generator pipeline we can see results
|
| 139 |
+
|
| 140 |
+
### Expected Output Format
|
| 141 |
+
|
| 142 |
+
Using the pipeline
|
| 143 |
+
|
| 144 |
+
```{python}
|
| 145 |
+
text_gen = pipeline(
|
| 146 |
+
"text-generation",
|
| 147 |
+
model = model,
|
| 148 |
+
tokenizer = tokenizer,
|
| 149 |
+
dtype = torch.bfloat16,
|
| 150 |
+
device_map = "auto",
|
| 151 |
+
do_sample = False
|
| 152 |
+
)
|
| 153 |
+
|
| 154 |
+
text_gen(
|
| 155 |
+
prompt_example,
|
| 156 |
+
max_new_tokens = 128, # Limit generation length
|
| 157 |
+
return_full_text = False
|
| 158 |
+
)
|
| 159 |
+
```
|
| 160 |
+
|
| 161 |
+
We can see the model response, for the example given above the generated response is:
|
| 162 |
+
|
| 163 |
+
```{python}
|
| 164 |
+
Hey everyone! There is not much going on behind the shop at the moment. Sets are still breaking close to shore in the knee high range but pretty much nothing bigger than that. Wind is blowing ENE at 13mph keeping some texture on the surface. We are approaching high tide, slotted for 3:06pm. Check back Later!
|
| 165 |
+
```
|
| 166 |
+
|
| 167 |
+
|