| | --- |
| | license: cc-by-4.0 |
| | --- |
| | |
| | This model judges if a given output is sufficient to recreate a given instruction. |
| |
|
| | It's useful for filtering data to train a reverse instruct model. It could also have applications around determining if an output/instruction pair is linked, or around quality filtering data (data where the instruction can be recreated from the output might be higher quality). |
| |
|
| | The model is a binary classifier trained on top of Python 410m with 100k examples for 1 epoch. The final validation loss is .35. You can see an example of a dataset filtered with this model [here](https://huggingface.co/datasets/vikp/reverse_instruct). |
| |
|
| | To use it, pass in this prompt format: |
| |
|
| | ``` |
| | Output |
| | |
| | {output} |
| | |
| | Instruction |
| | |
| | {instruction} |
| | ``` |
| |
|
| | Output should be the output from a model, and instruction should be the instruction that generated the output. The model will return a 0-1 score indicating how effectively the instruction can be recreated. |
