HelloImSteven/AppleScript-Summarizer

Model Description

This model summarizes individual lines of AppleScript code and provides insights into their purpose and functionality. The model is trained on the applescript-lines-annotated dataset.

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, AutoConfig, pipeline

model_selector = "HelloImSteven/AppleScript-Summarizer"
tokenizer = AutoTokenizer.from_pretrained(model_selector)

model = AutoModelForSeq2SeqLM.from_pretrained(model_selector)
config = AutoConfig.from_pretrained(model_selector)

pipe = pipeline("summarization", model=model, config=config, tokenizer=tokenizer)

raw_code = [
    "if myThing is 0 then set success to true",
    "set numInputs to 5",
    "tell application \"System Events\" to click button \"OK\" of window 1 of application process \"Finder\""
]

res = pipe(raw_code)
for i in range(len(res)):
    print("\n\nInput: '" + raw_code[i] + "'")
    print("\nSummary: " + res[i]["summary_text"] + '\n')

The model performs best when given adequate context, such as meaningful variable and handler names. For example, it will perform better when provided the input "on makeDirectory(dirPath)" than it will for "on makeDir(x)".

Example Outputs

Input	Output
on makeDirectory(dirPath)	This line of AppleScript code begins the definition of a handler named "makeDirectory". Based on its name, the handler is likely used to create a directory path from the path stored in the variable "dirPath".
open location "https://huggingface.co"	This is a line of AppleScript code that calls the built-in "open location" handler, providing the URL "https://huggingface.co" as a parameter. This is likely used to open a web browser.
do shell script "open -a 'Google Chrome'"	This is a line of AppleScript code that calls the "open -a" handler of the "Google Chrome" command, providing the output as a shell command. When executed, this line will open the Chrome web browser.

Limitations

This model is a work in progress. It is trained on a small dataset of annotated AppleScript lines, but that dataset will grow in time. For now, however, this leads to several limitations of the model:

1. There are concepts and line structures that are simply absent from the dataset, so the model will struggle with them. If you identify such a concept, please consider providing feedback (e.g. expected output).
2. While the model is fine-tuned from Bart and thus has adequate knowledge of outside concepts, it is not able to give explanations of all possible concepts. For example, for inputs involving the do shell script command, it will be able to provide explanations of some shell scripts better than others. This is not an area of focus for this model, so do not expect significant improvements in future versions.
3. This is not a conversational model.

Validation Metrics

• Loss: 0.20628
• Rouge1: 0.60299
• Rouge2: 0.41118
• rougeL: 0.52186
• RougeLsum: 0.52207