| | --- |
| | datasets: |
| | - mvasiliniuc/iva-swift-codeint-clean-train |
| | - mvasiliniuc/iva-swift-codeint-clean-valid |
| | language: |
| | - code |
| | tags: |
| | - gpt2 |
| | - code |
| | - swift |
| | - mobile |
| | - generation |
| | widget: |
| | - text: "/*\n A function that returns the time zone currently configured on the device.\n*/\n" |
| | example_title: "Get the current time zone of the device" |
| | - text: "/*\n A function that returns the current version of the operating system.\n*/\n" |
| | example_title: "Get current device operating system" |
| | - text: "/* \nA function that fires an NSNotification named 'MyUpdate'. \n*/\npublic func post" |
| | example_title: "Post NSNotification" |
| | - text: "/* \nA public function that saves a given String value in UserPreference at a given String key.\n*/\n" |
| | example_title: "Save to UserPreference" |
| | --- |
| | iva-codeint-swift-small GPT-2 is (small version - 239.4M parameters) trained from scratch to obtain results in the text-to-code task tailored for Swift language used |
| | in native mobile development (iOS). |
| |
|
| | ## Usage |
| |
|
| | ```Python |
| | from transformers import pipeline |
| | |
| | pipe = pipeline("text-generation", model="mvasiliniuc/iva-codeint-swift-small") |
| | outputs = pipe("func triggerNSNotification") |
| | |
| | ``` |
| |
|
| | ### Inference |
| | ```Python |
| | API_URL = "https://api-inference.huggingface.co/models/mvasiliniuc/iva-codeint-swift-small" |
| | headers = {"Authorization": "Bearer <key>"} |
| | def query(payload): |
| | response = requests.post(API_URL, headers=headers, json=payload) |
| | return response.json() |
| | |
| | output = query({ |
| | "inputs": """ |
| | /* |
| | A function that gets the current device operating system. |
| | */ |
| | """ |
| | }) |
| | pprint.pprint(output, compact=True) |
| | ``` |
| |
|
| | ## Training |
| |
|
| | | Config | Value | |
| | |------|------------------| |
| | | seq length | 1024 | |
| | | weight decay | 0.1 | |
| | | learning rate | 0.0005 | |
| | | max eval steps | -1 | |
| | | shuffle buffer | 10000 | |
| | | max train steps | 150000 | |
| | | mixed precision | fp16 | |
| | | num warmup steps | 2000 | |
| | | train batch size | 5 | |
| | | valid batch size | 5 | |
| | | lr scheduler type | cosine | |
| | | save checkpoint steps | 15000 | |
| | | gradient checkpointing | false | |
| | | gradient accumulation steps | 1 | |
| |
|
| | ## Resources |
| |
|
| | Resources used for research: |
| | * [Training a causal language model from scratch](https://huggingface.co/learn/nlp-course/chapter7/6) |
| | * [CodeParrot a GPT-2 model (1.5B parameters) trained to generate Python code](https://huggingface.co/codeparrot/codeparrot) |
| |
|
