| --- |
| license: apache-2.0 |
| base_model: codeparrot/codeparrot-small |
| tags: |
| - generated_from_trainer |
| model-index: |
| - name: solidity-generator |
| results: [] |
| datasets: |
| - mwritescode/slither-audited-smart-contracts |
| pipeline_tag: text-generation |
| language: |
| - en |
| library_name: transformers |
| widget: |
| - text: "contract MyToken is ERC20{" |
| --- |
| |
|
|
| # solidity-generator |
|
|
| This model is a model specialized in generating Solidity contract codes. Derived from the [codeparrot/codeparrot-small](https://huggingface.co/codeparrot/codeparrot-small) model, it's been meticulously trained on an extensive set of Solidity contracts and patterns, making it apt for assisting in drafting or suggesting contract structures. |
|
|
|
|
| ## Model description |
|
|
| This model has been designed specifically for generating Solidity contracts. Being a derivative of the `codeparrot-small` model, it retains the broader capabilities of the parent model while demonstrating a keen proficiency in understanding and generating Solidity-centric texts. |
|
|
| ### Performance |
|
|
| The model reported a loss of `0.2180` on the evaluation set. |
|
|
| ## Intended Uses & Limitations |
|
|
|
|
| ### Intended Uses: |
| 1. Assist developers by auto-generating contract code snippets based on prompts. |
| 2. Help in understanding and drafting complex contract structures. |
|
|
| ### Limitations: |
| 1. The generated code must be reviewed for security and functional correctness. |
| 2. The clarity of the generated code largely depends on the specificity of the prompt. |
|
|
| ## Training Details |
|
|
| ### Dataset |
| The model was fine-tuned on [mwritescode/slither-audited-smart-contracts](https://huggingface.co/datasets/mwritescode/slither-audited-smart-contracts) dataset comprised of a range of Solidity contracts. |
|
|
|
|
| ## Training and evaluation data |
|
|
| More information needed |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 7e-05 |
| - train_batch_size: 5 |
| - eval_batch_size: 5 |
| - seed: 42 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - lr_scheduler_warmup_steps: 144 |
| - num_epochs: 8 |
| |
| ### Training results |
| |
| | Training Loss | Epoch | Step | Validation Loss | |
| |:-------------:|:-----:|:-----:|:---------------:| |
| | 0.302 | 0.35 | 2000 | 0.3237 | |
| | 0.298 | 0.69 | 4000 | 0.2871 | |
| | 0.232 | 1.04 | 6000 | 0.2645 | |
| | 0.2415 | 1.38 | 8000 | 0.2522 | |
| | 0.2261 | 1.73 | 10000 | 0.2431 | |
| | 0.1924 | 2.07 | 12000 | 0.2332 | |
| | 0.1913 | 2.42 | 14000 | 0.2282 | |
| | 0.2152 | 2.76 | 16000 | 0.2215 | |
| | 0.1508 | 3.11 | 18000 | 0.2180 | |
| |
| |
| ### Framework versions |
| |
| - Transformers 4.31.0 |
| - Pytorch 2.0.1+cu118 |
| - Datasets 2.14.3 |
| - Tokenizers 0.13.3 |
| |
| |
| ## How to Use |
| If you wish to use this model to generate Solidity contract code, follow the steps below: |
| |
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| # Load the model and tokenizer |
| tokenizer = AutoTokenizer.from_pretrained("ckandemir/solidity_generator") |
| model = AutoModelForCausalLM.from_pretrained("ckandemir/solidity_generator") |
| |
| # Input your code prompt |
| input_text = "contract MyToken is ERC20{" |
| input_ids = tokenizer.encode(input_text, return_tensors='pt') |
| sample_output = model.generate(input_ids, do_sample=True, max_length=400, num_return_sequences=1, temperature=0.7) |
| |
| # Decode and print the generated text |
| generated_text = tokenizer.decode(sample_output[0], skip_special_tokens=True) |
| print(generated_text) |
| ``` |
| |
