| license: apache-2.0 | |
| library_name: pytorch | |
| pipeline_tag: text-generation | |
| tags: | |
| - code | |
| - code-generation | |
| model-index: | |
| - name: selection-coder-100m | |
| results: | |
| - task: | |
| type: text-generation | |
| dataset: | |
| name: HumanEval | |
| type: openai_humaneval | |
| metrics: | |
| - type: pass@1 | |
| value: 0.0 | |
| - task: | |
| type: text-generation | |
| dataset: | |
| name: MBPP | |
| type: mbpp | |
| metrics: | |
| - type: pass@1 | |
| value: 1.0 | |
| # selection-coder-100m | |
| A 211.3M-parameter Python code generation model. Context length 1024. | |
| ## Benchmarks (pass@1, greedy) | |
| | Benchmark | pass@1 | n | | |
| |---|---|---| | |
| | HumanEval | 0.0% | 164 | | |
| | MBPP | 1.0% | 200 | | |
| ## Usage | |
| Prompt format: | |
| ``` | |
| ### Instruction: | |
| <your task> | |
| ### Response: | |
| ``` | |