Update README.md
Browse files
README.md
CHANGED
|
@@ -16,10 +16,6 @@ included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet, each analyzing the model
|
|
| 16 |
explanations across these languages and formats.
|
| 17 |
|
| 18 |
Performance Scores (on a scale of 5):
|
| 19 |
-
| Model | Score | # params (Billion) | size (GB) |
|
| 20 |
-
|:----------------------------------------------------------|:--------:|:------------------:|:---------:|
|
| 21 |
-
| ('Qwen/Qwen2.5-Coder-1.5B', 'Java') | 0.545072 |\n| ('Qwen/Qwen2.5-Coder-1.5B', 'JavaScript') | 0.436294 |\n| ('Qwen/Qwen2.5-Coder-1.5B', 'Python') | 0.434845 |\n| ('bigcode/starcoder2-15b', 'Java') | 0.62873 |\n| ('bigcode/starcoder2-15b', 'JavaScript') | 0.536091 |\n| ('bigcode/starcoder2-15b', 'Python') | 0.510945 |\n| ('bigcode/starcoder2-3b', 'Java') | 0.595437 |\n| ('bigcode/starcoder2-3b', 'JavaScript') | 0.500828 |\n| ('bigcode/starcoder2-3b', 'Python') | 0.479801 |\n| ('bigcode/starcoder2-7b', 'Java') | 0.574745 |\n| ('bigcode/starcoder2-7b', 'JavaScript') | 0.465017 |\n| ('bigcode/starcoder2-7b', 'Python') | 0.453512 |\n| ('cmarkea/CodeLlama-34b-hf-4bit', 'Java') | 0.724881 |\n| ('cmarkea/CodeLlama-34b-hf-4bit', 'JavaScript') | 0.609453 |\n| ('cmarkea/CodeLlama-34b-hf-4bit', 'Python') | 0.596516 |\n| ('cmarkea/CodeLlama-70b-hf-4bit', 'Java') | 0.760241 |\n| ('cmarkea/CodeLlama-70b-hf-4bit', 'JavaScript') | 0.652255 |\n| ('cmarkea/CodeLlama-70b-hf-4bit', 'Python') | 0.640797 |\n| ('cmarkea/CodeLlama-7b-hf-4bit', 'Java') | 0.66728 |\n| ('cmarkea/CodeLlama-7b-hf-4bit', 'JavaScript') | 0.542181 |\n| ('cmarkea/CodeLlama-7b-hf-4bit', 'Python') | 0.529498 |\n| ('deepseek-ai/deepseek-coder-1.3b-base', 'Java') | 0.586311 |\n| ('deepseek-ai/deepseek-coder-1.3b-base', 'JavaScript') | 0.450518 |\n| ('deepseek-ai/deepseek-coder-1.3b-base', 'Python') | 0.487962 |\n| ('deepseek-ai/deepseek-coder-6.7b-base', 'Java') | 0.6271 |\n| ('deepseek-ai/deepseek-coder-6.7b-base', 'JavaScript') | 0.496296 |\n| ('deepseek-ai/deepseek-coder-6.7b-base', 'Python') | 0.532503 |\n| ('deepseek-ai/deepseek-coder-7b-base-v1.5', 'Java') | 0.623582 |\n| ('deepseek-ai/deepseek-coder-7b-base-v1.5', 'JavaScript') | 0.503935 |\n| ('deepseek-ai/deepseek-coder-7b-base-v1.5', 'Python') | 0.521639 |\n| ('meta-llama/CodeLlama-13b-hf', 'Java') | 0.680939 |\n| ('meta-llama/CodeLlama-13b-hf', 'JavaScript') | 0.563145 |\n| ('meta-llama/CodeLlama-13b-hf', 'Python') | 0.546593 |\n| ('meta-llama/CodeLlama-34b-hf', 'Java') | 0.728144 |\n| ('meta-llama/CodeLlama-34b-hf', 'JavaScript') | 0.613087 |\n| ('meta-llama/CodeLlama-34b-hf', 'Python') | 0.599918 |\n| ('meta-llama/CodeLlama-70b-hf', 'Java') | 0.76577 |\n| ('meta-llama/CodeLlama-70b-hf', 'JavaScript') | 0.658665 |\n| ('meta-llama/CodeLlama-70b-hf', 'Python') | 0.647235 |\n| ('meta-llama/CodeLlama-7b-hf', 'Java') | 0.670398 |\n| ('meta-llama/CodeLlama-7b-hf', 'JavaScript') | 0.546067 |\n| ('meta-llama/CodeLlama-7b-hf', 'Python') | 0.533356 |\n| ('mistralai/Codestral-22B-v0.1', 'Java') | 0.631256 |\n| ('mistralai/Codestral-22B-v0.1', 'JavaScript') | 0.553193 |\n| ('mistralai/Codestral-22B-v0.1', 'Python') | 0.544975 |
|
| 22 |
-
|
| 23 |
| Model | Score | # params (Billion) | size (GB) |
|
| 24 |
|---------------------------------------------:|:--------:|:------------------:|:---------:|
|
| 25 |
| gpt-4o | 4.51 | N/A | N/A |
|
|
|
|
| 16 |
explanations across these languages and formats.
|
| 17 |
|
| 18 |
Performance Scores (on a scale of 5):
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
| Model | Score | # params (Billion) | size (GB) |
|
| 20 |
|---------------------------------------------:|:--------:|:------------------:|:---------:|
|
| 21 |
| gpt-4o | 4.51 | N/A | N/A |
|