Update README.md
Browse files
README.md
CHANGED
|
@@ -70,13 +70,17 @@ CoALa-1 is a **Base Model (Pretrained)**. It has been trained to predict the nex
|
|
| 70 |
|
| 71 |
## Evaluation Results
|
| 72 |
|
| 73 |
-
CoALa-1 was evaluated using the `lm-evaluation-harness`.
|
| 74 |
|
| 75 |
| Benchmark | Metric | CoALa-1 (183M) | GPT-2 (124M) | OPT-125M |
|
| 76 |
|---|---|---|---|---|
|
| 77 |
| **ARC-Easy** | acc_norm | **28.87%** | 27.00% | 24.50% |
|
| 78 |
| **HellaSwag** | acc_norm | **26.96%** | 28.50% | 26.00% |
|
| 79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
## Technical Specifications
|
| 81 |
|
| 82 |
* **Hidden Size:** 768
|
|
|
|
| 70 |
|
| 71 |
## Evaluation Results
|
| 72 |
|
| 73 |
+
CoALa-1 was evaluated using the `lm-evaluation-harness`. It shows a strong performance in factual knowledge compared to other models in its weight class.
|
| 74 |
|
| 75 |
| Benchmark | Metric | CoALa-1 (183M) | GPT-2 (124M) | OPT-125M |
|
| 76 |
|---|---|---|---|---|
|
| 77 |
| **ARC-Easy** | acc_norm | **28.87%** | 27.00% | 24.50% |
|
| 78 |
| **HellaSwag** | acc_norm | **26.96%** | 28.50% | 26.00% |
|
| 79 |
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
> **Figure 1:** Comparison of ARC-Easy (Knowledge) and HellaSwag (Reasoning) scores. CoALa-1 leads in factual knowledge retrieval among sub-200M parameter models.
|
| 83 |
+
|
| 84 |
## Technical Specifications
|
| 85 |
|
| 86 |
* **Hidden Size:** 768
|