Add CyberSLM-33M (base + instruct) results
#43
by sabari2005 - opened
Fills in the null placeholder entries from #42 with evaluated scores and corrects the model card links (repos are cyberslm-33m-base / cyberslm-33m-instruct).
Evaluation: lm-eval harness, zero-shot, acc_norm, dtype float32, batch 32, CUDA (L40S); ArithMark-2 via the official benchmark_arithmark-2.0.py.
| Model | ARC-Easy | ARC-Challenge | HellaSwag | PIQA | ArithMark-2 |
|---|---|---|---|---|---|
| CyberSLM-33M-Base | 32.15 | 23.98 | 26.30 | 52.23 | 26.56 |
| CyberSLM-33M-Instruct | 29.04 | 22.53 | 26.38 | 52.34 | 27.04 |
Models: https://huggingface.co/sabari2005/cyberslm-33m-base , https://huggingface.co/sabari2005/cyberslm-33m-instruct (open weights, Apache-2.0, 33,531,264 params, trained from scratch on a cybersecurity corpus).
Full lm-eval result JSONs available for verification — happy to attach them in the discussion.
Datdanboi25 changed pull request status to merged