Update README.md
#3
by j0no12 - opened
README.md
CHANGED
|
@@ -1,46 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
library_name: transformers
|
| 3 |
-
pipeline_tag: text-generation
|
| 4 |
-
tags:
|
| 5 |
-
- qwen3
|
| 6 |
-
- byte-level
|
| 7 |
-
- pretraining
|
| 8 |
-
- symbolic
|
| 9 |
-
license: apache-2.0
|
| 10 |
-
datasets:
|
| 11 |
-
- karpathy/climbmix-400b-shuffle
|
| 12 |
-
- openbmb/Ultra-FineWeb-L3
|
| 13 |
-
- dotlabs/rewrite
|
| 14 |
-
- nvidia/OpenMathInstruct-2
|
| 15 |
-
language:
|
| 16 |
-
- en
|
| 17 |
-
---
|
| 18 |
-
|
| 19 |
-
> [#1 on sub-100m on the open slm leaderboard!](https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard)
|
| 20 |
-
|
| 21 |
-
# 👋 Meet void.1
|
| 22 |
-
|
| 23 |
-
State of the art, small language model pretrained from scratch on a diverse set of high-quality texts and an internal symbolic kernel. This is the first step into a series of models designed for fine-grained understanding of abstract/symbolic reasoning on text while being grounded on english.
|
| 24 |
-
|
| 25 |
-
<video src="https://huggingface.co/dotlabs/void.1/resolve/main/teaser.mp4" autoplay loop width="100%"></video>
|
| 26 |
-
|
| 27 |
-
## Comparison
|
| 28 |
-
|
| 29 |
-
| Model | Params | HellaSwag | PIQA | ARC-Easy | ARC-Challenge | ArithMark-3 | Intelligence Index |
|
| 30 |
-
| ----------------------------------------------------------------- | ------ | --------- | ------ | -------- | ------------- | ----------- | ------------------ |
|
| 31 |
-
| [void.1](https://huggingface.co/dotlabs/void.1)\* | **90.15M** | **38.68%** | **67.46%** | 47.31% | **28.16%** | **44.80%** | **23.92** |
|
| 32 |
-
| [100M-exp](https://huggingface.co/User01110/100M-exp) | 98.16M | 37.78% | 66.97% | **49.83%** | 27.22% | 40.00% | 22.47 |
|
| 33 |
-
| [Rose-1.5-Medium](https://huggingface.co/GODELEV/Rose-1.5-Medium) | 98.28M | 38.09% | 64.80% | 47.22% | 27.13% | 40.70% | 21.07 |
|
| 34 |
-
| [tinctura-v1](https://huggingface.co/bench-labs/tinctura-v1) | 96.2M | 37.96% | 65.61% | 47.98% | 25.77% | 38.40% | 20.81 |
|
| 35 |
-
| [Surjo-100m](https://huggingface.co/SurjoLabs/Surjo-100m) | 97.7M | 35.05% | 63.87% | 47.64% | 25.85% | 38.90% | 18.86 |
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
We used the revision on step 900,000 for evaluations which trained for around 120 billion bytes which is around 30 to 35 billion bpe tokens. For more details on the evals and inference, please have a look at the [official notebook](https://huggingface.co/dotlabs/void.1/blob/main/evals.ipynb).
|
| 39 |
-
|
| 40 |
-
```bibtex
|
| 41 |
-
@misc{dotlabs,
|
| 42 |
-
title = {void: bytes is all you need},
|
| 43 |
-
author = {appvoid},
|
| 44 |
-
year = {2026},
|
| 45 |
-
url = {https://huggingface.co/dotlabs/void.1}
|
| 46 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|