File size: 946 Bytes
34a1ec6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | ---
library_name: mlx
pipeline_tag: text-generation
tags:
- music
- piano-roll
- causal-lm
- qwen3
- safetensors
---
# Musicroll-50M
Musicroll-50M is a small Qwen3-style causal language model over a 259-token
byte-level music vocabulary. It was used as the MusicRoll negative-control model
in the replication artifact for "Many Next-Token Predictors are In-Context
Learners".
The checkpoint is provided as `model.safetensors` with a compact tokenizer:
- vocabulary size: 259
- hidden size: 512
- layers: 16
- attention heads: 4
- key/value heads: 2
- context length: 4096
- BOS/EOS/PAD token ids: 256/257/258
The bitstring replication harness encodes tasks as piano-roll-like text and
loads this checkpoint through the local MLX model loader.
## Intended Use
This upload is primarily for reproducing the MusicRoll negative-control runs in
the ICLManyReplication artifact. General music-generation behavior has not been
characterized here.
|