Instructions to use matteospanio/abc-gpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use matteospanio/abc-gpt with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="matteospanio/abc-gpt")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("matteospanio/abc-gpt") model = AutoModelForCausalLM.from_pretrained("matteospanio/abc-gpt", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use matteospanio/abc-gpt with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf matteospanio/abc-gpt:F16 # Run inference directly in the terminal: llama cli -hf matteospanio/abc-gpt:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf matteospanio/abc-gpt:F16 # Run inference directly in the terminal: llama cli -hf matteospanio/abc-gpt:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf matteospanio/abc-gpt:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf matteospanio/abc-gpt:F16
Use Docker
docker model run hf.co/matteospanio/abc-gpt:F16
- Llama new
- LM Studio
- Jan
- vLLM
How to use matteospanio/abc-gpt with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "matteospanio/abc-gpt" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matteospanio/abc-gpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/matteospanio/abc-gpt:F16
- SGLang
How to use matteospanio/abc-gpt with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "matteospanio/abc-gpt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matteospanio/abc-gpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "matteospanio/abc-gpt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "matteospanio/abc-gpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use matteospanio/abc-gpt with Ollama:
ollama run hf.co/matteospanio/abc-gpt:F16
- Unsloth Desktop
- Docker Model Runner
How to use matteospanio/abc-gpt with Docker Model Runner:
docker model run hf.co/matteospanio/abc-gpt:F16
- Lemonade
How to use matteospanio/abc-gpt with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull matteospanio/abc-gpt:F16
Run and chat with the model
lemonade run user.abc-gpt-F16
List all available models
lemonade list
- Atomic Chat
abc-gpt
A small GPT-2 (26.8M parameters) trained from scratch to write Irish folk tunes in ABC notation. It was pretrained on raw ABC text, finetuned to follow a control code that sets the structure of the tune (number of sections, bars per section, similarity between sections), then trained with GRPO on rewards computed with music21.
It is the companion model of a presentation on the Hugging Face ecosystem: each stage uses a library
of that stack (tokenizers, datatrove, transformers, trl, gradio). Code:
github.com/matteospanio/abc-gpt.
Files
| Path | What it is |
|---|---|
/ (root) |
the final model, after GRPO: what from_pretrained("matteospanio/abc-gpt") loads |
sft/ |
the model after supervised finetuning, before GRPO |
pretrain/ |
the model after pretraining only: it writes ABC but does not read the control code |
abc-gpt-f16.gguf |
the final model in GGUF (F16), for llama.cpp and compatible runtimes |
Usage
The prompt is the control code in an ABC comment line, followed by the tune header; the model writes the body of the tune and stops with its end-of-text token.
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("matteospanio/abc-gpt")
model = AutoModelForCausalLM.from_pretrained("matteospanio/abc-gpt") # subfolder="sft" or "pretrain" for the earlier stages
prompt = "%S:2 B:8 E:5 B:8\nX:1\nL:1/8\nM:6/8\nK:D\n" # a jig in D: 2 sections of 8 bars, the second 50% similar to the first
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, do_sample=True, temperature=1.0, max_new_tokens=512,
pad_token_id=tokenizer.pad_token_id)
print(prompt + tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Use sampling: with greedy decoding a prompt always gives the same tune, and a repetitive one (in a test, a 6/8 tune of 16 bars had only 5 different ones). Paste the result in an ABC player (for example abcjs or EasyABC) to see and hear it.
The GGUF file loads in transformers too (with the gguf package installed):
model = AutoModelForCausalLM.from_pretrained("matteospanio/abc-gpt", gguf_file="abc-gpt-f16.gguf")
tokenizer = AutoTokenizer.from_pretrained("matteospanio/abc-gpt", gguf_file="abc-gpt-f16.gguf")
llama.cpp
hf download matteospanio/abc-gpt abc-gpt-f16.gguf --local-dir .
llama completion -m abc-gpt-f16.gguf -p '%S:2 B:8 E:5 B:8\nX:1\nL:1/8\nM:6/8\nK:D\n' -e \
-n 512 --temp 1.0 --top-k 0 --top-p 1.0 --min-p 0.0
Use completion (plain text continuation), not the chat interface. Give the prompt with -p and -e
(escapes): with -f llama.cpp drops the final newline of the file, and the model was trained on
headers that end with one. --top-k 0 --top-p 1.0 --min-p 0.0 turn off llama.cpp's default filters
and give the plain sampling of the evaluation below. Tested with llama.cpp build 12182: the prompt is
split into the same tokens as with transformers, and the tunes end at the end-of-text token.
Prompt format
| Line | Meaning |
|---|---|
%S:2 B:8 E:5 B:8 |
control code: S: sections, then for each section one E: (0-10, similarity to each earlier section) and B: (bars) |
X:1 |
tune number |
L:1/8 |
unit note length (the model was trained with the headers of IrishMAN, mostly 1/8) |
M:6/8 |
meter: 6/8 jig, 4/4 reel, 3/4, 2/4, 9/8 slip jig, ... |
K:D |
key and mode: D, G, Emin, Amin, Dmix, Ador, ... |
The control code follows IrishMAN: sections end
at [|, ||, |], |:, :: or :|, B counts the bars of a section, and E is
1 - levenshtein(c, p) / max(|c|, |p|) between section c and an earlier section p, in 11 levels.
Speed
On a laptop CPU (Intel Core Ultra 5 226V, 4 threads), a tune of about 100 tokens takes:
| Runtime | Tokens per second | Time per tune |
|---|---|---|
transformers (float32) |
about 180 | about 0.4 s |
llama.cpp (abc-gpt-f16.gguf) |
about 770 | about 0.1 s |
With transformers, more threads are slower for a model this small: 8 threads give about 110 tokens
per second, so set torch.set_num_threads(4).
Model
GPT-2 architecture (GPT2LMHeadModel), trained from random weights.
| Layers Γ width (heads) | 8 Γ 512 (8) |
| Parameters | 26.8M |
| Context | 1,024 positions; trained on sequences of at most 512 tokens |
| Tokenizer | byte-level BPE trained on the IrishMAN tunes, 2,048 tokens; `< |
| Weights | float32 (model.safetensors, 107 MB); F16 in the GGUF file (55 MB) |
Training
Data: IrishMAN, 214,122 training and 2,162 validation tunes in ABC notation, with titles and lyrics removed and a control code computed for each tune. All three stages ran with TRL, in bf16.
| Stage | Data | Settings | Result |
|---|---|---|---|
Pretraining (trl sft) |
214,122 training tunes, raw ABC text | 3 epochs, 20,076 steps, batch 32, lr 6e-4 cosine, 500 warmup steps, weight decay 0.1 | validation loss 1.610 (step 2,000) β 1.010 |
Supervised finetuning (trl sft) |
214,107 tunes (those with a body) as prompt-completion pairs: control code and header β body; loss on the completion only | 1 epoch, 6,691 steps, batch 32, lr 5e-5 cosine, 200 warmup steps | validation loss 1.068 (step 1,000) β 0.994 |
GRPO (trl grpo) |
5,321 prompts (2.5% of the training tunes, after dropping unreadable and too-long tunes) | 500 steps, 8 tunes per prompt, batch 8 Γ 4 accumulation, lr 1e-5, no KL penalty (beta: 0), temperature 1.0 |
mean reward 3.27 β 3.36 (first and last 50 steps) |
GRPO rewards, each between 0 and 1:
| Reward | Weight | Measures | First 50 steps | Last 50 steps |
|---|---|---|---|---|
syntax |
1.0 | music21 parses the whole tune without warnings and finds notes | 0.968 | 0.992 |
meter |
1.0 | share of bars whose length matches the time signature | 0.906 | 0.955 |
note_order |
0.5 | share of notes written in the canonical ABC order (accidental, pitch, octave, length) | 0.998 | 0.999 |
structure |
1.0 | how close the tune's own control code is to the one in the prompt | 0.894 | 0.908 |
Framework versions: TRL 1.17.1, Transformers 5.22.2, PyTorch 2.15.0, Datasets 5.2.0, Tokenizers 0.23.3.
Evaluation
100 tunes per checkpoint, from the same prompts: the first 100 of 176 prompts drawn from the IrishMAN
validation split (10% of the validation tunes that pass the data filter, seed 42), sampled at
temperature 1.0 with python -m abcgpt.evaluate on CPU. Each reward column is a mean between 0 and 1;
the last column counts the tunes that music21 exports to MusicXML.
| Checkpoint | syntax | meter | note_order | structure | MusicXML |
|---|---|---|---|---|---|
pretrain/ |
0.930 | 0.292 | 0.996 | 0.435 | 93 |
sft/ |
1.000 | 0.941 | 1.000 | 0.898 | 100 |
| root (GRPO) | 1.000 | 0.948 | 0.999 | 0.902 | 100 |
The control code of each generated tune, compared with the one in its prompt:
| Checkpoint | same number of sections | same sections and bars | identical code |
|---|---|---|---|
pretrain/ |
27% | 1% | 0% |
sft/ |
88% | 42% | 18% |
| root (GRPO) | 90% | 44% | 14% |
Finetuning makes the difference: the pretrained model never saw the control-code line in training.
GRPO adds little on these prompts: the SFT model already scored close to 1 on syntax and
note_order, and its gains on meter (+0.007) and structure (+0.004) are smaller than the standard
error of these means over 100 tunes (about 0.015-0.019).
Limitations
- The model knows only the tunes of IrishMAN: Irish and other traditional dance tunes, single melody line, a few chord symbols. It does not write titles, lyrics or other styles.
- The rewards check that the ABC is readable and that bars and sections add up. They say nothing about whether a tune is good, and nobody has listened to the generated tunes systematically.
- The control code is followed approximately: in the evaluation above the number of sections was right in 90% of the tunes, the bars of every section in 44%, the whole code in 14%.
- A model trained for several epochs on 214k tunes may reproduce tunes of the training set; this has not been measured.
- IrishMAN is MIT-licensed, but its card says it is "for research use only and not for commercial purposes". Keep that in mind before using generated tunes commercially.
Citation
The dataset and the control codes come from TunesFormer:
@inproceedings{DBLP:conf/hcmir/WuLY023,
author = {Shangda Wu and Xiaobing Li and Feng Yu and Maosong Sun},
title = {TunesFormer: Forming Irish Tunes with Control Codes by Bar Patching},
booktitle = {Proceedings of the 2nd Workshop on Human-Centric Music Information Retrieval 2023
co-located with the 24th International Society for Music Information Retrieval
Conference (ISMIR 2023), Milan, Italy, November 10, 2023},
series = {CEUR Workshop Proceedings},
volume = {3528},
publisher = {CEUR-WS.org},
year = {2023},
url = {https://ceur-ws.org/Vol-3528/paper1.pdf}
}
- Downloads last month
- 43