Update README.md

README.md

@@ -1,3 +1,256 @@
 ---
 license: apache-2.0
+library_name: transformers
+pipeline_tag: fill-mask
+tags:
+  - music
+  - symbolic-music
+  - chords
+  - chord-embeddings
+  - deberta-v2
+  - masked-language-modeling
+  - feature-extraction
 ---
+
+# ChordBERT
+
+ChordBERT is a compact DeBERTa-v2 masked language model for symbolic chord
+sequences. It can predict masked chords or act as an encoder that maps a chord
+progression to a 256-dimensional embedding.
+
+The model operates on chord tokens, not raw audio, MIDI files, natural-language
+descriptions, or note-level symbolic music.
+
+## Model details
+
+| Property | Value |
+|---|---|
+| Architecture | DeBERTa-v2 masked language model |
+| Transformer layers | 4 |
+| Attention heads | 4 |
+| Hidden size | 256 |
+| Intermediate size | 1,024 |
+| Vocabulary size | 3,230 |
+| Parameters | 4,713,886 |
+| Recommended maximum length | 256 tokens including boundary tokens |
+| Training objective | Masked language modeling |
+| Training domain | Chordonomicon chord sequences |
+| License | Apache-2.0 |
+
+This repository contains the original Chordonomicon-pretrained ChordBERT
+checkpoint. It is not the separately adapted `ChordBERT + WIR` checkpoint.
+
+## Intended uses
+
+ChordBERT is intended for research with harmonic sequences, including:
+
+- chord and progression embeddings;
+- harmonic similarity and retrieval;
+- clustering and visualization of musical works;
+- masked-chord prediction;
+- feature extraction for downstream music-information-retrieval models.
+
+The model was not designed for audio transcription, music generation,
+copyright detection, composer attribution, or high-stakes cultural and
+historical judgments.
+
+## Input format
+
+Input must be a whitespace-separated sequence of tokens from the supplied
+tokenizer vocabulary:
+
+```text
+C G Amin F
+Cmaj7 Amin7 Dmin7 G7
+Bb F Gmin Eb
+```
+
+Chord spelling follows the Chordonomicon convention:
+
+- major triads use only the root, such as `C`;
+- sharps use `s`, such as `Csmin` for C-sharp minor;
+- flats use `b`, such as `Bb`;
+- common suffixes include `min`, `7`, `maj7`, `min7`, `dim`, `dim7`, and
+  `aug`;
+- some structural markers, such as `<verse_1>`, are present in the vocabulary.
+
+Unknown or differently formatted symbols may become `<unk>`. Check token
+coverage before embedding a new corpus:
+
+```python
+from transformers import AutoTokenizer
+
+model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+tokens = "C G Amin F".split()
+unknown = [token for token in tokens if token not in tokenizer.get_vocab()]
+print("Unknown tokens:", unknown)
+```
+
+## Masked-chord prediction
+
+```python
+from transformers import pipeline
+
+model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"
+fill_mask = pipeline("fill-mask", model=model_id, tokenizer=model_id)
+
+predictions = fill_mask("C G <mask> F", top_k=5)
+for prediction in predictions:
+    print(prediction["token_str"], prediction["score"])
+```
+
+## Generate a progression embedding
+
+The checkpoint does not include a trained sentence-pooling head. A practical
+default is mean pooling over non-special tokens:
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModel.from_pretrained(model_id).eval()
+
+progressions = [
+    "C G Amin F",
+    "Dmin7 G7 Cmaj7",
+]
+
+batch = tokenizer(
+    progressions,
+    padding=True,
+    truncation=True,
+    max_length=256,
+    return_tensors="pt",
+)
+
+with torch.inference_mode():
+    hidden = model(**batch).last_hidden_state
+
+pool_mask = batch["attention_mask"].bool()
+for special_id in tokenizer.all_special_ids:
+    pool_mask &= batch["input_ids"] != special_id
+
+weights = pool_mask.unsqueeze(-1).to(hidden.dtype)
+embeddings = (hidden * weights).sum(dim=1) / weights.sum(dim=1).clamp(min=1)
+
+print(embeddings.shape)  # torch.Size([2, 256])
+```
+
+The resulting vectors are not normalized. Apply L2 normalization if cosine
+similarity is the downstream comparison:
+
+```python
+embeddings = torch.nn.functional.normalize(embeddings, p=2, dim=1)
+```
+
+## Long sequences
+
+For progressions longer than 254 chord tokens:
+
+1. split the sequence into chunks of at most 254 tokens;
+2. embed each chunk with the pooling procedure above;
+3. combine chunk embeddings using the number of chord tokens as weights.
+
+This leaves room for the two boundary tokens added by the tokenizer and matches
+the evaluation setup used for this checkpoint.
+
+## Evaluation
+
+The checkpoint was evaluated on a leakage-audited validation partition derived
+from DCML corpora v2.3:
+
+- 367 musical records;
+- 7 composers;
+- 77,482 chord tokens before evaluation chunking;
+- overlaps with the 2,000-work Cross-era production corpus removed;
+- random seed 42.
+
+These are local research evaluations, not standardized Hugging Face benchmark
+scores.
+
+### Masked language modeling
+
+Fifteen percent of eligible tokens were selected using the standard
+80% mask / 10% random / 10% unchanged corruption scheme.
+
+| Metric | ChordBERT | Randomly initialized DeBERTa |
+|---|---:|---:|
+| Loss | 2.0286 | 8.1270 |
+| Perplexity | 7.6035 | 3,384.7431 |
+| Top-1 accuracy | 53.26% | 0.00% |
+| Top-5 accuracy | 76.60% | 0.00% |
+| Top-10 accuracy | 84.92% | 0.03% |
+| Mean reciprocal rank | 0.6391 | 0.0014 |
+
+The MLM evaluation contains 11,552 masked target tokens.
+
+### Embedding retrieval
+
+Embeddings used non-special-token mean pooling and token-count-weighted chunk
+aggregation.
+
+| Task | R@1 | R@5 | R@10 | MRR |
+|---|---:|---:|---:|---:|
+| Retrieve another movement from the same work | 30.56% | 69.44% | 82.94% | 0.4674 |
+| Match the two halves of a sequence | 49.59% | 73.57% | 85.56% | 0.6082 |
+
+Same-work movement retrieval additionally obtained MAP@10 = 0.2843.
+
+## Limitations
+
+- The model represents tokenized chord labels and ignores melody, rhythm,
+  instrumentation, dynamics, voicing, and audio timbre.
+- Embedding quality depends strongly on chord-recognition quality and agreement
+  with the tokenizer's spelling conventions.
+- The vocabulary is closed. Unsupported chord qualities and enharmonic
+  spellings may map to `<unk>` or require an explicit conversion policy.
+- A 256-token context cannot represent long works in one pass. Chunk pooling
+  loses some long-range harmonic order.
+- Mean pooling is a research convention, not a contrastively trained embedding
+  objective. Similarity scores should be validated for each downstream task.
+- Training-corpus coverage may introduce genre, era, notation, and repertoire
+  biases. Results should not be interpreted as objective measures of musical
+  quality, influence, nationality, or authorship.
+
+## Training information
+
+The supplied checkpoint is recorded in the project artifacts as a
+Chordonomicon-pretrained masked language model. The original detailed training
+hyperparameters and a complete training-data statement are not included with
+this checkpoint; they should not be inferred from the later evaluation and
+adaptation scripts.
+
+The model configuration identifies the architecture as
+`DebertaV2ForMaskedLM`. Weights are stored in `safetensors` format.
+
+## Reproducibility
+
+Recommended dependencies:
+
+```text
+torch
+transformers
+safetensors
+```
+
+For deterministic comparisons, keep the same token conversion, maximum length,
+special-token handling, pooling, chunk weighting, and vector normalization
+across all corpora.
+
+## Citation
+
+```bibtex
+@misc{dad,
+      title={XXX}, 
+      author={XXX},
+      year={XX},
+      eprint={XX},
+      archivePrefix={XX},
+      primaryClass={XX},
+      url={XX}, 
+}
+```