Tabahi/timbrespace-b16e-120

b16e-120

Prosodic sequence encoder and frame-level prosodic clustering model. The clustering model converts six-dimensional prosodic features into 120 cluster IDs. The sequence encoder consumes .vs1.json sequences from the pr field and produces sequence embeddings or training logits, depending on the forward mode.

Checkpoint

The compressed model files are weights_only/m_b16e-120_weights.pt and weights_only/m_b16e-120_weights.safetensors. They contain model weights only; the original optimizer and Lightning training state are not included.

Configuration is preserved in hp_perbert.yaml. The model implementation is included as model_prbert.py. The frame clustering files are under clustering_model/.

Loading

from argparse import Namespace
import torch
from model_prbert import ProsodicSequenceEncoder

checkpoint = torch.load("weights_only/m_b16e-120_weights.pt", weights_only=True)
hp = Namespace(
    pos_max_len=4096,
    hidden_dim=768,
    num_layers=8,
    phoneme_vocab_size=120,
    dropout_rate=0.25,
)

model = ProsodicSequenceEncoder(
    pos_max_len=hp.pos_max_len,
    hidden_dim=hp.hidden_dim,
    num_layers=hp.num_layers,
    vocab_size=hp.phoneme_vocab_size + 1,
    dropout_rate=hp.dropout_rate,
    padding_token_id=-100,
)
state_dict = {key.replace("model.", ""): value for key, value in checkpoint.items()}
model.load_state_dict(state_dict)
model.eval()

# Alternatively, use safetensors.torch.load_file with
# `weights_only/m_b16e-120_weights.safetensors`.

Training data source recorded in the checkpoint: libri_500h.