CLSE sublexical embeddings (ES / EN / FR)

Word2Vec CBOW embeddings at three levels (phoneme, syllable, word) for Spanish, English, and French. They accompany the manuscript CLSE: Cross-Linguistic Evidence from Sublexical Embeddings (Puertas, Espinosa Orjuela, Payares, Boula de Mareüil, Martinez-Santos; submitted to Language Resources and Evaluation).

Code, training pipeline, and evaluation notebooks: https://github.com/VerbaNexAI/PhoneticAffectIntensityMultilingual.

Files

models/<level>_embedding_<condition>_<lang>.model
frequency/frequency_<level>_embedding_<condition>_<lang>.csv   (token;freq)
<condition> Corpus Role in the paper
wiki Wikipedia 20231101 dump, stopwords kept Main Wikipedia condition
mixed 70% Wikipedia + 30% Exorde social media (sentence level) Wikipedia–Exorde mixed condition (descriptive RSA comparison, Appendix B)

<level> is phoneme, syllable, or word; <lang> is es, en, or fr. That gives 18 models and 18 accompanying frequency tables.

  • Phoneme tokens are IPA from Epitran (spa-Latn, eng-Latn, fra-Latn). The filtered inventories contain 29 ES, 38 EN and 36 FR phonemes in the main Wikipedia condition, and 26 ES, 38 EN and 36 FR phonemes in the Wikipedia–Exorde mixed condition.
  • Syllables come from spaCy tokens split with spacy-syllables / Pyphen hyphenation dictionaries, at the word-token level, then transcribed to IPA with Epitran (for example ka, ɾa).
  • Words are cleaned, lower-cased tokens.

Training settings

Gensim 3.8 Word2Vec CBOW, 150 dimensions, window 5, 10 epochs, 20 negative samples, subsampling 6e-5, alpha 0.03 → 0.0007, min_count 50 (word) / 10 (phoneme, syllable).

The three main-condition word models (word_embedding_wiki_es.model, word_embedding_wiki_en.model, and word_embedding_wiki_fr.model) contain the word vectors only. Their output-layer training weights (syn1neg) were removed to reduce size, so they can be loaded and queried but not trained further. All other models keep the full training state.

Usage

from huggingface_hub import snapshot_download
from gensim.models import Word2Vec

path = snapshot_download("UTB/clse-sublexical-embeddings")
model = Word2Vec.load(f"{path}/models/syllable_embedding_wiki_es.model")
model.wv.most_similar("ta", topn=5)

The evaluation uses Gensim 3.8.3 to load these files.

Sources and license

The embeddings and frequency tables are released under CC BY 4.0. The training text is not included. It came from wikimedia/wikipedia (CC BY-SA 4.0) and, for the mixed condition, Exorde/exorde-social-media-one-month-2024 (MIT); those sources remain under their own terms.

Citation

Puertas, E., Espinosa Orjuela, J. F., Payares, E., Boula de Mareüil, P., Martinez-Santos, J. C. CLSE: Cross-Linguistic Evidence from Sublexical Embeddings. Manuscript submitted to Language Resources and Evaluation.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train UTB/clse-sublexical-embeddings