🧠 Static Word Embeddings for Hungarian (huBERT & XLM-RoBERTa)

This repository contains static word embedding models extracted from the following BERT-based models:

πŸ“¦ Available Embedding Variants

Each model is provided in three static embedding variants:

  • Decontextualized: Token embeddings extracted without any surrounding context.
  • Aggregate: Static embeddings computed by averaging token representations of different contexts the word appears in.
  • X2Static: Learned static embeddings trained via the X2Static method, designed to optimize static representations from contextual models.

πŸ§ͺ Use Case

These embeddings were developed and evaluated as part of the paper: A Comparative Analysis of Static Word Embeddings for Hungarian by MΓ‘tΓ© Gedeon. They can be used for intrinsic tasks (e.g., word analogies) and extrinsic tasks (e.g., POS tagging, NER) in Hungarian NLP applications.

The paper can be found here: https://arxiv.org/abs/2505.07809

The corresponding GitHub repository: https://github.com/gedeonmate/hungarian_static_embeddings

πŸ™ Citation

If you use these models, code, or any part of the accompanying materials in your research, please cite:

@article{Gedeon_2025,
   title={A Comparative Analysis of Static Word Embeddings for Hungarian},
   volume={17},
   ISSN={2061-2079},
   url={http://dx.doi.org/10.36244/ICJ.2025.2.4},
   DOI={10.36244/icj.2025.2.4},
   number={2},
   journal={Infocommunications Journal},
   publisher={Infocommunications Journal},
   author={Gedeon, MΓ‘tΓ©},
   year={2025},
   pages={28–34}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for gedeonmate/static_hungarian_bert

Finetuned
(4,325)
this model

Paper for gedeonmate/static_hungarian_bert