Create README.md

README.md

@@ -0,0 +1,129 @@
+---
+library_name: transformers
+tags:
+- spanish
+- mental-health
+- longformer
+- domain-adaptation
+- nlp
+language:
+- es
+base_model:
+- PlanTL-GOB-ES/longformer-base-4096-bne-es
+---
+
+## Model Description
+
+Longformer-es-mental-base is the base-sized version of the Longformer-es-mental family, a Spanish domain-adapted language model designed for mental health text analysis on long user-generated content.
+The model is intended for scenarios where relevant mental health signals are distributed across multiple messages, such as social media timelines, forum threads, or user message histories.
+
+It is based on the Longformer architecture, which extends the standard Transformer attention mechanism to efficiently process long sequences.
+The model supports input sequences of up to 4096 tokens, enabling it to capture long-range dependencies and temporal patterns that are particularly relevant for mental health screening tasks.
+
+Longformer-es-mental-base was obtained through domain-adaptive pre-training (DAP) on a large corpus of mental health–related texts translated into Spanish from Reddit communities focused on psychological support and mental health discussions.
+This adaptation allows the model to better capture emotional expression, self-disclosure patterns, and discourse structures characteristic of mental health narratives in Spanish.
+
+The model is released as a foundational model and does not include task-specific fine-tuning.
+
+- Developed by: ELiRF group, VRAIN (Valencian Research Institute for Artificial Intelligence), Universitat Politècnica de València
+- Funded by: Spanish Agencia Estatal de Investigación (AEI), MCIN/AEI, ERDF
+- Shared by: ELiRF
+- Model type: Transformer-based masked language model (Longformer)
+- Language: Spanish
+- License: Same as base model (PlanTL-GOB-ES models)
+- Finetuned from model: PlanTL-GOB-ES/longformer-base-4096-bne-es
+
+## Uses
+
+This model is intended for research purposes in the mental health NLP domain.
+
+### Direct Use
+
+The model can be used directly as a language encoder or feature extractor for Spanish mental health–related texts when long input sequences are required and computational efficiency is a concern.
+
+### Downstream Use
+
+Longformer-es-mental-base is primarily intended to be fine-tuned for downstream tasks such as:
+
+- Mental disorder detection
+- Mental health screening
+- User-level and context-level classification
+- Early risk detection tasks involving long message histories
+- Social media analysis related to psychological well-being
+
+### Out-of-Scope Use
+
+- Real-time intervention systems without human supervision
+- Use on languages other than Spanish
+- High-stakes decision-making affecting individuals’ health or safety
+
+## Bias, Risks, and Limitations
+
+- Training data originates from social media platforms, which may introduce demographic, cultural, and linguistic biases.
+- All texts were automatically translated into Spanish, potentially introducing translation artifacts or subtle semantic shifts.
+- Mental health language is highly contextual and subjective; predictions may be unreliable when very limited evidence is available.
+- The model does not provide explanations or clinical interpretations of its outputs.
+
+## How to Get Started with the Model
+
+```python
+from transformers import AutoTokenizer, AutoModel
+
+tokenizer = AutoTokenizer.from_pretrained("ELiRF/Longformer-es-mental-base")
+model = AutoModel.from_pretrained("ELiRF/Longformer-es-mental-base")
+
+inputs = tokenizer(
+    "Ejemplo de texto relacionado con salud mental.",
+    return_tensors="pt",
+    truncation=True,
+    max_length=4096
+)
+
+outputs = model(**inputs)
+```
+
+## Training Details
+
+### Training Data
+
+The model was domain-adapted using a merged corpus composed of:
+
+- Reddit SuicideWatch and Mental Health Collection (SWMH)
+- Reddit Mental Health Narratives (RMHN)
+
+All texts were automatically translated into Spanish using neural machine translation.
+The resulting dataset contains approximately 1.9 million posts from multiple mental health–related communities (e.g., depression, anxiety, suicide ideation, loneliness), providing broad coverage of informal mental health discourse.
+
+### Training Procedure
+
+The model was trained using domain-adaptive pre-training (DAP) with a masked language modeling objective.
+
+- Training regime: fp16 mixed precision
+- Number of epochs: 20
+- Hardware: multiple NVIDIA A40 GPUs
+- Training duration: approximately 4 days
+
+No task-specific fine-tuning is included in this checkpoint.
+
+## Evaluation
+
+### Results
+
+When fine-tuned on Spanish mental health benchmarks, Longformer-es-mental-base shows competitive performance.
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+- Architecture: Longformer
+- Objective: Masked Language Modeling
+- Model size: approximately 150M parameters (base version)
+
+## Citation
+
+This model is part of an ongoing research project.
+The associated paper is currently under review and will be added to this model card once the publication process is completed.
+
+## Model Card Authors
+
+ELiRF research group (VRAIN, Universitat Politècnica de València)