Update README.md

README.md

@@ -15,11 +15,11 @@ base_model:
 - google/electra-base-discriminator
 ---
 
-# Themida-ELECTRA v2: A Greek Legal Language Model
+# GEM-ELECTRA Legal: A Greek Legal Language Model
 
 ## Model Description
 
-**Themida-ELECTRA v2** is an improved ELECTRA-base model pre-trained from scratch on a large, 17GB corpus of Greek legal, parliamentary, and governmental text. This second version incorporates refined training hyperparameters for enhanced performance and stability. It is designed for understanding the complex vocabulary and context of the legal domain in Greece and the EU.
+**GEM-ELECTRA Legal** is an improved ELECTRA-base model pre-trained from scratch on a large, 17GB corpus of Greek legal, parliamentary, and governmental text. This second version incorporates refined training hyperparameters for enhanced performance and stability. It is designed for understanding the complex vocabulary and context of the legal domain in Greece and the EU.
 
 This model was trained as part of a research project and has been optimized for downstream tasks such as Named Entity Recognition (NER), Text Classification, and Question Answering within the legal field. The ELECTRA architecture provides more efficient pre-training compared to masked language models like BERT by using a generator-discriminator approach.
 
@@ -33,8 +33,8 @@ from transformers import pipeline
 # Load the model
 fill_mask = pipeline(
     "fill-mask",
-    model="novelcore/themida-electra-legal-17G-8-gpu-v2",
-    tokenizer="novelcore/themida-electra-legal-17G-8-gpu-v2"
+    model="novelcore/gem-electra-legal",
+    tokenizer="novelcore/gem-electra-legal"
 )
 
 # Example from a legal context
@@ -43,29 +43,6 @@ text = "Ο κ. Μητσοτάκης <mask> ότι η κυβέρνηση σέβε
 # Get predictions
 predictions = fill_mask(text)
 print(predictions)
-
-# Get predictions
-predictions = fill_mask(text)
-[{'score': 0.20120874047279358,
-  'token': 4014,
-  'token_str': ' ειπε',
-  'sequence': ' ο κ . μητσοτακης ειπε οτι η κυβερνηση σεβεται πληρως τις αποφασεις του συμβουλιου της επικρατειας .'},
- {'score': 0.19406235218048096,
-  'token': 12702,
-  'token_str': ' δηλωσε',
-  'sequence': ' ο κ . μητσοτακης δηλωσε οτι η κυβερνηση σεβεται πληρως τις αποφασεις του συμβουλιου της επικρατειας .'},
- {'score': 0.18023167550563812,
-  'token': 11151,
-  'token_str': ' δηλωνει',
-  'sequence': ' ο κ . μητσοτακης δηλωνει οτι η κυβερνηση σεβεται πληρως τις αποφασεις του συμβουλιου της επικρατειας .'},
- {'score': 0.08440685272216797,
-  'token': 8534,
-  'token_str': ' υποστηριζει',
-  'sequence': ' ο κ . μητσοτακης υποστηριζει οτι η κυβερνηση σεβεται πληρως τις αποφασεις του συμβουλιου της επικρατειας .'},
- {'score': 0.05247046798467636,
-  'token': 3523,
-  'token_str': ' λεει',
-  'sequence': ' ο κ . μητσοτακης λεει οτι η κυβερνηση σεβεται πληρως τις αποφασεις του συμβουλιου της επικρατειας .'}]
 ```
 
 For downstream tasks:
@@ -74,8 +51,8 @@ For downstream tasks:
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
 # For legal document classification
-tokenizer = AutoTokenizer.from_pretrained("novelcore/themida-electra-legal-17G-8-gpu-v2")
-model = AutoModelForSequenceClassification.from_pretrained("novelcore/themida-electra-legal-17G-8-gpu-v2")
+tokenizer = AutoTokenizer.from_pretrained("novelcore/gem-electra-legal")
+model = AutoModelForSequenceClassification.from_pretrained("novelcore/gem-electra-legal")
 ```
 
 ## Training Data
@@ -134,69 +111,4 @@ The model achieved the following performance metrics:
 - **Final Training Loss**: 0.0056
 - **Final Evaluation Loss**: 0.0054
 - **Training Infrastructure**: 8x NVIDIA A100 40GB GPUs
-- **Total Training Steps**: 200,000
-
-### Improvements in v2
-
-This second version incorporates the following improvements over the initial model:
-
-- **Reduced Learning Rate**: Lowered from 8e-4 to 1e-4 for more stable convergence
-- **Extended Training**: Increased from 120,000 to 200,000 steps for better performance
-- **Enhanced Warmup**: Extended warmup period from 6,000 to 12,000 steps for smoother training initialization
-
-## Evaluation Results
-
-The model's performance was evaluated by fine-tuning it on downstream Named Entity Recognition (NER) tasks and comparing it against other legal language models.
-
-*This section should be filled with your specific results. For example:*
-
-| Model | NER F1-score (strict) |
-| :--- | :--- |
-| `AI-team-UoA/GreekLegalRoBERTa_v3` | `[F1-Score for Baseline]` |
-| `Themida-ELECTRA v1` | `[F1-Score for v1]` |
-| `Themida-ELECTRA v2` (this model) | `[F1-Score for v2]` |
-
-## Intended Uses
-
-### Primary Use Cases
-- Legal document analysis and classification
-- Named entity recognition in Greek legal texts
-- Legal question answering systems
-- Compliance monitoring and regulatory analysis
-- Legal text similarity and retrieval
-
-### Secondary Use Cases
-- General Greek text understanding (with potential performance degradation)
-- Legal document summarization
-- Contract analysis and review
-
-## Limitations and Bias
-
-- The model may reflect biases present in Greek legal and governmental texts
-- Performance may degrade on informal or colloquial Greek text
-- Limited knowledge of legal concepts post-training data cutoff
-- Optimized specifically for Greek legal domain; may not generalize well to other domains
-- The ELECTRA architecture may require different fine-tuning approaches compared to BERT-like models
-
-## Model Card Authors
-
-[Your Name / Your Organization's Name]
-
-## Citation
-
-If you use this model in your research, please cite it as follows:
-
-```bibtex
-@misc{your_name_2025_themida_electra_v2,
-  author = {[Your Name/Organization]},
-  title = {Themida-ELECTRA v2: A Greek Legal Language Model},
-  year = {2025},
-  publisher = {Hugging Face},
-  journal = {Hugging Face Hub},
-  howpublished = {\url{https://huggingface.co/[Your Username]/[Your Model Name]}},
-}
-```
-
-## Acknowledgments
-
-We thank the Greek government institutions for making their legal texts publicly available, enabling the creation of this specialized language model for the Greek legal domain.
+- **Total Training Steps**: 200,000