Update README.md

README.md

@@ -16,11 +16,11 @@ base_model:
 - convbert-base
 ---
 
-# Themida-ConvBERT Legal 21G: A Greek Legal Language Model with Quality-Based Data Repetition
+# GEM-ConvBERT HQ Legal: A Greek Legal Language Model with Quality-Based Data Repetition
 
 ## Model Description
 
-**Themida-ConvBERT Legal 21G** is a ConvBERT-base model pre-trained from scratch on a strategically curated 21GB corpus of Greek legal, parliamentary, and governmental text. This model employs an innovative **quality-based data repetition strategy**, where higher-quality legal sources are repeated multiple times during training to enhance the model's understanding of premium legal terminology and concepts.
+**GEM-ConvBERT HQ Legal** is a ConvBERT-base model pre-trained from scratch on a strategically curated 21GB corpus of Greek legal, parliamentary, and governmental text. This model employs an innovative **quality-based data repetition strategy**, where higher-quality legal sources are repeated multiple times during training to enhance the model's understanding of premium legal terminology and concepts.
 
 ConvBERT combines the strengths of BERT with span-based dynamic convolution, replacing some self-attention heads with more efficient convolutional layers. This hybrid architecture provides better efficiency and performance, particularly suitable for understanding local patterns in legal text while maintaining global context awareness.
 
@@ -36,8 +36,8 @@ from transformers import pipeline
 # Load the model
 fill_mask = pipeline(
     "fill-mask",
-    model="novelcore/themida-convbert-legal-21G-8-gpu",
-    tokenizer="novelcore/themida-convbert-legal-21G-8-gpu"
+    model="novelcore/gem-convbert-hq-legal",
+    tokenizer="novelcore/gem-convbert-hq-legal"
 )
 
 # Example from a legal context
@@ -54,8 +54,8 @@ For downstream tasks:
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
 # For legal document classification
-tokenizer = AutoTokenizer.from_pretrained("novelcore/themida-convbert-legal-21G-8-gpu")
-model = AutoModelForSequenceClassification.from_pretrained("novelcore/themida-convbert-legal-21G-8-gpu")
+tokenizer = AutoTokenizer.from_pretrained("novelcore/gem-convbert-hq-legal")
+model = AutoModelForSequenceClassification.from_pretrained("novelcore/gem-convbert-hq-legal")
 ```
 
 ## Training Data
@@ -175,79 +175,4 @@ The ConvBERT architecture is particularly well-suited for legal text processing:
 
 ### Training Efficiency
 
-The model achieved exceptional training efficiency, completing training in only **45 hours 32 minutes** - significantly faster than comparable architectures while processing the expanded 21.12GB dataset.
-
-## Evaluation Results
-
-The model shows stable convergence with the quality-based repetition strategy and ConvBERT architecture:
-
-| Model | Architecture | Training Loss | Evaluation Loss | Training Time |
-| :--- | :--- | :--- | :--- | :--- |
-| `Themida-ConvBERT Legal 21G` (this model) | ConvBERT-base | 0.6413 | 0.604455 | 45h 32m |
-
-*Performance on downstream tasks will be updated as evaluation results become available.*
-
-## Intended Uses
-
-### Primary Use Cases
-- Legal document analysis and classification
-- Named entity recognition in Greek legal texts
-- Legal question answering systems
-- Compliance monitoring and regulatory analysis
-- Legal text similarity and retrieval
-- Legal terminology extraction and understanding
-- Legal clause and entity span detection
-
-### Secondary Use Cases
-- General Greek text understanding (with potential performance degradation)
-- Contract analysis and review
-- Legislative text analysis
-- Regulatory compliance checking
-
-### Advantages of ConvBERT + Quality-Based Training
-- **Enhanced legal vocabulary**: Better understanding of sophisticated legal terminology
-- **Improved pattern recognition**: ConvBERT's convolutions excel at legal phrase patterns
-- **Efficient processing**: Faster training and inference than pure attention models
-- **Better span understanding**: Superior performance on legal entity and clause detection
-- **EU legal compliance**: Better handling of European regulatory language
-
-## Limitations and Bias
-
-- The model may reflect biases present in Greek legal and governmental texts
-- Quality-based repetition may amplify biases present in higher-quality sources
-- Performance may degrade on informal or colloquial Greek text
-- Limited knowledge of legal concepts post-training data cutoff
-- Optimized specifically for Greek legal domain; may not generalize well to other domains
-- ConvBERT architecture may require specific fine-tuning approaches different from BERT
-
-## Technical Specifications
-
-- **Model Size**: ~106M parameters
-- **Architecture**: ConvBERT-base (12 layers, 12 attention heads, conv kernel size 9)
-- **Training Time**: 45 hours 32 minutes on 8x A100 GPUs
-- **Effective Dataset Size**: 21.12GB (with quality-based repetition)
-- **Memory Requirements**: Efficient memory usage due to hybrid architecture
-- **Inference Speed**: Faster than pure attention models due to convolutional components
-
-## Model Card Authors
-
-[Your Name / Your Organization's Name]
-
-## Citation
-
-If you use this model in your research, please cite it as follows:
-
-```bibtex
-@misc{your_name_2025_themida_convbert_21g,
-  author = {[Your Name/Organization]},
-  title = {Themida-ConvBERT Legal 21G: A Greek Legal Language Model with Quality-Based Data Repetition},
-  year = {2025},
-  publisher = {Hugging Face},
-  journal = {Hugging Face Hub},
-  howpublished = {\url{https://huggingface.co/novelcore/themida-convbert-legal-21G-8-gpu}},
-}
-```
-
-## Acknowledgments
-
-We thank the Greek government institutions for making their legal texts publicly available, enabling the creation of this specialized language model for the Greek legal domain. Special recognition for the innovative combination of ConvBERT architecture with quality-based data repetition strategy, resulting in exceptional training efficiency and enhanced legal text understanding capabilities.
+The model achieved exceptional training efficiency, completing training in only **45 hours 32 minutes** - significantly faster than comparable architectures while processing the expanded 21.12GB dataset.