Update README.md

README.md

@@ -20,11 +20,11 @@ datasets:
 - pile-of-law
 ---
 
-# Themida-RoBERTa Legal 26G: A Bilingual Greek-English Legal Language Model
+# GEM-RoBERTa Legal Bilingual: A Bilingual Greek-English Legal Language Model
 
 ## Model Description
 
-**Themida-RoBERTa Legal 26G** is a RoBERTa-base model pre-trained from scratch on a comprehensive 26GB bilingual corpus of Greek and English legal, parliamentary, and governmental text. This model represents the first large-scale bilingual legal language model combining Greek and English legal domains, enabling cross-lingual legal understanding and applications.
+**TGEM-RoBERTa Legal Bilingual** is a RoBERTa-base model pre-trained from scratch on a comprehensive 26GB bilingual corpus of Greek and English legal, parliamentary, and governmental text. This model represents the first large-scale bilingual legal language model combining Greek and English legal domains, enabling cross-lingual legal understanding and applications.
 
 The model employs the RoBERTa architecture optimized for legal text understanding across both languages, with dynamic masking and focused Masked Language Modeling (MLM) training. The bilingual approach allows the model to leverage legal concepts and terminology from both the Greek and Anglo-American legal traditions.
 
@@ -40,19 +40,14 @@ from transformers import pipeline
 # Load the model
 fill_mask = pipeline(
     "fill-mask",
-    model="novelcore/themida-roberta-el-en-legal-26G-8-gpu",
-    tokenizer="novelcore/themida-roberta-el-en-legal-26G-8-gpu"
+    model="novelcore/gem-roberta-bilingual",
+    tokenizer="novelcore/gem-roberta-bilingual"
 )
 
 # Example in Greek
 text_gr = "Ο κ. Μητσοτάκης <mask> ότι η κυβέρνηση σέβεται πλήρως τις αποφάσεις του Συμβουλίου της Επικρατείας."
 predictions_gr = fill_mask(text_gr)
 print("Greek predictions:", predictions_gr)
-
-# Example in English
-text_en = "The Supreme Court <mask> that the constitutional amendment was valid under federal law."
-predictions_en = fill_mask(text_en)
-print("English predictions:", predictions_en)
 ```
 
 For downstream tasks:
@@ -61,8 +56,8 @@ For downstream tasks:
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
 # For bilingual legal document classification
-tokenizer = AutoTokenizer.from_pretrained("novelcore/themida-roberta-el-en-legal-26G-8-gpu")
-model = AutoModelForSequenceClassification.from_pretrained("novelcore/themida-roberta-el-en-legal-26G-8-gpu")
+tokenizer = AutoTokenizer.from_pretrained("novelcore/gem-roberta-bilingual")
+model = AutoModelForSequenceClassification.from_pretrained("novelcore/gem-roberta-bilingual")
 
 # Process texts in both languages
 greek_text = "Το Συνταγματικό Δικαστήριο αποφάσισε..."
@@ -148,105 +143,4 @@ The model achieved the following performance metrics:
 - **Total Training Steps**: 150,000
 - **Total Training Time**: 25 hours 7 minutes
 - **Train/Validation Split**: 95%/5%
-- **Total Training Data**: 26GB bilingual corpus
-
-### Training Infrastructure
-
-The model was trained using distributed training with the following optimizations:
-- **Backend**: NCCL for efficient multi-GPU communication
-- **Mixed Precision**: BFloat16 for improved training stability
-- **Evaluation Frequency**: Every 5,000 steps
-- **Checkpointing**: Every 5,000 steps
-- **Logging**: Every 200 steps
-
-## Key Innovations
-
-### Bilingual Legal Architecture
-This model introduces several innovations for bilingual legal language modeling:
-
-1. **Cross-lingual Legal Understanding**: First model to combine Greek civil law and English common law traditions
-2. **Balanced Language Exposure**: 60:40 Greek-English ratio optimized for both language preservation and cross-lingual transfer
-3. **Legal System Integration**: Combines EU legal frameworks available in both languages for enhanced multilingual legal comprehension
-4. **Efficient Training**: Achieved strong bilingual performance in just 25 hours compared to monolingual equivalents
-
-### Computational Efficiency
-Despite processing a larger 26GB corpus across two languages, the model demonstrates remarkable efficiency:
-- **62% faster training** than previous monolingual large variants (25h 7m vs ~66h)
-- **Enhanced cross-lingual capabilities** without proportional computational cost increase
-- **Optimized tokenizer** handling both Greek and Latin scripts efficiently
-
-## Evaluation Results
-
-The model shows stable convergence across both languages:
-
-| Model | Languages | Training Loss | Evaluation Loss | Training Time | Corpus Size |
-|-------|-----------|---------------|-----------------|---------------|-------------|
-| `Themida-RoBERTa Legal 26G` | Greek + English | 0.7479 | 0.69405 | 25h 7m | 26GB |
-
-*Performance on downstream bilingual legal tasks will be updated as evaluation results become available.*
-
-## Intended Uses
-
-### Primary Use Cases
-- **Bilingual legal document analysis** and classification
-- **Cross-lingual legal information retrieval** and similarity
-- **Greek-English legal translation** and terminology alignment
-- **EU legal compliance** analysis in multilingual contexts
-- **Comparative legal analysis** between civil and common law systems
-- **Multilingual legal question answering** systems
-
-### Secondary Use Cases
-- **Cross-border legal research** and case law analysis
-- **International contract analysis** and review
-- **Legal terminology extraction** in both languages
-- **Regulatory compliance** for multinational operations
-- **Legal education** resources for comparative law
-
-### Advantages of Bilingual Training
-- **Cross-lingual legal transfer**: Understanding legal concepts across different legal traditions
-- **Enhanced EU legal processing**: Better handling of multilingual EU regulatory frameworks
-- **Comparative legal analysis**: Native understanding of both civil law (Greek) and common law (English) concepts
-- **Multilingual legal applications**: Single model for diverse legal language tasks
-
-## Limitations and Bias
-
-- The model may reflect biases present in both Greek and English legal corpora
-- **Language imbalance effects**: 60:40 ratio may lead to stronger Greek legal concept representation
-- Performance may vary between formal legal text and colloquial usage in either language
-- **Cross-lingual interference**: Potential mixing of legal concepts from different legal systems
-- Limited knowledge of legal developments post-training data cutoff
-- May not generalize well to legal domains outside the training corpus scope
-
-## Technical Specifications
-
-- **Model Size**: ~125M parameters
-- **Architecture**: RoBERTa-base (12 layers, 12 attention heads)
-- **Languages**: Greek (el) + English (en)
-- **Training Time**: 25 hours 7 minutes on 8x H100 GPUs
-- **Dataset Size**: 26GB bilingual legal corpus
-- **Language Ratio**: 60.3% Greek, 39.7% English
-- **Memory Requirements**: Efficient base architecture suitable for production deployment
-- **Inference Speed**: Optimized for both Greek and English legal text processing
-
-## Model Card Authors
-
-[Your Name / Your Organization's Name]
-
-## Citation
-
-If you use this model in your research, please cite it as follows:
-
-```bibtex
-@misc{your_name_2025_themida_roberta_bilingual_26g,
-  author = {[Your Name/Organization]},
-  title = {Themida-RoBERTa Legal 26G: A Bilingual Greek-English Legal Language Model},
-  year = {2025},
-  publisher = {Hugging Face},
-  journal = {Hugging Face Hub},
-  howpublished = {\url{https://huggingface.co/novelcore/themida-roberta-el-en-legal-26G-8-gpu}},
-}
-```
-
-## Acknowledgments
-
-We thank the Greek government institutions and legal organizations for making their legal texts publicly available. We also acknowledge the [Pile of Law](https://huggingface.co/datasets/pile-of-law) dataset contributors for providing comprehensive English legal corpora that enabled this bilingual approach. This work represents a significant step forward in multilingual legal language modeling, combining the rich traditions of Greek civil law and English common law in a single, efficient model.
+- **Total Training Data**: 26GB bilingual corpus