Upload v3_vi2en_transformer model

README.md

@@ -0,0 +1,126 @@
+---
+language:
+- vi
+- en
+tags:
+- translation
+- transformer
+- seq2seq
+license: mit
+datasets:
+- iwslt2015
+metrics:
+- bleu
+---
+
+# v3_vi2en - Vietnamese-English Translation
+
+## Model Description
+
+Vi→En optimized large model with advanced techniques
+
+This model is trained from scratch using the Transformer architecture for machine translation.
+
+### Model Details
+
+- **Language pair**: Vietnamese → English
+- **Architecture**: Transformer (Encoder-Decoder)
+- **Parameters**: 
+  - d_model: 1024
+  - n_heads: 16
+  - n_encoder_layers: 6
+  - n_decoder_layers: 6
+  - d_ff: 4096
+  - dropout: 0.3
+
+### Training Details
+
+- **Optimizer**: ADAMW
+- **Learning Rate**: 0.0001
+- **Batch Size**: 48
+- **Label Smoothing**: 0.1
+- **Scheduler**: cosine
+- **Dataset**: IWSLT 2015 Vi-En
+
+### Performance
+
+
+### Improvements
+
+- Larger model (1024-dim, 16 heads)
+- BPE tokenization
+- Cosine learning rate schedule
+- Mixed precision training
+- Larger beam search (10)
+- Longer sequences (256)
+
+## Usage
+
+```python
+# Load model and translate
+from src.models.transformer import Transformer
+from src.inference.translator import Translator
+from src.data.vocabulary import Vocabulary
+import torch
+
+# Load vocabularies
+src_vocab = Vocabulary.load('src_vocab.json')
+tgt_vocab = Vocabulary.load('tgt_vocab.json')
+
+# Load model
+model = Transformer(
+    src_vocab_size=len(src_vocab),
+    tgt_vocab_size=len(tgt_vocab),
+    d_model=512,
+    n_heads=8,
+    n_encoder_layers=6,
+    n_decoder_layers=6,
+    d_ff=2048,
+    dropout=0.1,
+    max_seq_length=512,
+    pad_idx=0
+)
+
+checkpoint = torch.load('best_model.pt')
+model.load_state_dict(checkpoint['model_state_dict'])
+
+# Create translator
+translator = Translator(
+    model=model,
+    src_vocab=src_vocab,
+    tgt_vocab=tgt_vocab,
+    device='cuda',
+    decoding_method='beam',
+    beam_size=5
+)
+
+# Translate
+vietnamese_text = "Xin chào, bạn khỏe không?"
+translation = translator.translate(vietnamese_text)
+print(translation)
+```
+
+## Training Data
+
+- **Dataset**: IWSLT 2015 Vietnamese-English parallel corpus
+- **Training pairs**: ~500,000 sentence pairs
+- **Validation pairs**: ~50,000 sentence pairs
+- **Test pairs**: ~3,000 sentence pairs
+
+## Limitations
+
+- Trained specifically for Vietnamese to English translation
+- Performance may vary on out-of-domain text
+- Medical/technical domains may require fine-tuning
+
+## Citation
+
+```bibtex
+@misc{nlp-transformer-mt,
+  author = {MothMalone},
+  title = {Transformer Machine Translation Vi-En},
+  year = {2025},
+  publisher = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/MothMalone}}
+}
+```

best_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf5d084d0f5e95a3110c4c9ac57905cc477f8abd93c3fb067824eeb9a1a4b17f
+size 3347137710

config.yaml

@@ -0,0 +1,88 @@
+# Version 3: Vi→En Optimized Transformer
+# Larger model with advanced techniques
+
+# Data Configuration
+data:
+  src_lang: "vi"
+  tgt_lang: "en"
+  train_src: "data/raw_opus100/train.vi.txt"
+  train_tgt: "data/raw_opus100/train.en.txt"
+  # Validation files (if not exist, will auto-split from training)
+  val_src: "data/raw_opus100/val.vi.txt"
+  val_tgt: "data/raw_opus100/val.en.txt"
+  val_split: 0.1 # 10% of training data for validation if val files don't exist
+  test_src: "data/raw_opus100/public_test.vi.txt"
+  test_tgt: "data/raw_opus100/public_test.en.txt"
+  max_seq_length: 256 # Increased
+
+# Vocabulary - BPE tokenization
+vocab:
+  src_vocab_size: 50000 # Larger vocabulary
+  tgt_vocab_size: 50000
+  min_freq: 1
+  tokenization: "bpe" # Use BPE instead of word-level
+
+# Model - Larger (Transformer Big config)
+model:
+  d_model: 1024 # Increased
+  n_heads: 16 # Increased
+  n_encoder_layers: 6
+  n_decoder_layers: 6
+  d_ff: 4096 # Increased
+  dropout: 0.3 # Higher dropout for regularization
+  max_seq_length: 512
+
+# Training - Advanced
+training:
+  batch_size: 48 # Smaller due to larger model
+  epochs: 10 # Good balance of quality and time
+  optimizer: "adamw"
+  learning_rate: 0.0001
+  weight_decay: 0.01
+  betas: [0.9, 0.98]
+  scheduler: "cosine" # Cosine annealing
+  warmup_steps: 8000 # Longer warmup
+  label_smoothing: 0.1
+  gradient_accumulation_steps: 8 # Effective batch size = 64
+  max_grad_norm: 1.0
+
+  # Mixed precision training (if available)
+  use_amp: true
+  use_wandb: true # Enable Weights & Biases logging
+
+  save_every: 20000 # Save less frequently to save disk space
+  eval_every: 100
+  log_every: 100
+  early_stopping_patience: 10
+
+# Inference
+inference:
+  beam_size: 10 # Larger beam
+  max_decode_length: 256
+  length_penalty: 0.8
+
+# Paths
+paths:
+  checkpoint_dir: "experiments/v3_vi2en/checkpoints"
+  log_dir: "experiments/v3_vi2en/logs"
+  vocab_dir: "data/vocab_v3_vi2en"
+
+device: "cuda"
+seed: 42
+
+# Weights & Biases
+wandb:
+  project: "nlp-transformer-mt"
+  entity: null # Your wandb username (optional)
+
+# Version info
+version:
+  name: "v3_vi2en"
+  description: "Vi→En optimized large model with advanced techniques"
+  improvements:
+    - "Larger model (1024-dim, 16 heads)"
+    - "BPE tokenization"
+    - "Cosine learning rate schedule"
+    - "Mixed precision training"
+    - "Larger beam search (10)"
+    - "Longer sequences (256)"

requirements.txt

@@ -0,0 +1,4 @@
+torch>=2.0.0
+numpy>=1.21.0
+pyyaml>=6.0
+tqdm>=4.65.0