Upload transformer-vi2en-v2 model

README.md

@@ -0,0 +1,125 @@
+---
+language:
+- vi
+- en
+tags:
+- translation
+- transformer
+- seq2seq
+license: mit
+datasets:
+- iwslt2015
+metrics:
+- bleu
+---
+
+# v2_vi2en - Vietnamese-English Translation
+
+## Model Description
+
+Improved Vi→En training with label smoothing and AdamW
+
+This model is trained from scratch using the Transformer architecture for machine translation.
+
+### Model Details
+
+- **Language pair**: Vietnamese → English
+- **Architecture**: Transformer (Encoder-Decoder)
+- **Parameters**: 
+  - d_model: 512
+  - n_heads: 8
+  - n_encoder_layers: 6
+  - n_decoder_layers: 6
+  - d_ff: 2048
+  - dropout: 0.1
+
+### Training Details
+
+- **Optimizer**: ADAMW
+- **Learning Rate**: 0.0001
+- **Batch Size**: 32
+- **Label Smoothing**: 0.1
+- **Scheduler**: warmup
+- **Dataset**: IWSLT 2015 Vi-En
+
+### Performance
+
+
+### Improvements
+
+- Label smoothing (0.1)
+- AdamW optimizer with weight decay
+- Beam search (size=5)
+- Gradient accumulation
+- Early stopping
+
+## Usage
+
+```python
+# Load model and translate
+from src.models.transformer import Transformer
+from src.inference.translator import Translator
+from src.data.vocabulary import Vocabulary
+import torch
+
+# Load vocabularies
+src_vocab = Vocabulary.load('src_vocab.json')
+tgt_vocab = Vocabulary.load('tgt_vocab.json')
+
+# Load model
+model = Transformer(
+    src_vocab_size=len(src_vocab),
+    tgt_vocab_size=len(tgt_vocab),
+    d_model=512,
+    n_heads=8,
+    n_encoder_layers=6,
+    n_decoder_layers=6,
+    d_ff=2048,
+    dropout=0.1,
+    max_seq_length=512,
+    pad_idx=0
+)
+
+checkpoint = torch.load('best_model.pt')
+model.load_state_dict(checkpoint['model_state_dict'])
+
+# Create translator
+translator = Translator(
+    model=model,
+    src_vocab=src_vocab,
+    tgt_vocab=tgt_vocab,
+    device='cuda',
+    decoding_method='beam',
+    beam_size=5
+)
+
+# Translate
+vietnamese_text = "Xin chào, bạn khỏe không?"
+translation = translator.translate(vietnamese_text)
+print(translation)
+```
+
+## Training Data
+
+- **Dataset**: IWSLT 2015 Vietnamese-English parallel corpus
+- **Training pairs**: ~500,000 sentence pairs
+- **Validation pairs**: ~50,000 sentence pairs
+- **Test pairs**: ~3,000 sentence pairs
+
+## Limitations
+
+- Trained specifically for Vietnamese to English translation
+- Performance may vary on out-of-domain text
+- Medical/technical domains may require fine-tuning
+
+## Citation
+
+```bibtex
+@misc{nlp-transformer-mt,
+  author = {MothMalone},
+  title = {Transformer Machine Translation Vi-En},
+  year = {2025},
+  publisher = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/MothMalone}}
+}
+```

best_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b3688b771169052d1bc9937c7bd3a218b2ac7a191094ce9f5ff3423e7a72239
+size 1022502766

config.yaml

@@ -0,0 +1,82 @@
+# Version 2: Improved Transformer
+# With label smoothing, better optimizer, and beam search
+
+# Data Configuration
+data:
+  src_lang: "vi"
+  tgt_lang: "en"
+  train_src: "data/raw_opus100/train.vi.txt"
+  train_tgt: "data/raw_opus100/train.en.txt"
+  # Validation files (if not exist, will auto-split from training)
+  val_src: "data/raw_opus100/val.vi.txt"
+  val_tgt: "data/raw_opus100/val.en.txt"
+  val_split: 0.1  # 10% of training data for validation if val files don't exist
+  test_src: "data/raw_opus100/public_test.vi.txt"
+  test_tgt: "data/raw_opus100/public_test.en.txt"
+  max_seq_length: 128
+
+# Vocabulary
+vocab:
+  src_vocab_size: 40000  # Increased for better Vietnamese coverage
+  tgt_vocab_size: 40000
+  min_freq: 2
+
+# Model - Same architecture
+model:
+  d_model: 512
+  n_heads: 8
+  n_encoder_layers: 6
+  n_decoder_layers: 6
+  d_ff: 2048
+  dropout: 0.1
+  max_seq_length: 512
+
+# Training - Improved
+training:
+  batch_size: 32
+  epochs: 8  # Good balance of quality and time
+  optimizer: "adamw"  # Changed to AdamW
+  learning_rate: 0.0001
+  weight_decay: 0.01  # Added weight decay
+  scheduler: "warmup"
+  warmup_steps: 4000
+  label_smoothing: 0.1  # Added label smoothing
+  gradient_accumulation_steps: 2  # Effective batch size = 64
+  max_grad_norm: 1.0
+  
+  use_wandb: true
+  save_every: 1000
+  eval_every: 500
+  log_every: 100
+  early_stopping_patience: 5
+
+# Inference
+inference:
+  beam_size: 5  # Beam search
+  max_decode_length: 128
+  length_penalty: 0.6
+
+# Paths
+paths:
+  checkpoint_dir: "experiments/v2_vi2en/checkpoints"
+  log_dir: "experiments/v2_vi2en/logs"
+  vocab_dir: "data/vocab_v2_vi2en"
+
+device: "cuda"
+seed: 42
+
+# Weights & Biases
+wandb:
+  project: "nlp-transformer-mt"
+  entity: null
+
+# Version info
+version:
+  name: "v2_vi2en"
+  description: "Improved Vi→En training with label smoothing and AdamW"
+  improvements:
+    - "Label smoothing (0.1)"
+    - "AdamW optimizer with weight decay"
+    - "Beam search (size=5)"
+    - "Gradient accumulation"
+    - "Early stopping"

requirements.txt

@@ -0,0 +1,4 @@
+torch>=2.0.0
+numpy>=1.21.0
+pyyaml>=6.0
+tqdm>=4.65.0