OSTLM-V1: English to Hebrew Translator 🚀

A Neural Machine Translation (NMT) model based on a custom Transformer (Encoder-Decoder) architecture, trained from scratch. This model is designed to translate English sentences into Hebrew using multilingual encoding and specialized layer configurations.

📝 Model Description

OSTLM (Open Source Translation Language Model) demonstrates translation capabilities in a compact and efficient format. Unlike generic pre-trained models, this model was trained specifically on English-Hebrew pairs to capture the nuances of the language transition.

• Architecture: Custom Transformer (6 Encoder layers, 6 Decoder layers).
• Dataset: Trained on the OPUS-100 (en-he) dataset.
• Tokenization: Uses BertTokenizer (multilingual-cased).

🚀 How to Use (Inference)

The model performs best when output length is controlled via inference parameters. This prevents hallucinations and ensures the translation stops at the correct point.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

# Load the model and tokenizer
model_id = "Raziel1234/ostlm"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id, trust_remote_code=True)

def translate(text, max_tokens=10):
    inputs = tokenizer(text, return_tensors="pt", padding=True)
    
    with torch.no_grad():
        outputs = model.generate(
            input_ids=inputs["input_ids"],
            max_new_tokens=max_tokens, # Adjust based on expected Hebrew length
            num_beams=5,
            repetition_penalty=2.5,
            no_repeat_ngram_size=2,
            decoder_start_token_id=tokenizer.cls_token_id,
            eos_token_id=tokenizer.sep_token_id
        )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example Usage
english_text = "Hey! "
print(f"EN: {english_text}")
print(f"HE: {translate(english_text, max_tokens=3)}")