dekthedev/opus-mt-en-vi-ct2-int8

opus-mt-en-vi — CTranslate2 int8

Quantized by @dekthedev.

CTranslate2 int8 conversion of Helsinki-NLP/opus-mt-en-vi. Optimized for CPU inference, especially ARM64 (Cortex-A series).

Model info

Property	Value
Architecture	MarianMT (Transformer encoder-decoder)
Parameters	74M
Direction	English → Vietnamese
Quantization	CTranslate2 int8
Disk size	~145 MB
BLEU (Tatoeba EN-VI)	37.2
Speed on ARM64 (est.)	300–500 tok/s

Files

File	Description
model.bin	CTranslate2 int8 weights
config.json	CTranslate2 model config
tokenizer_config.json	MarianTokenizer config
vocab.json	Vocabulary
source.spm	SentencePiece source model
target.spm	SentencePiece target model

Usage

import ctranslate2
from transformers import MarianTokenizer
from huggingface_hub import snapshot_download

model_dir  = snapshot_download("dekthedev/opus-mt-en-vi-ct2-int8")
translator = ctranslate2.Translator(model_dir, device="cpu")
tokenizer  = MarianTokenizer.from_pretrained(model_dir)

def translate(text: str) -> str:
    encoded    = tokenizer([text], return_tensors=None, padding=False)
    src_tokens = [tokenizer.convert_ids_to_tokens(ids) for ids in encoded["input_ids"]]
    results    = translator.translate_batch(src_tokens)
    tgt_tokens = results[0].hypotheses[0]
    tgt_ids    = tokenizer.convert_tokens_to_ids(tgt_tokens)
    return tokenizer.decode(tgt_ids, skip_special_tokens=True)

print(translate("Hello, how are you?"))
# → Xin chào, bạn có khỏe không?